Skip to main content
William & Ida Friday Institute for Educational Innovation

Distinguished Leadership in Practice (DLP): First Annual RttT Evaluation Report

Executive Summary

Providing high-quality, accessible professional development to all teachers and principals is a critical component of the professional development plan funded by North Carolina’s federal Race to the Top (RttT) grant. One key professional development program funded through RttT focuses on providing professional development for practicing principals. The North Carolina Department of Public Instruction (NCDPI) has partnered with the North Carolina Principals and Assistant Principals’ Association (NCPAPA) to provide a leadership development program for practicing school principals. This professional development model, entitled Distinguished Leadership in Practice (DLP), is aligned to the performance evaluation standards adopted by the State Board of Education for North Carolina’s school leaders (i.e., the North Carolina Standards for School Executives).

Overview of NC RttT DLP Activities

The DLP initiative employs a non-traditional professional development model. Participants examine the meaning and application of school leadership through a problem-based approach delivered via a series of face-to-face, regional, cohort-based sessions, which are followed by online activities. Throughout the year-long experience, practicing North Carolina principals are coached using a continuous improvement model. Participating principals are provided with models of exemplary school leadership, which allows them to study the behaviors, attitudes, and competencies that define a distinguished school leader. The DLP experience is built around six components:

  • Component One: Strategic Leadership for High-Performing Schools
  • Component Two: Maximizing Human Resources for Goal Accomplishment
  • Component Three: Building a Collaborative Culture through Distributive Leadership
  • Component Four: Improving Teaching and Learning for High Performance
  • Component Five: Creating a Strong Internal and External Stakeholder Focus
  • Component Six: Leading Change to Drive Continuous Improvement

Overview of NC RttT DLP Evaluation Activities

North Carolina’s RttT proposal included a commitment to independent evaluations of each initiative. Over the next three years, the RttT Evaluation Team will document the DLP activities and collect data about participation in, satisfaction with, and the impact of DLP professional development activities through surveys and focus groups with DLP participants and facilitators, as well as analysis of longitudinal education data on students, teachers, leaders, and schools. The purpose of this evaluation is to provide detailed information about the implementation and impact of this professional development effort that targets practicing principals. This evaluation study is one part of a larger effort to evaluate the implementation and impact of NC’s RttT professional development initiatives in order to determine if the initiatives as implemented have led to the intended outcomes with respect to school leader practice, the culture and climate of achievement at those leaders’ schools, and, potentially, teacher and student performance.

The questions for the DLP evaluation fall into seven categories and are aligned with the overarching evaluation questions for RttT professional development.

  1. Program Description: How is the DLP initiative operationalized and implemented?
  2. Participation: To what extent does DLP reach the intended participants?
  3. Program Quality: To what extent does the DLP program meet standards of high-quality professional development?
  4. Short-Term Outcomes: To what extent did participants acquire intended knowledge and skills as a result of their participation in DLP?
  5. Intermediate Outcomes: What was the impact of DLP on participants’ practice?
  6. Long-Term Outcomes: What was the impact of the principals’ participation in DLP on their schools’ culture/climate of achievement?
  7. Distal Outcome: To what extent are gains in student performance outcomes associated with principals’ participation in DLP?

This report addresses questions I through IV (program description, participation, program quality, and short-term outcomes), and it also provides some initial information related to question V (intermediate outcomes). Questions VI and VII (long-term and distal outcomes) will be addressed in future evaluation reports.

Evaluation Findings

  1. Program Description: The DLP program employs a non-traditional professional development model that allows participants to examine critically the meaning and application of school leadership through a problem-based, real-world approach. This cohort-based, experiential program is delivered over a one-year period using a blended model of face-to-face sessions supplemented by online sessions. Sessions are facilitated by 20 highly-qualified individuals who are former or current principals.
  2. Participation: DLP sessions were conducted in four regions (Central, Northeast, Southeast, and West). The program began with 194 principals participating across the regions, 157 of whom completed all six components. This participation level was in line with the proposed target of serving 200 principals annually. Data from participants’ applications indicate that participants come from a variety of backgrounds and school contexts, and that they are representative of principals around the state, based on Teacher Working Conditions Survey data.
  3. Program Quality: The DLP program components most closely align with the RttT focus on updating the education workforce, in that DLP’s goal is to help principals progress professionally, as measured by the North Carolina Standards for School Executives. Nearly all of the participants (96%) agreed or strongly agreed that both the face-to-face sessions and the DLP program as a whole were of high quality overall; a high percentage (91%) of participants also agreed or strongly agreed that the online sessions were of high quality. The observational data provided converging evidence of the overall quality of the DLP program. Participants reported overwhelmingly positive reactions to DLP; they enjoyed the experience, and they found it was well worth the significant time commitment.
  4. Short-Term Outcomes: Almost all participants (95% to 100%, depending on the objective) indicated they were satisfied or very satisfied that the specific learning objectives had been accomplished, and only a small fraction of respondents indicated that they were somewhat satisfied or not satisfied. Results from the participant survey show that most of the principals agreed or strongly agreed that they developed specific knowledge (87% to 95%) and skills (86% to 98%) targeted by DLP. Focus group results also provide evidence that participants acquired knowledge and skills—from the facilitators as well as from each other—that will help them become better leaders.
  5. Intermediate Outcomes: The results were overwhelmingly positive, with at least 94% of respondents agreeing or strongly agreeing that they had applied the knowledge and skills learned in DLP in ways that reflect progress along the NC Standards for School Executives.

Recommendations

As detailed in this report, the data show that the DLP team has designed and implemented a high-quality program that meets the professional development needs of the participating school leaders. This level of quality reflects the DLP team’s commitment to a continuous improvement process. To continue to strengthen the program, data collected for this report suggest that the DLP team should:

  • Differentiate Activities: Feedback from participants suggested the need for further differentiation of activities based on participants’ years of experience and types of experience. A pre-DLP needs assessment survey may help clarify those differentiation needs.
  • Provide Opportunities for Participant Leadership: Participants could be assigned to lead group discussions or give formal presentations on short segments of material or about their areas of expertise. Small groups of participants also could present to each other after working on a collaborative problem-solving project in face-to-face or online sessions.
  • Increase Time for Collaboration and Networking: A reoccurring theme across all participant data sources was the value of providing time for networking and collaboration. Participants would have liked even more time to share experiences and collaborate to solve shared problems.
  • Improve Quality of Feedback in the Online Sessions: Data from participants and program observers suggest that there is room for improvement in the quantity, quality, and consistency of feedback that participants receive in the online sessions.
  • Increase Variety of Activities and Use of Technology Tools: A review of the online sessions indicated that the variety of activities was inconsistent—some sessions offered good variety, while others consisted almost entirely of asynchronous, text-based activities (e.g., reading a document, writing a response, and replying to a peer). Tools that can be integrated include, but are not limited to: wikis, video-making tools, audio editing tools, data visualization tools, simulations, synchronous interaction platforms, blogs, survey tools, and mind mapping tools.
  • Improve Data Collection Instruments: The current instruments used by DLP leadership have several limitations in terms of their length, the prompt-item-response option alignment, item wording, and the response options provided. The Evaluation Team recommends that NCPAPA staff and DLP leadership collaborate with the Team to develop, implement, and analyze all instruments related to DLP professional development activities.

Next Steps for the DLP Evaluation

Data on the long-term and distal outcomes of the DLP program are not yet available. However, over the course of the RttT grant period (through 2014), the Evaluation Team will seek to assess the impact the program has on the culture and climate of achievement, as well as on student performance, at participating principals’ schools. While student outcomes will be the primary focus, the report also will examine the impact on school culture and climate, including teacher working conditions. The evaluation also will benefit from surveying participants some time after they have completed the program, which may allow them to better report on how they applied what they learned, as well as on any related issues they encountered.

View Resource

Full Report PDF Full Report PDF

Projects

Evaluation of Race to the Top

This evaluation was designed to provide formative feedback for program improvement and determine impact on the target goals of each initiative and on overall state-level outcome goals.

Published

September 1, 2012

Resource Type

Report

Published By

Consortium for Educational Research and Evaluation–North Carolina