Executive Summary

The North Carolina Race to the Top (RttT) Professional Development Initiative is an expansive and multi-faceted effort to increase student achievement by updating the knowledge and skills of the state’s public education workforce, which includes more than 100,000 teachers and 2,400 principals. This initiative is driven by a host of substantial policy changes, including: new curriculum standards and student evaluations; revised educator evaluation processes; increased emphasis on formative assessment to inform instructional decisions; rapid changes in technologies and digital resources for teaching and learning; and a heightened emphasis on improving college and career readiness for all student groups. The challenge is formidable.

Four general questions guide the evaluation, organized in the sequence in which they could be addressed over the course of the RttT grant. The first two address statewide efforts while the second two address local impact in Local Education Agencies (LEAs), schools, and classrooms:

1. State Strategies: To what extent did the State implement and support the proposed RttT professional development efforts?
2. Short-Term Outcomes: What were the direct outcomes of state-level RttT professional development efforts?
3. Intermediate Outcomes: To what extent did RttT professional development efforts successfully update the North Carolina education workforce?
4. Impacts on Student Performance: To what extent are gains in student performance outcomes associated with RttT professional development?

The RttT Evaluation Team has documented the State’s RttT professional development efforts since the beginning of the grant.^[1] This Year 4 report focuses primarily on Evaluation Question 3, the intermediate outcomes of updating the education workforce. In addition, this report documents the direct outcomes of the State’s RttT professional development efforts for the 2013-14 school year and describes trends over the four-year course of the evaluation. This report is informed by multiple data sources, including observations, surveys, interviews, and, for the online components, evaluation rubrics and site analytics. Although RttT professional development activities are ongoing, due to reporting timelines, data for this Year 4 report were not collected after February 2014.

The overall RttT plan includes additional professional development activities that are housed under other RttT-funded initiatives that address specific groups of educators: principals; educators in the lowest-achieving schools; educators in selected STEM schools; and new teachers entering low-performing schools. The evaluations of these activities are detailed in separate evaluation reports for those RttT initiatives.^[2]

A separate report on the overall impact of RttT (to be completed in mid-2015) will assess the impacts of the highest-priority initiatives, including professional development, and will include an analysis of impact on student learning (Evaluation Question 4).

Summary of the Findings

Overall, the State has fulfilled the requirements of the RttT professional development plan: tens of thousands of educators have been reached; high-priority needs have been addressed; the statewide professional development opportunities have been well received by participants; and LEA professional development capacity has increased. As a result of the RttT professional development programs, North Carolina educators are more knowledgeable about the new curriculum standards, student assessments, data systems, educator evaluations, technologies, and overall expectations. They are beginning to integrate these changes into their day-to-day professional practices. In addition, progress has been made toward developing local leaders’ professional development capacity, built on strong LEA and regional collaborative networks.

This evaluation report highlights a number of summative findings across the four years of RttT-supported professional development programs that can help inform future programmatic and funding decisions.

Preparation of LEA and School Leadership to Provide Local Professional Development

The North Carolina Department of Public Instruction (NCDPI) focused on preparing LEA and school leadership teams to provide effective local professional development on high-priority topics, and on providing resources to help support local programs. Activities each year centered on large-scale Summer Institutes in six locations across the state, followed by regional NCDPI-Regional Education Services Agency (RESA) sessions for LEA and charter school leadership teams, conducted through a collaboration between NCDPI and each of the eight RESAs. Supporting resources were provided online through: self-paced professional development modules; cohort-based instructor-led online modules; webinars; and wiki and chat sites.

In the first three years, state-level professional development focused on the transition to new curriculum standards across all grades levels and content areas. This focus continued in the fourth year, and offerings were expanded to include four other priority areas: implementation of formative and summative assessments; use of data to support instruction; effective utilization of the North Carolina Educator Evaluation System; and use of technology for teaching and learning.

As in prior years, a substantial number of school and LEA leaders participated^[3] in the 2013 Summer Institutes (2,349) and the NCDPI-RESA sessions (5,105). Many educators received certificates of completion for at least one online module (20,413) and participated in webinars (approximately 3,000 officially registered, with many more participating as part of shared registrations via professional learning teams).

Surveys asked participants whether they agreed with statements about the usefulness and quality of these sessions. The NCDPI-RESA sessions were rated most highly, with 94% of participants agreeing that their professional development needs were met. The Summer Institutes also were rated well, with 88% agreeing that their needs were met. When asked to rate specific outcomes, the NCDPI-RESA session were very highly rated, with at least 90% of participants agreeing that these sessions led to positive learning and impacted their professional practices for most outcomes. Comparable data was generally in the 80%-89% range for the Summer Institutes, though a few outcomes (e.g., understanding of teacher evaluation and school accountability) were much lower (in the 60%-65% range). For the online resources, 91% of webinar participants and 78% of module participants agreed that their professional development needs were met. The ratings of online modules may reflect whether they were completed as recommended—that is, as part of professional learning teams that discussed the content—or by individual educators on their own. Observational, interview, and content review data were consistent with these findings.

Overall, these opportunities and resources were valuable to the many participating educators, with the NCDPI-RESA sessions receiving the highest ratings by participants. The fact that different needs and content were addressed across the different professional development offerings should be considered when interpreting these results.

Impact on Local Professional Development Programs

Evaluation Question 3 focused on the professional development offerings provided by individual LEAs, looking specifically at whether there were: (a) increases in local capacity to provide high-quality professional development; (b) a focus on supporting RttT priorities; and (c) positive changes in local professional development instructional practices.

LEA Professional Development Coordinators reported that since RttT began their LEAs have used data from a diverse range of decision-making tools to plan professional development that would meet the needs of teachers. At the school level, Professional Learning Communities (PLCs) were the most consistently referenced approach to translating professional development into instructional practice. Many also emphasized the use of virtual resources and multi-team collaborations. In addition, principals highlighted their more frequent use of School Improvement Teams, teacher leaders, and LEA-level support staff to help set the direction for professional development opportunities. Coordinators and principals mentioned that they have refined their local strategic plans to incorporate the RttT priorities. They also reported that the RESAs played a vital role in implementing regional professional development that fostered relationships among LEA and school leadership.

LEA Professional Development Coordinators emphasized the positive impact on their LEAs of having RttT funding specifically dedicated to professional development. RttT resources allowed LEAs the flexibility to use a teacher-focused approach, to make professional development activities more interactive and hands-on, and to provide teachers with access to local experts. However, while RttT funding has helped LEAs and schools to incorporate RttT goals into their local goals, Professional Development Coordinators and principals expressed uncertainty about how their LEAs and schools would pay for professional development efforts in the future.

Impact on Updating the Education Workforce

Significant efforts have been made to provide local professional development on the RttT priorities. While there certainly has been progress, a great deal remains to be done, as many teachers reported that they were not yet well-prepared to address all the changes expected of them. In addition, teachers and local Professional Development Coordinators had very different views of the local capacity and quality of professional development offerings. Some specific findings are summarized below:

1. Many LEAs are using blended approaches that combine face-to-face sessions, online modules, and PLC activities. While PLCs are used differently across LEAs, they generally serve as the main support for impacting teachers’ instructional practices.
2. Educators across leadership levels described a positive change from prescriptive to more collaborative, hands-on, and practice-focused professional development approaches.
3. As in prior years, consistent disconnects remain between teachers’ and Professional Development Coordinators’ views of local professional development. Both groups were asked whether they agreed with each of 16 statements about LEA capacity and processes to provide high-quality professional development (e.g., being able to select, plan, and implement high-quality professional development, using teacher evaluations and student data to inform professional development plans). In response, teachers were far less positive than were coordinators, with between 83% and 97% agreement by coordinators across items assessing current capacity but only between 58% and 83% agreement by teachers. Teachers also were far less positive than coordinators about the impact of professional development on their knowledge and skills in specific areas. For example, 93% of LEA Professional Development Coordinators agreed that local professional development increased teacher knowledge and skills in deepening students’ content knowledge, whereas only 61% of teachers agreed.
4. While about 60% of surveyed teachers in 2013-14 reported that their professional development experiences were both sustained and coherent, there was little change in teacher perceptions of the quality of local professional development programs between 2011 and 2014.
5. While the large majority of teachers (77% in 2013-14) reported that they felt well-prepared to help their students meet the new state curriculum standards, this percentage has declined since 2011-12 (88%). This decline may be due to a combination of an influx of new teachers and of teachers becoming more aware of challenges as they implement the standards. For example, teacher self-reports and classroom observations both show that teachers need further support in implementing the specific instructional practices recommended in the new standards.
6. Focus group data from 2013-14 revealed that teachers struggled with increased pressure on their performance given the emphasis on student outcome measures and their use in teacher evaluations. In addition, many felt that the evaluation process was arbitrary and lacked the appropriate attention to constructive criticism needed for improvement in instruction. Teachers also discussed the lack of time to engage in effective professional development and collaborations, and the inadequate compensation for the increase in effort that has been required. These findings are in line with qualitative results that suggest an overall decrease in teacher morale and confidence.

Conclusion

Overall, the results of this evaluation show that much has been accomplished, but also that professional development efforts need to continue and that the LEAs need additional resources and supports to ensure that effective professional development—focused on priority areas and supported to meet local needs—continues to be available throughout the state.

¹ Executive summaries of the formative evaluations from the Year 1, Year 2, and Year 3 reports are provided in Appendix B of the full report.
² https://fi.ncsu.edu/projects/evaluation-of-race-to-the-top/
³ Participant numbers do not include non-LEA-affiliated attendees (e.g., NCDPI staff, volunteers, IHE representatives, etc.); total including those attendees was 2,962.

View Resource

Full Report PDF

Projects

Evaluation of Race to the Top

This evaluation was designed to provide formative feedback for program improvement and determine impact on the target goals of each initiative and on overall state-level outcome goals.

Published

March 1, 2015

Resource Type

Report

Published By

Consortium for Educational Research and Evaluation–North Carolina