Skip to main content
William & Ida Friday Institute for Educational Innovation

RttT PD Evaluation: Year 2, Part I—Statewide Face-to-Face PD

Executive Summary

The North Carolina Race to the Top (RttT) professional development plan is an expansive and multi-faceted effort to increase student achievement by updating the knowledge and skills of the state’s entire public education workforce. This initiative is driven by a host of recent changes, including: adoption of new Common Core State Standards and North Carolina Essential Standards; increased use of data to inform classroom and school decisions; rapid changes in the technologies and digital resources available for teaching and learning; new teacher and administrator evaluation processes; and an increased emphasis on formative assessment to inform instructional decisions.

The human resources challenge of the initiative—to provide the state’s 100,000 teachers and 2,400 principals with professional development that will enable them to extend their knowledge, improve professional practices, and, ultimately, increase student achievement overall and close achievement gaps among student groups—is formidable. The timeframe (the four-year period of the grant), diversity of the State (from large metropolitan local education agencies [LEAs] to small, rural, and resource-limited LEAs, many of which continue to struggle under the weight of fiscal constraints), and expectations (to create a statewide professional development infrastructure that can be sustained after RttT funding ends) only increase that challenge. The RttT professional development evaluation is being conducted in full recognition of these circumstances, as well as of the deep commitment of the members of the RttT Professional Development Implementation Team. The intent of the evaluation is to provide data-driven information that can support reflection about and improvement of this effort throughout the RttT grant process.

Four general questions guide the evaluation:

  1. State Strategies: To what extent did the state implement and support proposed RttT professional development efforts?
  2. Short-Term Outcomes: What were direct outcomes of State-level RttT professional development efforts?
  3. Intermediate Outcomes: To what extent did RttT professional development efforts successfully update the NC education workforce?
  4. Impacts on Student Performance: To what extent are gains in student performance outcomes associated with RttT professional development?

The Evaluation Team is providing this second annual assessment of progress in three separate but related reports. This report—North Carolina’s RttT Statewide Face-to-Face Professional Development Formative Evaluation—documents the current status of the state’s RttT face-to-face professional development efforts related to the Annual Professional Development Cycle (described below) and addresses specific questions under Evaluation Question 1 (State Strategies) and Evaluation Question 2 (Short-Term Outcomes). The remaining reports address (1) LEA and school-level outcomes of statewide face-to-face RttT professional development efforts, and (2) implementation and impact of the State’s Online Professional Development efforts.

Overview of Annual Professional Development Cycle

The overall plan for RttT professional development is built around annual cycles that are comprised of Summer Institutes, formative support for LEA and charter school Professional Development Leadership Teams, and additional face-to-face support sessions provided by the NCDPI RttT Professional Development Implementation Team in collaboration with the Regional Education Service Agencies (RESAs):

  • 2012 Summer Institutes. The 2012 Summer Institutes—themed “Connecting to Serve All Learners”—were two-day, face-to-face institutes for LEA and charter school Professional Development Leaders. A collaborative effort of staff across NCDPI divisions, these six Institutes offered 102 hours of professional development to 2,541 attendees, with a goal of preparing the local Professional Development Leaders to design, develop, and implement local-level professional development to help teachers transition to the new standards.
  • NCDPI Intra-Agency Collaboration for Support of RttT Professional Development Efforts. Staff across five NCDPI agencies collaborated to develop and deploy ongoing professional development opportunities to educators statewide. In addition to working together on RttT professional development efforts, NCDPI staff also participated in an ongoing internal capacity-building effort (the North Carolina Learning Technology Initiative [NCLTI]).
  • NCDPI-RESA Sessions. For the 2011-12school year, NCDPI and the RESAs collaborated to provide 966 hours of training to 3,646 attendees across all eight regions on: Common Core State Standards; Fidelity Support; Technical Assistance; North Carolina Professional Teaching Standards; Teacher Effectiveness/New Accountability Model; and Principal Training for Common Core and Essential Standards.

Summary of Major Findings

Cross-Cutting Findings

  • Overall Quality. Data indicate that the overall quality of the Annual Professional Development Cycle events for the 2011-12 school year was high, with some variation across session type. Evaluators, participants, and facilitators all recognized various strengths of the professional development events but also recommended directions for improvements.
  • Needs Assessments. Professional development needs—identified in NCDPI-RESA Fidelity Support sessions, pre-Institute surveys, and data and findings from the First Annual Evaluation Report for RttT Professional Development—are clear; providing adequate time and resources to meet these needs remains a challenge for both the State and LEAs.
  • Alignment with RttT Priorities. State-level professional development efforts provided a total of 938 hours of professional development during the Annual Professional Development Cycle related to the four RttT priorities: (1) successful transition to the new standards (573 hours); (2) implementation of formative and summative assessments (146 hours); (3) use of data to support instruction (24 hours); and (4) effective utilization of the North Carolina Educator Evaluation System (195 hours). As this was the first full academic year of state-level professional development, efforts were focused on the transition to new standards and assessments, which was consistent with the proposal plan. An increased focus on the use of data to support instruction is anticipated in the next annual cycle.
  • Participant Knowledge and Skills. Participants’ assessments of both the NCDPI-RESA sessions and the Summer Institutes indicated a general belief that these events helped educators develop knowledge and skills around the new standards and some of the new statewide models for assessment and accountability.
  • Characteristics of Participants. Participating teachers, school administrators, and central office staff at the NCDPI-RESA events and Summer Institutes represented every grade level, curriculum area, and LEA in the state. Most participants also had a wealth of experience to draw on, with a large majority having more than ten years of experience in education.

Findings Related to 2011-12 NCDPI-RESA Sessions

  • Overall Quality. A high proportion of participants surveyed agreed or strongly agreed that the events incorporated traits of high-quality professional development, including clear objectives (85%), relevance (83%), and opportunities for participants to share knowledge (83%). Most respondents (80-85%) also agreed or strongly agreed that the online resources used were accessible and were effectively employed by the facilitators.
  • Additional data reveal that participants and observers were most positive about the quality of the content and materials presented in the Professional Standards Training for Principals and Assistant Principals, Detailed Scope of Work (DSW), and Content Support sessions, as well as about the facilitation of the Fidelity Support sessions. Participants rated the Principal Training for Common Core and Essential Standards lowest, and adequate opportunities for participants to share knowledge, experiences, and insights were observed the least often in the Teacher Effectiveness sessions.

Findings Related to 2012 Summer Institutes

  • Institute Quality. Most surveyed participants agreed or strongly agreed that the Summer Institute had clear objectives (94%), was relevant to their professional development needs (90%), was well structured (90%), was of high quality overall (88%), built upon previous professional development efforts (87%), was engaging (86%), and met their expectations (85%). Most (90-95%) also noted the effective integration of online resources.

    In participants’ ratings of specific session content quality, a majority gave the Content sessions (56%) and Facilitative Team Time (58%) sessions excellent ratings. Only about one-third of participants gave an excellent rating to the content of the Leaders with Leaders, Understanding Assessments and Accountability, and Listening Lunches sessions.

    Of all the strands, K-5 Mathematics received the highest percentage of “excellent” ratings for content from participants (82%), followed by English Language Development (77%), Guidance (74%), and Arts Education (73%). K-5 Mathematics also received the highest percentage of “excellent” ratings for facilitation from participants (88%), followed again by Arts Education (83%), English Language Development (78%), and Guidance (77%). The lowest percentage of “excellent” content ratings (39-42%) were given for the K-5 Science, 6-12 Social Studies, 6-12 English Language Arts, Information and Technology Standards, and Media strands. Participants assigned the highest percentage of “poor/fair” ratings for facilitation to many of these same strands (K-5 Science [15%], K-5 English Language Arts [14%], and 6-12 Social Studies [13%]).

    Overall, observers found that sessions exhibited characteristics of accomplished, effective professional development, though there was some variation by session type, with Content Strand and Facilitative Team Time sessions rated the most positively, and Assessment and Accountability and Leaders with Leaders sessions rated at lower levels.
  • Participant Application of Skills. A majority of participants agreed or strongly agreed that the RttT-related professional development events would help their LEAs develop, refine, and implement their transition to the new standards. Large majorities (85-92%) also agreed or strongly agreed that the Summer Institute would help them develop skills aligned to the North Carolina Professional Teaching and North Carolina School Executive Standards. Participants were less confident that the Summer Institutes would help them develop rigorous and engaging assessments for the new Common Core and Essential Standards.

Summary of Major Recommendations

Cross-Cutting Recommendations

  1. Emphasize interconnectedness of RttT priorities. For example, NCDPI staff should include explicit information about how the new standards and assessments are connected to the Teacher Evaluation Process, data literacy, and instructional technology tools and resources.
  2. Continue commitment to work in the field. NCDPI should continue to build relationships with RESAs, LEAs, and schools by engaging directly with local-level staff across the state.
  3. Continue to build NCDPI intra-agency collaboration. Revise and improve the structures already in place to further facilitate development of strong collaborations within cross-divisional work teams. Improve communication and collaboration across the divisions by clarifying purpose, expectations, roles, and responsibilities for divisions. The success of NCLTI can serve as a good working model.
  4. Use the NC Education registration system for all face-to-face RttT sessions. A statewide system is essential in order to track the overall impacts and outcomes of the RttT professional development initiative. A major roadblock to building local capacity is participant turnover; use of a linked, statewide registration system can help ensure that a consistent team of educators from each LEA attends the NCDPI-RESA sessions and the Summer Institutes.

2011-12 NCDPI-RESA Sessions

  1. Increase opportunities for participant discussion and reflection. Session activities could include more opportunities for participants to share knowledge; consider classroom applications of resources, strategies, and techniques; reflect about concepts, strategies, and issues; and share experiences and insights.
  2. Continue to provide opportunities for participants to collaborate across schools and LEAs. Participants regularly identified collaboration with colleagues as one of the most beneficial aspects of the regional RttT trainings.
  3. Increase focus on instructional tools aligned to the new standards. Continue to provide opportunities for session participants to review, share, and identify or create high-quality units and lessons aligned to the new standards.
  4. Increase differentiation for LEAs based on stage of transition to new standards. Different LEAs need different levels of information and support, depending on where they are with implementing their plan for transitioning to the new state standards and assessments.
  5. Clarify objectives for professional development events. Identify and regularly communicate session objectives to local educators. Once those objectives are set, do not change the purpose or focus; LEAs use the calendar and description of events from NCDPI to plan the timeline for local training events.

2012 Summer Institutes

  1. Work toward developing more engaging activities. For example, include more opportunities for discussion and sharing among participants; more hands-on, interactive activities; and activities focused on developing resources to take back to their LEAs.
  2. Provide more facilitative team time. Teams wanted even more time devoted to working together.
  3. Increase differentiation. To the extent possible, 6-12 grade Content Strand sessions should be broken-up into middle and high school groups, and Leaders with Leaders sessions should separate out school- and LEA-level administrators. It would also be beneficial to differentiate sessions based on LEA progress on the RttT priorities.
  4. Reduce redundancy and review. Provide introductory sessions for those participants who have never attended a state-supported RttT professional development event.
  5. Allow LEA teams to facilitate some sessions. Staff from local schools and LEAs can share best practices and resources related to many of the RttT priorities. This strategy also addresses participants’ need for additional networking and collaboration opportunities.
  6. Reconsider the structure of the Accountability and Assessment session. The timing and format of the session made the information difficult for some participants to absorb; recommendations included moving to an interactive session with Assessment and Accountability staff during the Leaders with Leaders strand, and providing additional professional development in the areas of deepening understanding of assessment, using data, and the new school accountability model.
  7. Incorporate participant-recommended topics into future professional development events. Some of the requested topics include: formative and summative assessments; Common Core and Essential Standards; instructional practices for, planning under, and cross-curricular integration of the new standards; meeting diverse needs of learners; the new teacher evaluation process; collecting and interpreting data; and technology integration.
  8. Seek out more flexible facilities. Identify better workspaces for facilitating learning for adults with room for small group collaboration and comfortable seats and tables.
  9. Shorten the length of each day. Nine hours a day was too long to engage in deep thinking around RttT priorities. Consider a three-day event that starts later and ends earlier each day.

View Resource

Full Report PDF Full Report PDF

Projects

Evaluation of Race to the Top

This evaluation was designed to provide formative feedback for program improvement and determine impact on the target goals of each initiative and on overall state-level outcome goals.

Published

March 1, 2013

Resource Type

Report

Published By

Friday Institute for Educational Innovation