NC NTSP Year 1 Report

Executive Summary

Based on research showing that beginning teachers represent more than 6% of North Carolina’s public school teachers, have the greatest potential for rapid improvement in terms of raising student achievement, and exit teaching at significantly higher rates than more experienced peers, providing comprehensive induction supports to increase the effectiveness and retention of beginning teachers was identified as a high-priority reform initiative in the North Carolina Race to the Top (RttT) proposal. To meet this need, the University of North Carolina General Administration (UNC-GA), in partnership with the North Carolina Department of Public Instruction (NCDPI), has developed and implemented the North Carolina New Teacher Support Program (NC NTSP) for novice teachers in the lowest-achieving schools in the state.

Overview of RttT NC NTSP Activities

NC NTSP provides a three-phase induction program, modeled in part after the Teach For America support program used for teachers in similar school settings. The program begins before teachers enter the classroom and continues for three years, at the conclusion of which successful teachers will qualify for their full (continuing) license.

NC NTSP has three main components:

1. An intensive one-week Summer Institute before teachers begin their first school year;
2. Six full-day follow-up professional development sessions, three each during the fall and spring semesters of Years 1 and 2;
3. Classroom observations and mentoring by instructional coaches, at least once per month continuing for three years.

The program goal is to improve the effectiveness of novice teachers through intensive and relevant induction support, aligned to each teacher’s individual teaching assignments and school contexts and designed to help them (a) understand and apply the Common Core State Standards and North Carolina Essential Standards at the grade level and in the content area they will be teaching; (b) engage in instructional planning focused on effective teaching practices, effective use of data, and classroom lessons aligned with the curriculum goals; and (c) address the specific challenges of working with diverse groups of students, many of whom have a history of low achievement in challenging schools.

Overview of RttT NC NTSP Evaluation Activities

North Carolina’s RttT proposal included a commitment to independent evaluations for each initiative. Over the next two years, the RttT Evaluation Team will document NC NTSP activities and collect data about program implementation, participation, and short- and long-term outcomes for teacher effectiveness, retention, and ultimately student achievement. Data will be collected via surveys, direct classroom observation, and analysis of administrative data on students, teachers, and schools. The purpose of this evaluation is to provide detailed information about the implementation and outcomes of NC NTSP for new teachers in the lowest-achieving schools.

The evaluation of the RttT New Teacher Support Program is guided by four primary research questions that address the topics of program implementation, program participation, short-term outcomes, intermediate outcomes, and long-term outcomes. We structured the Findings section of this report around these four questions:

1. Is NC NTSP being implemented as intended, and what recommendations could improve implementation if needed?
2. Does access to the teacher induction program (NC NTSP) increase teachers’ confidence and retention?
3. Does access to the teacher induction program (NC NTSP) increase teachers’ effectiveness in terms of student achievement, instructional proficiency, and student engagement?
4. What are the recommendations to sustain the benefits attributed to NC NTSP, if any?

This report addresses program implementation and participation and short-term/intermediate outcomes of NC NTSP. Long-term outcomes (student achievement and teacher retention) and recommendations to sustain the benefits attributed to NC NTSP, if any, will be addressed in future evaluation reports.

Evaluation Findings

Program Implementation and Participation

NC NTSP was designed to support novice teachers working in low-achieving schools with the goal of enhancing classroom instruction, improving teacher effectiveness, and decreasing teacher turnover. The three-component model is comprised of an intensive five-day Summer Institute, direct instructional coaching throughout the school year, and targeted professional development to meet the needs of these teachers. The intended population for NC NTSP in 2011-12 was all beginning teachers (approximately 200) in schools in the lowest-achieving 5% within four of the eight regions of North Carolina. Of the 54 lowest-achieving schools eligible for NC NTSP in 2011-12, 13 schools chose to participate. In all, 33 beginning teachers from these 13 schools attended the 2011 NC NTSP Summer Institute, 24 of these teachers received NC NTSP instructional coaching, and 16 of these teachers participated in formal NC NTSP professional development sessions. An Executive Director for the NC NTSP was hired in March 2012; since that time, the program has demonstrated significant progress toward meeting the implementation goals for the statewide rollout of the program in 2012-13.

Short-Term Outcomes

According to a survey conducted in August 2011, almost all NC NTSP teachers reported feeling knowledgeable and confident in their ability to carry out practices essential to teaching success by the conclusion of the 2011 Summer Institute. Approximately 85% of participants reported being familiar with the NC Standard Course of Study for their grade and/or subject; over 90% of respondents had a clear vision of the knowledge and critical thinking skills their students should attain; and nearly all respondents were familiar with the components of a classroom management plan and were aware of management styles that best promote student achievement.

Intermediate Outcomes

This report focuses on teacher instructional practices, self-efficacy, and student engagement using CLASS observations and Omnibus survey data. Overall, these data suggest that NC NTSP teachers had slightly higher levels of instructional proficiency, but reported slightly lower levels of self-efficacy than the comparison sample of teachers chosen from the remaining four regions. Several factors require that one exercise caution when interpreting this data. First, NC NTSP teachers were not exposed to the full NC NTSP model as intended, so one would not expect them to have received the full potential benefit. Second, and perhaps most importantly, the small sample size of this first cohort makes it challenging to detect statistically significant differences between NC NTSP and comparison teachers.

Recommendations

Data from the NC NTSP implementation in 2011-12 suggest several strategies that could be employed to help strengthen the education workforce through the support of novice teachers. To further enhance the program’s effectiveness, the RttT Evaluation Team recommends the following considerations as NC NTSP is implemented statewide.

1. Offer differentiated instruction

   Feedback from participants at the 2011 Summer Institute suggests the need for further differentiation of instruction based on the types of preparation for teaching that individuals had received prior to the institute. Some traditionally prepared teachers reported that the material covered was redundant with their formal training, whereas it was reported as novel by the alternatively prepared teachers. Maximizing differentiation among grades, subjects, and preparation types would promote a more tailored learning environment. It is recommended that consideration also be given to how best to tailor the information for alternative-entry teachers versus traditionally prepared teachers.

2. Provide immediate treatment supports after the Summer Institute

   During the 2011-12 start-up year, instructional coaches were not hired until the spring of 2012, and many participants in NC NTSP did not receive further support (after the Summer Institute) until well into their second semester of teaching. Coaching visits and professional development sessions should be scheduled immediately following the start of the school year.

3. Improve standardization of treatment components

   The strategy of using anchor institutes to support the coaching and professional development activities of proximal geographic regions receiving services from NC NTSP has the benefit of localized knowledge and support of participating teachers and schools. However, data from 2011-12 revealed significant variation in both the frequency and content of the treatment components between the two anchor institutes. For example, instructional coaches in the UNC-Charlotte region averaged over 27 visits per NC NTSP teacher, while instructional coaches in the ECU region averaged 6 coaching visits per NC NTSP teacher.

4. Investigate factors influencing attendance at the Summer Institute

   The Summer Institute is a distinguishing treatment component of the NC NTSP. However, the timing of late hires, and the prevalence of late hires within these schools, provides a significant challenge for widespread exposure among eligible teachers. The UNC-GA program implementers are in the process of conducting additional follow-up with schools and LEAs to explore alternatives that would facilitate a broader reach of this component. Viable options will be vetted with NCDPI for the 2013-14 school year to determine an optimal approach given these realistic constraints.

5. Formalize the social networking efforts

   Informal networking was observed at the 2011 Summer Institute among new teachers within the same districts, schools, and grade level/content areas. Participants expressed a desire to continue those connections after the institute had concluded, but this was not a part of the original program design. The program design should be expanded to include the use of forums, such as Edutopia or other social networking platforms, to facilitate networking amongst participants, along with explicit requirements for networking and social events for the institutions and instructional coaches in each of the regions.

6. Consider adjusting the reporting timeline in relation to reporting key outcome variables

   Currently, the evaluation reports are due in October, which is before data sources such as McREL evaluations, value-added estimates of teachers’ effectiveness, and student surveys (Tripod) are available for analysis. The RttT Evaluation Team, UNC-GA program implementers, and NCDPI RttT administrators should consider altering the due dates for future evaluation reports to maximize the timeliness of the information that can be provided. Moving the next delivery deadline to March 2014 would allow time for the RttT Evaluation Team to obtain, merge, analyze, and report on the additional data.

Addendum

Discussions surrounding the content of this report revealed discrepancies in understanding and expectations between the NCDPI RttT leadership and the UNC-GA program implementers. The challenges posed by these discrepancies in understanding can be mitigated going forward by a commitment on both sides to establishing and maintaining an agreed-upon communication plan, and to prioritizing a collaborative approach in all programmatic decision-making. To optimize program effectiveness, these discussions should incorporate consideration of formative feedback recommendations regarding the intervention design and implementation.

Next Steps for the NC NTSP Evaluation

Data necessary to fully assess the impact of NC NTSP on teacher retention and student achievement is not available for analysis at the time of this report. In this report, the Evaluation Team assesses implementation and short-term/intermediate outcomes. Future evaluations will expand analyses of these intermediate outcomes by including McREL evaluation scores and student survey responses and will also examine longer-term student achievement and teacher retention outcomes. In addition, a detailed and targeted survey of teachers’ perceptions of the quality and effectiveness of each component of NC NTSP will be administered to the treatment group, while a tailored version will be administered to a control sample to try to isolate the relative impact of this program above and beyond standard induction practices that may already exist in these low-performing schools. Evaluation data collected after the final year of the program will be essential to identify the full impact of the program.

View Resource

Full Report PDF

Projects

Evaluation of Race to the Top

This evaluation was designed to provide formative feedback for program improvement and determine impact on the target goals of each initiative and on overall state-level outcome goals.

Published

April 1, 2013

Resource Type

Report

Published By

Consortium for Educational Research and Evaluation–North Carolina