North Carolina New Teacher Support Program: Interim Evaluation Report
Executive Summary
The Consortium for Educational Research and Evaluation–North Carolina is evaluating North Carolina’s use of Race to the Top (RttT) funds to support the North Carolina New Teacher Support Program (NC NTSP), a comprehensive induction program that targets beginning teachers in schools across the state that qualified for RttT services by being in the lowest 5% of student achievement or by having a graduation rate below 60% (hereafter referred to as RttT schools). The overall purpose of this evaluation is to assess the extent to which this program improves the instructional practice quality, self-efficacy, value-added effectiveness, and retention of participating teachers through the provision of three support components: an institute, instructional coaching, and professional development.
Program Overview
The NC NTSP is implemented by the University of North Carolina General Administration (UNC-GA), in partnership with the North Carolina Department of Public Instruction (NCDPI). The program is organized and administered through a central NC NTSP office and four regional anchor sites located at UNC institutions—East Carolina University, the UNC Center for School Leadership Development, UNC Greensboro, and UNC Charlotte.
The program began supporting a small number of beginning teachers in 2011-12 at qualifying and participating schools with low student achievement in four of North Carolina’s eight education regions. For the 2012-13 academic year, the NC NTSP expanded to offer services to first-, second-, and third-year teachers at participating lowest-achieving schools in all eight regions across the state. To support these teachers in 2012-13, the NC NTSP offered: a week-long Summer Institute in August 2012 and a make-up three-day Winter Institute in December 2012; intensive face-to-face and virtual instructional coaching; and six professional development sessions within each of the four NC NTSP regional anchor sites.
Purpose and Structure of the Report
The purpose of this report is to provide UNC-GA and NCDPI with formative feedback on Year 2 (2012-2013) program implementation ahead of the final summative report due in fall 2014. To this end, the report covers the following areas: (1) descriptions of the evaluation sample for the NC NTSP and of the comparison sample teachers; (2) data sources for this interim evaluation report; (3) data on program implementation, participation, and participants’ perceptions of quality for each of the three NC NTSP components; and (4) reports of teacher self-efficacy and job satisfaction.
Evaluation Sample
The NC NTSP evaluation sample includes first-, second-, and third-year teachers who received NC NTSP services and were employed in participating NC NTSP-eligible schools as of December 2012. This criterion produced an evaluation sample of 384 teachers in 59 schools across 16 Local Education Agencies (LEAs). The sample excluded: (1) novice teachers receiving NC NTSP services at non-RttT schools[1]; (2) novice teachers at RttT schools whose schools opted into the program after December 2012; and (3) novice teachers at RttT schools who were hired or entered the program after December 2012.
To construct a comparison sample for 2012-13, the Evaluation Team selected novice (first- and second-year) teachers[2] within a set of comparably low-performing schools (lowest-achieving 10%) for a total of 838 first- and second-year teachers in 148 schools across 48 unique LEAs.
Data Sources
The Evaluation Team collected the data presented in this 2012-13 interim evaluation report from the UNC-GA implementation team, NC NTSP instructional coaches, NC NTSP evaluation sample teachers, and comparison sample teachers. Specifically, this interim evaluation report draws upon the following data sources: (1) participation records from each of the components of the NC NTSP—Summer Institute, Winter Institute, instructional coaching, and professional development; (2) the Instructional Coach Survey responses of NC NTSP instructional coaches; and (3) the Perceptions of Success Inventory for Beginning Teachers (PSI-BT) survey responses of NC NTSP evaluation sample teachers and comparison sample teachers.
Findings
Institutes
To address the challenge of providing institutes for new teachers who were hired after the school year began or who were required to complete early-year LEA induction programs, the UNC-GA added a Winter Institute to its existing Summer Institute component. Combined, nearly 57% of the NC NTSP evaluation sample teachers attended either the Summer or Winter institute, with Institute attendance varying across regions. For those teachers who attended an Institute and responded to the PSI-BT survey, evidence indicates positive perceptions of Institute quality, with stronger perceptions of quality among those who attended the Winter Institute.
Instructional Coaching
The average number of monthly visits per teacher varied between the four regional anchor sites and over the course of the year, ranging between one and eight visits per month. The average amount of time the instructional coaches spent with each teacher varied as well, with an average of over three hours per month. Over the course of the 2012-13 academic year, NC NTSP instructional coaches made a total of 11,903 in-person coaching visits to participating evaluation sample teachers.
Responses to the NC NTSP Instructional Coach Survey indicate that the most frequent activities employed by instructional coaches were observing teaching and giving feedback, providing strategies for improved instruction, identifying necessary resources, and promoting teachers’ self-efficacy. Overall, NC NTSP teachers who responded to the PSI-BT survey reported positive perceptions of instructional coach quality, and indicated significantly higher levels of satisfaction with their NC NTSP instructional coach than with their school- or LEA-provided mentor. NC NTSP teachers also indicated significantly higher levels of satisfaction with their NC NTSP instructional coach than the comparison sample teachers expressed about their school- or LEA-provided mentor. Comparing across regions, NC NTSP teachers in the UNCG region provided the highest ratings of their instructional coaches.
Professional Development
Overall, professional development attendance was generally low, but was variable across regions. The region that had the highest level of attendance—the UNC Center for School Leadership Development region—also offered multiple intra-region professional development sessions and received the most support from LEAs and schools via requirements that their teachers participate. Those teachers who attended at least one NC NTSP professional development session and responded to the PSI-BT expressed a higher level of satisfaction with program-provided professional development than with their school-provided professional development. They also expressed a higher degree of satisfaction with program-provided professional development than the comparison sample expressed about their school-provided professional development. These professional development ratings varied across NC NTSP regions, with teachers in the UNC-CSLD region reporting lower levels of satisfaction with NC NTSP professional development.
Self-Efficacy and Job Satisfaction
Finally, NC NTSP teachers who responded to the PSI-BT survey reported higher levels of self-efficacy and job satisfaction than did comparison sample teachers. Within the NC NTSP, teachers in the UNC-CSLD region expressed less satisfaction with their current job and a greater likelihood of leaving their LEA/school or the profession.
Next Steps
The findings in this report represent intermediate outcomes that are hypothesized to influence program impact over time. The summative evaluation report, due in fall 2014, will include the 2012-13 primary outcome data of teacher value-added, teacher retention, classroom observations, and principal evaluations, as well as the 2013-2014 follow-up to the implementation data presented in this report.
1 In the 2012-13 academic year, the NC NTSP provided services to a small number of schools that are not part of the original Race to the Top sample—in the lowest 5% of student achievement or graduation rates below 60%.
2 There were a small number of third year teachers in the NC NTSP sample; therefore, the Evaluation Team did not include third year teachers in the comparison sample.
View Resource
Full Report PDF Projects
This evaluation was designed to provide formative feedback for program improvement and determine impact on the target goals of each initiative and on overall state-level outcome goals.
Published
March 1, 2014
Resource Type
Report
Published By
Consortium for Educational Research and Evaluation–North Carolina