North Carolina New Teacher Support Program: Final Race to the Top Evaluation Report
Executive Summary
Overview
The North Carolina New Teacher Support Program (NC NTSP) was developed to provide induction supports to beginning teachers in North Carolina’s lowest-achieving schools and to meet two high-priority needs identified by the state’s Race to the Top (RttT) grant: (1) helping teachers to succeed during their initial years in teaching; and (2) retaining qualified teachers, particularly in high-need schools. The NC NTSP aims to improve the instructional knowledge, skills, attitudes, effectiveness, and retention of participating teachers through the provision of three support components: institutes (multi-day trainings); instructional coaching; and professional development. This report reflects findings from an independent external evaluation of the NC NTSP conducted by the Education Policy Initiative at Carolina as part of the statewide RttT evaluation undertaken by the Consortium for Educational Research and Evaluation–North Carolina (CERE–NC).
Data and Methods
This final evaluation report draws upon the following data sources: (1) participation records from each of the components of the NC NTSP; (2) Perception of Success Inventory for Beginning Teachers (PSI-BT) survey responses; and (3) student demographics and test scores, classroom rosters, teacher evaluation ratings, EVAAS scores, certified salaries, school personnel, and school characteristics files provided by the North Carolina Department of Public Instruction. With these data sources, the Evaluation Team used a comparison group design to examine: levels of participation in NC NTSP components; NC NTSP teachers’ perceptions of the impact of program components on their confidence, knowledge, and skills for teaching; teachers’ perceptions of their self-efficacy and job satisfaction; and the impact of the NC NTSP on teacher value-added, teacher evaluation ratings, and teacher retention.
Summary and Findings
This report reflects evaluation findings for the two full years of program implementation spanning the 2012-13 and 2013-14 school years. The study design and analysis were developed to address evaluation questions across four overarching areas: implementation; teacher knowledge, skills, and attitudes; teacher effectiveness; and teacher retention.
Implementation
To what extent was the NC NTSP implemented as intended, and to what extent did it reach its target population? Over the course of the four-year grant period, the NC NTSP was developed, staffed, and implemented from the ground up and has grown to scale, serving over 1,100 teachers in 114 schools as of the 2013-14 school year. There were substantial differences in program participation and implementation across the four NC NTSP regions. As the NC NTSP doubled in size in 2013-14, there was a drop in attendance at institutes and professional development sessions and fewer instructional coach visits per teacher.
Teacher Knowledge, Skills, and Attitudes
How do teachers perceive the impact of NC NTSP components on their confidence, knowledge, skills, and attitudes toward teaching? There was a statistically significant difference in the proportion of NC NTSP evaluation sample respondents who felt the program components had a positive impact on their teaching, compared with similar services provided by their own school. This significant difference also existed between NC NTSP respondents’ perceptions of program utility and comparison sample perceptions of analogous school-provided services.
Teacher Effectiveness
To what extent does the NC NTSP impact teacher effectiveness as measured by teacher value-added to student achievement (EVAAS) and the North Carolina Educator Evaluation System (NCEES) teacher evaluation ratings? Overall, NC NTSP evaluation sample teachers had significantly higher EVAAS estimates than comparison sample teachers in fifth and eighth grade science. When assessing results by cohort, positive and significant EVAAS results were concentrated within NC NTSP Cohort 1 teachers, while NC NTSP Cohort 2 teachers were generally no more or less effective. Regarding teacher evaluation ratings, there were no significant differences between NC NTSP evaluation sample teachers and comparison teachers in overall models. By cohort, NC NTSP Cohort 1 teachers had significantly higher evaluation ratings on four standards in 2013-14.
Teacher Retention
To what extent does the NC NTSP impact the retention of novice teachers in the same schools or local education agencies (LEAs) and in the state? Overall, NC NTSP evaluation sample teachers were significantly more likely to return to teaching in North Carolina public schools, to the same LEA, and to the same low-performing school. NC NTSP teachers from both cohorts were significantly more likely than comparison sample teachers to return to the same low-performing school.
Limitations
Two primary limitations should be taken into account when interpreting the findings in this report: (1) the ability to isolate the impact of the NC NTSP is diminished due to the lack of a comparison group of teachers working in schools similar to those served by the NC NTSP in all ways other than participating in the program; and (2) given the time required for program development and scaling up, the length of the evaluation period is not adequate to assess the effectiveness of a consistent program model implemented as intended for first-, second-, and third-year teachers over the course of three years.
Recommendations and Next Steps
There are three overarching recommendations for the NC NTSP after the close of the RttT grant:
- The impacts of the NC NTSP on teacher value-added to student achievement and teacher retention support a recommendation to sustain the program beyond the end of the RttT grant;
- Findings from this evaluation should be used to explore the disparities in program implementation by region, and moving forward, implementation fidelity should be formally monitored; and
- Strategies should be explored to counter the decline in program participation and effectiveness seen in 2013-14, such as requiring participating schools and Local Education Agencies (districts) to enforce mandatory participant attendance, and/or concentrating resources toward instructional coaching (the most intensive program component).
View Resource
Full Report PDF Projects
This evaluation was designed to provide formative feedback for program improvement and determine impact on the target goals of each initiative and on overall state-level outcome goals.
Published
August 1, 2015
Resource Type
Report
Published By
Consortium for Educational Research and Evaluation–North Carolina