An Evaluation of the North Carolina Educator Evaluation System for School Administrators: 2010-11 through 2013-14
Executive Summary
The purpose of this report is to evaluate the effects of adding an eighth standard, school-level Education Value-Added Assessment System (EVAAS) scores, to the evaluation of school principals that is based on the seven standards in the North Carolina Standards for School Executives (NCSSE). To that end, this report describes the relationship between the principal evaluation ratings and other measures of administrator effectiveness, as well as trends in the administrator evaluation between the 2010-11 and 2013-14 school years.
Major Findings
- From 2010-11 through 2013-14, 27.8 percent of principals were found to need improvement based on their superintendents’ ratings and/or school-level EVAAS results. In 2013-14, superintendents’ ratings assigned 7.6 percent of principals (128 principals) to the “needs improvement” category.
- In 2013-14, 25.1 percent of principals (515 principals) were rated as at least Proficient by their superintendents on all seven standards but had school EVAAS scores at the Did Not Meet Expected Growth level.
- Principal Instructional Leadership measures (including clear communication, high standards and data use) were strongly correlated with superintendents’ ratings of their principals, which indicates that superintendents are primarily rating principals based on their instructional leadership rather than differentiating between the seven standards.
- Even though the Teacher Working Conditions survey is a suggested source of evidence for the principal evaluation process, key items on the survey are only loosely correlated with principals’ scores, if at all, indicating the survey information is not being used systematically in the evaluation process.
- The direct measures of principal effectiveness that were most highly correlated with the principal evaluation scores were measures that were not available for superintendents to use as artifacts in the principal evaluation process, indicating that the recommended measures of principal performance were not used by superintendents.
- Objective measures of principals’ performance, such as retention of effective teachers or school value-added scores, are not strongly correlated with superintendents’ ratings of principals’ performance, indicating that these measures do not systematically influence principal evaluation ratings.
- Of the 20 objective measures of principal performance compiled for this evaluation, most are uncorrelated with principal ratings or, at best, loosely correlated, indicating that objective measures rarely influence superintendents’ ratings of principals.
- Most of the items from the Teacher Working Conditions survey that significantly correlate with composite principal evaluation scores are not items that would be expected to be important indicators of principal effectiveness.
- Superintendents’ ratings of principal effectiveness do not appear to be equally distributed across school context. As the percentage of black students or free/reduced price lunch students increases, the composite evaluation score of the schools’ principal tends to decrease. It was not possible to test whether less effective principals were assigned to schools with concentrated poverty or black populations or if superintendents rate principals lower, regardless of actual principal quality, when they oversee those types of schools. Both explanations are plausible and both raise concerns for the evaluation of principals.
- Superintendents rated principals either Proficient or Accomplished, on average, 75 percent of the time, which provided limited information on individual principals’ specific strengths and weaknesses. Superintendents rate principals globally rather than providing meaningful distinctions on principals’ performance on each standard.
- Superintendents’ ratings have not varied over time, indicating little refinement in using NCSSE ratings to provide principals with feedback on strengths and weaknesses.
Overall, it seems that, in spite of a strong theory that systematic evaluation of principals could lead to improving principals’ performance through the NCSSE ratings, it is unlikely that the system as it is currently implemented will do so. The vast majority of principals receive ratings above Proficient for all standards, even though many schools are classified as performing below expectations. In addition, this evaluation provides some evidence that principals’ rating may not be entirely fair—principals in schools with higher concentrations of African-American and economically disadvantaged students receive lower ratings. To overcome some of these issues, the State Board of Education may wish to incorporate other measures of principal performance—such as retention of effective teachers, teacher survey ratings of principals’ instructional leadership, and teacher survey ratings of the fairness and feedback provided in teacher evaluations—into a composite quantitative rating of principals’ overall performance.
View Resource
Full Report PDF Projects
This evaluation was designed to provide formative feedback for program improvement and determine impact on the target goals of each initiative and on overall state-level outcome goals.
Published
February 1, 2016
Resource Type
Report
Published By
Consortium for Educational Research and Evaluation–North Carolina