Productive Connections (DST 2011-12)

Executive Summary

Background

As one key component of North Carolina’s 4-year, $400 million Race to the Top (RttT) grant activities, the District and School Transformation Division (DST) of the NC Department of Public Instruction (NCDPI) is intervening to improve student achievement and high school graduation rates in the lowest-achieving five percent of the state’s high schools, middle schools, and elementary schools, some 118 schools in all. In addition, the DST is working with twelve of the lowest-achieving school districts in the state to strengthen their ability to lead and support effective school reform. As expressed in the current scope of work for the RttT grant, the goals of the initiative are these:

Turn around the lowest 5% of conventional elementary, middle, and high schools based on the 2009-10 Performance Composite and grade span,
Turn around conventional high schools with a 4-year cohort graduation rate below 60% in 2009-10 and either 2008-09 or 2007-08, and
Turn around the lowest-achieving districts with a 2009-10 Local Education Agency (LEA) performance Composite below 65%.

Three organizations—the Carolina Institute for Public Policy (UNC-Chapel Hill), the Friday Institute for Educational Innovation (NC State University), and SERVE (UNC-Greensboro)—have formed a consortium to evaluate the Race to the Top grant. The primary purpose of the evaluation is to provide objective research to help the NCDPI adjust RttT work as it progresses, a type of evaluation often labeled “formative” because it seeks mainly to help the client organization shape work in progress rather than simply to render an up-or-down “summative” judgment on its impact.

The present report is the second of four reports focusing on the work of the DST. The first examined the division’s pre-Race to the Top interventions in low-achieving schools in an effort to distill lessons from that earlier work that could guide interventions during the Race to the Top grant period. In that study, we found that improvement had taken place through a process we called scaffolded craftsmanship. The scaffolding consisted of a planning framework, professional development, and coaching provided by the NCDPI and its partner organizations. With these supports, school leaders and staff gradually learned how to improve performance through guided reconstruction of key school functions rather than implementation of externally designed models.
The present report shifts the focus from school-level interventions to the DST’s district-level work in the twelve lowest-achieving school districts in the state. In this round of study, we found that in low-achieving districts, connections are weak or missing between and within levels of the systems—the central office, schools, and classrooms. The essence of what the DST is doing is to strengthen or create productive connections across and between levels of the systems. Before explaining more fully what we mean by this term, we pause briefly to outline our methods and purpose.

Study Methods and Purpose

By conducting interviews with DST coaches and local educators in four districts that are at different points in the transformation process, supplemented by review of documents such as strategic plans and needs assessments, we sought to identify the essential elements of the district transformation process and to develop a rough “theory” of how that process takes place. That is, we attempted not only to describe the steps taken by the DST and local educators, but to highlight the essentials of that process, including the factors that impede as well as those which facilitate progress. By isolating the essential components of the process, we sought to develop a clear and concise account of the dynamics of district transformation. In this report, we offer a relatively detailed account of the process, but we also present a more theoretical account—a series of propositions or hypotheses that sum up the process in an economical way. This theoretical account is intended to help DST leaders and coaches grasp the essentials of the transformation process in a way that may be difficult in the midst of the very complex change process they are engaged in. It is, however, just a “first draft” of a theory that we will test and refine through additional study over the next two years.

Findings

The theory that we have derived from the interview and other data we collected in the four districts in our sample can be summarized in the following propositions:

In low-achieving districts, a first challenge is to establish the improvement of student achievement and related student outcomes as the central goal of the school board and superintendent, not just as a broad policy, but in the continuing flow of specific decisions that arise over time. The proliferation of plans based on mandates or requirements imposed from many sources and the potentially conflicting claims of multiple community constituencies pose ongoing threats to the preservation of a dominant focus on student achievement.
Further, in low-achieving districts, connections are missing or weakly developed at many junctures up and down the system. That is, many junctures across and within the levels of a district lack one or more of the elements of a productive connection. Productive connections include:

the combination of assertive accountability and bonds of relational trust and engagement that fosters commitment to improve student achievement,
the provision of guidance, instruction, and assistance that builds the knowledge and skills necessary to improve performance, and
the ongoing support for and monitoring of good practice, assessment of outcomes, and use of assessment results to improve practice which assure that commitment, knowledge, and skills are actually put into practice to produce the desired outcomes.

By “junctures” we mean the connections between superintendents and their boards; between superintendents and central office administrators; among central office administrators; between superintendents and central office administrators on the one hand and principals on the other; among principals across schools; between principals and teachers within schools, among teachers within schools or departments; between principals and teachers on the one hand and parents on the other; and between teachers and students.
The DST’s district level interventions are essentially efforts to strengthen or create productive connections at all of these junctures, thus weaving a web of support for the improvement of student achievement.
The elements of productive connections are similar across all of these junctures. In slightly different forms, they all involve (1) the combination of accountability and trusting relationships, (2) guidance, instruction, and assistance; and (3) monitoring practice, assessing outcomes, and using assessment results to improve practice.
To carry out the latter two functions—(2) to guide, instruct, assist, and (3) to monitor and improve practice — at any level of the system, leaders at each juncture need a clear, explicitly-defined concept of good practice. For example, a principal needs a well-defined image of good teaching as a basis for monitoring and shaping classroom instruction.
Similarly, central administrators responsible for supervising principals need a well-defined image of good principal leadership.
To improve practice over time, leaders at each juncture also need an effective assessment system and knowledge of how to use assessment results to make changes in the shared image of good practice and in actual practice.
The more complete the web of productive connections in a district, the more student achievement will rise over time.
Pockets of poor achievement—such as a low-performing school or department—indicate failures to complete the web of productive connections.
A complete web of productive connections includes both links in the administrative chain of command between levels of the system and links among colleagues within levels of the system, the latter often referred to as professional communities. Absent productive professional links, productive administrative links will not be adequate to raise student achievement sharply.
The key capacity of an individual at any level of the system is the capacity to make productive connections, both with the people s/he is responsible for leading and with colleagues.
Professional development and coaching that are well-calibrated to the level of trust in the coach-client relationship and that attend to all elements of productive connections up and down the system can make strong contributions to the improvement of student achievement, but where connections remain weak after sustained intervention, personnel replacement is required.

In sum, “district transformation” is essentially the process of changing a disconnected district into a productively connected district.

Conclusion

It would be premature to make any summative judgment of the degree to which the District and School Transformation division has succeeded in transforming the districts where it is intervening, but it may be useful to offer an interim assessment of progress to date. In schools served by the DST, the two-year improvement in Performance Composites from the 2009-10 to the 2011-12 school year clearly outpaced the statewide average improvement—by 8.8 percentage points at the high school level, 4.7 points at the middle school level, and 7.1 points at the elementary school level. Further, the improvement in Performance Composites among schools where the DST was intervening at the district level in addition to the school level outpaced the improvement in schools where the DST was intervening solely at the school level—by 13.2 percentage points at the high school level, 3.5 points at the middle school level, and 2 points at the elementary school level. These findings suggest that the DST’s school-level interventions are making a notable difference in performance improvement, and that the district-level interventions are adding additional value beyond the school-level interventions by themselves.

In addition to improving student achievement as measured by Performance Composites, the DST also set the goal of improving high school graduation rates. On this goal, the evidence is encouraging for DST interventions overall, but offers less support for a unique contribution for the district-level interventions. Statewide, from 2009-10 to 2011-12, high school graduation rates improved by 6.2 percentage points. In high schools served by the DST, the average two-year improvement was 9.5 percentage points—3.3 points more than in the state as a whole. This suggests that the DST has contributed to improvement in the graduation rate for the schools it served. But average graduation rates in high schools served solely through DST intervention at the school level actually improved 1.2 points more than did high schools where the DST was also intervening at the district level. The latter finding is not entirely surprising. The schools in districts where DST chose to intervene started with what amounts to a double disadvantage—they were low performing as schools but were also situated in low-performing districts. So it may take longer to make a difference in these schools than in those located outside of low achieving districts.

All in all, it appears that the DST is making a measurable contribution to the improvement of both performance and graduation rates in the schools it serves. Our findings from two rounds of study suggest that the school-level improvements take place through a process of scaffolded craftsmanship and that the additional contributions of the district level interventions may result from making productive connections up and down the school systems, thus supporting scaffolded craftsmanship in the initially low-achieving schools in those districts. During the remaining two years of our evaluation of the Race to the Top-supported efforts of the District and School Transformation unit, we will examine these processes more fully in order to refine our findings and test their validity.

View Resource

Full Report PDF

Projects

Evaluation of Race to the Top

This evaluation was designed to provide formative feedback for program improvement and determine impact on the target goals of each initiative and on overall state-level outcome goals.

Published

April 1, 2013

Resource Type

Report

Published By

Consortium for Educational Research and Evaluation–North Carolina