Executive Summary

Introduction and Evaluation Goals

North Carolina’s four-year Race to the Top (RttT) grant provides an unprecedented opportunity to further the state’s vision for science, technology, engineering, and mathematics (STEM) education and to develop its understanding of what constitutes a successful STEM school. The RttT STEM schools initiative will support two major activities in North Carolina:

• Establishment of four STEM anchor schools (STEM-focused high schools that will serve as regional leaders in STEM education), each of which will be focused on a major area relevant to North Carolina economic development (health and life sciences, biotechnology and agriscience, energy and sustainability, and aerospace); and
• Support for and growth of a broad network of STEM schools across the state, with the anchor schools serving as centers for professional development for principals and teachers in these networked schools.

This report on the first year of the RttT STEM implementation activities provides a descriptive study and documentation of the implementation of the RttT STEM initiative in participating schools. Additionally, it provides formative feedback on the initiative’s long-term goals of building articulated and cohesive models of a STEM school and of a network of STEM schools.

The evaluation is guided by the following research question:
To what extent have the following elements of the network of STEM anchor and affinity schools been implemented as intended?

• A structure for the network of STEM anchor and affinity schools;
• Professional development for STEM school teachers and principals;
• Curriculum of the STEM schools; and
• Partnerships between STEM schools and IHEs, community, and businesses.

In addition to documenting project activities to date, this report considers whether these activities as implemented are adequate to ensure the intended short-term outcomes.

Data and Methods

The evaluation is being conducted via a mixed methods approach, with an emphasis on qualitative data and analyses and survey data and analyses; secondary data and analyses play a larger role in the baseline report (submitted December 2011) and in the final phase of the evaluation (2014). Qualitative data for this report consist of various project documents collected by the North Carolina New Schools Project (NCNSP), as well as observational and interview data collected by the RttT STEM evaluation team.

Findings and Recommendations

Findings and recommendations are organized in the following sections:

Creating an Articulated and Cohesive Model of a Network of STEM Schools
STEM School and Network Model Implementation

1. Structure of the Network
2. Professional Development
3. Development of Integrated Curriculum with Project Units
4. Partnerships

Creating an Articulated and Cohesive Model of a Network of STEM Schools

NCNSP is developing new STEM school and school network models. Creating new models is a complex undertaking that necessarily includes trial and error and refinement of strategies to achieve desired outcomes. The process of refining these models will take a few years.

These new models build on NCNSP’s previous success with implementing early college and redesigned school models using NCNSP’s Design Principles, and it adds a STEM vision that includes (1) incorporating a STEM theme across all subjects in the school; (2) improving math and science teachers’ content knowledge and teaching strategies via extensive professional development; (3) designing and implementing a new project-based STEM curriculum; and (4) becoming a member of a theme-based network of schools, business, and IHE partners. The Design Principles and STEM vision currently are not integrated with each other.

Recommendation:

• To address the challenges that schools in the network face in terms of learning about and implementing multiple components of the model, the North Carolina New Schools Project should integrate the six Design Principles with the various components of the STEM vision.

STEM School and Network Model Implementation

Structures for networking, professional development, curriculum development, and partnerships are somewhat on track; however, as with any plan or proposal, implementation requires a substantial amount of real-time development. In particular, the North Carolina Department of Public Instruction (NCDPI) and NCNSP interpreted two elements of the proposal differently: (1) the criteria for identifying network schools, and (2) deliverables for the integrated curriculum with inquiry-based project unit(s). Delays in identifying participating schools and in reaching a shared understanding regarding deliverables for the curriculum development work subsequently have delayed overall implementation.

I. Structure of the Network

School enrollment.

• Three anchor schools had students enrolled in the 2010–11 school year, and the fourth will be opened in 2012–13 school year, as planned.
• There were significant delays in establishing the network of affinity schools due to discrepancies in understandings about school eligibility and criteria for selecting schools to participate. The list of network schools was not finalized until November 2011. These delays affected the effectiveness of
  the first-year activities and are likely to affect short-term outcomes.
• One of the four anchor schools opened for the 2011-2012 school year, and one will not open until 2012-2013. Therefore, these schools may not be ready to serve as model schools for the first few years of project implementation.

Face-to-face and online networking among schools.

• NCNSP provides many face-to-face networking opportunities for participating schools.
• The online community has four types of networks with varying numbers of participants and intensity of communication: (1) main STEM Affinity Network; (2) theme networks; (3) content networks; and (4) school networks.
• The online networks are used mainly for sharing resources and general information and for announcements. The level of interaction online has been relatively low so far, despite moderators’ efforts.
• Participants appreciate networking opportunities and express willingness to collaborate across schools.

Recommendations:

• To reduce potential negative impacts, initiative leads should develop a plan for getting those schools that joined the network late rapidly up-to-speed with respect to implementation of all model components.
• To increase collaboration among schools, NCNSP should consider assigning groups of schools to complete common tasks or projects together.
• Two of the four anchor schools are brand-new schools; NCNSP may want to rethink the role of these anchor schools as role models for other network schools.
• To address the challenges of designing a new, complex model with a number of schools that are either brand-new or new to the network (including two of the four anchor schools),
  initiative leads may want to consider continued use of the four NC Learning Lab Schools as sites for study visits by teams from network schools until anchor schools demonstrate excellence in implementing the STEM vision.

II. Professional Development

• NCNSP offered 10 formal face-to-face professional development sessions, supplemented by on-site leadership and instructional coaching. The majority of the professional development activities focused on implementing the Design Principles, with an emphasis on using inquiry- and project-based learning to teach math and science content.
• External observers rated the majority of the sessions as accomplishing their goals and as effective or exemplary professional development. An area for improvement that observers identified was the need for additional time and opportunities for participants to reflect on what they had learned and to
  consider its application in the classroom.
• Participants identified the most significant barriers for implementation as those related to: (1) obtaining buy-in for the work from the different constituencies; and (2) having time for planning and implementation.
• The potential impact of the professional development was reduced by the changes in the list of schools participating in the RttT STEM network.

Recommendations:

• Provide opportunities for schools that joined the network late to catch up via provision of the professional development they will need for successful implementation of the STEM model.
• Provide participants with additional time and opportunities during the professional development sessions to debrief on the activities and discuss how the activities can be implemented in the classroom. It would be particularly useful to help participants explicitly understand the nature of student learning occurring in the activities and how those activities
  might address (or potentially reinforce if not done well) students’ misconceptions about the content.
• Explicitly address concerns about lack of time by providing models of schedules that provide adequate time for collaboration and planning. Additionally, this year, the STEM initiative provided funding for additional planning days in the summer; it might be worthwhile to find additional resources to continue and expand this option.
• To increase buy-in among staff, consider explicit training for leadership teams on creating a common STEM vision for their staff. Part of this involves creating and communicating a well-defined STEM framework with a compelling rationale for its adoption.
• Add STEM themes and new project-based curriculum areas to the coaching report template to help the coaches explicitly focus their work on the STEM vision components.
• To improve the NCNSP’s data collection methods, both participant evaluations and event sign-up should be completed online, with all evaluations following a standardized form, designed in conjunction with the evaluation team.

III. Development of Integrated Curriculum with Project Units

• NCNSP conducted a number of activities to support the development of project units: a three-day Summer Project Development Workshop; two days of in-school project development; and a two-day Common Practices Symposium in October 2011.
• Most of the 13 schools that participated in the summer are actively engaged in project development; however, only four of those are on the final STEM school network list. The rest of the schools in the network started their project-related professional development in October.
• School staff working on project design encountered a number of challenges, such as: lack of time to do very time-consuming project design work in addition to teacher workload; effective integration of projects with the regular curricula and creation of meaningful experiences for students; and insufficient resources needed for successful project implementation.

Recommendations:

• Provide more background knowledge to teachers about the STEM themes and the engineering design process prior to their work on projects.
• Conduct theme-related webinars to make learning more accessible for everyone in the school.
• Encourage schools to work collaboratively on fewer projects, so that they can combine their human resources.
• Engage instructional coaches in supporting the project work.
• Reach shared understanding of expectations for the deliverables associated with integrated curricula with inquiry-based project units, to ensure that NCNSP and the network of schools developing these deliverables align resources to meet those expectations.
• Consider more active involvement on the part of IHE and business partners in designing a project-based curricula.
• Explore the possibility of contracting with a few highly skilled teachers to develop model projects for each of the four affinity networks.
• If the goal is to create a curriculum that is to be used by others, do not rely on school staff to do this unless significant resources are made available for this to occur over the summer.

IV. Partnerships

• NCNSP established four Industry Innovation Councils (IIC), one for each affinity network.
• Business and IHE partners started to participate in the network face-to-face events and to provide teachers and principals with their expertise about the network themes.
• Teachers found this sharing of information useful for their STEM-related work in the schools.

Next Steps

In preparation for the next report (December 2012), the Evaluation Team plans to:

• Analyze data collected through the end of the 2011–12 school;
• Continue to analyze project documents received from NCNSP related to all professional development and partners’ activities;
• Continue to monitor online and face-to-face networking;
• Collect and analyze any products generated by the project development work of participating schools;
• Conduct site visits in the anchor schools (site visits to network schools will occur in Years 3 and 4);
• Analyze coaches’ reports and interview selected instructional and leadership coaches about their work and about the effects on schools of participating in the STEM network;
• Conduct at least one focus group with teachers at one of the professional development or face-to-face networking events in the Spring;
• Conduct observations of Industry Innovation Council meetings and focus groups with business and IHE partners about supports they provide to the networks;
• Investigate RttT-funded NC STEM Collaborative activities; and
• Conduct a quantitative analysis comparing the background characteristics of schools in the network with those of other STEM and non-STEM schools in North Carolina.

View Resource

Full Report PDF

Projects

Evaluation of Race to the Top

This evaluation was designed to provide formative feedback for program improvement and determine impact on the target goals of each initiative and on overall state-level outcome goals.

Published

April 1, 2012

Resource Type

Report

Published By

Consortium for Educational Research and Evaluation–North Carolina