Skip to main content
William & Ida Friday Institute for Educational Innovation

Race to the Top Evaluation: STEM Affinity Network—Second-Year Report

Executive Summary

Overview of the Evaluation and Progress Made since the Last Report

This second annual report of the RttT STEM implementation activities documents ongoing implementation of RttT STEM initiative in participating schools and assesses intermediate outcomes for students and staff in anchor schools after one year of implementation.

The evaluation is guided by the following two research questions:

  1. To what extent have the four key elements of the network of STEM anchor and affinity schools (network structure, professional development, curriculum, and partnerships) been implemented as intended?

  2. What are the intermediate outcomes for students and staff in anchor schools after one year of implementation?

In addition, this report notes recommendations from the Year 1 evaluation report that were addressed during the second year of implementation of the initiative. In particular, the implementation team:

  • Integrated the six North Carolina New Schools Project (NCNSP) Design Principles with the various components of the STEM vision;

  • Provided explicit training for leadership teams on creating a common STEM vision for staff;

  • Continued using the four NC Learning Lab Schools as sites for study visits by teams from network schools while eventual anchor schools continued to develop;

  • Provided opportunities for schools that joined the network late to catch up via provision of PD necessary for successful implementation of the STEM model;

  • Provided more background knowledge to teachers about the STEM themes and the engineering design process prior to their work on projects;

  • Engaged instructional coaches in supporting the project work;

  • Actively involved IHE and business partners in designing a project-based curriculum;

  • Contracted with highly-skilled teachers to develop model projects for each of the four affinity networks; and

  • Designed and used a standardized participant evaluation form for evaluations of multiple PD offerings from NCNSP.

Findings and Recommendations

One of the initiative’s objectives was to “Work with partners to support the development of a small set of anchor/model STEM high schools that will serve as laboratory schools and sites for professional development around project-based learning.” There is definite progress toward this goal, with three of the anchor schools working hard to improve instruction and implement STEM
features such as project-based learning, their STEM theme, and additional STEM courses, and also utilizing partnerships for improvement of student learning. The fourth school is welcoming their first students in the 2012–13 school year (with one-year delay). Based on analyses of RttT STEM initiative activities to date, the Evaluation Team concluded that structures for networking, professional development, curriculum development, and partnerships are in place to support both anchor and affinity schools as intended, though some of these activities have been delayed. A
summary of findings and recommendations for each of the four areas of implementation strategies and for the intermediate outcomes observed in the three anchor schools are presented here.

I. Structure of the Network of Stem Anchor and Affinity Schools


Baseline characteristics of the RttT-funded STEM schools

  • Prior to the initiative’s launch, RttT STEM schools offered a lower proportion of advancedSTEM courses than did the average high school in the state. In most cases, student achievement in RttT STEM schools was not notably different from all other high schools; however RttT STEM school physical science EOC scores did tend to be higher.

  • Since its launch, the RttT STEM initiative has made progress toward its goal of serving minority and poor students, who are traditionally underrepresented in STEM fields. In 2010–11, North Carolina RttT STEM Affinity Network schools served a higher proportion of black and Hispanic students and a higher proportion of students of poverty than did the average high school in the state, hosted the same proportion of female students, and were more likely to be located in an urban area.

  • Also, while faculty credentials and experience were similar across RttT STEM Affinity Network schools and all other high schools, per-pupil expenditures for STEM schools typically were slightly higher on average, and school sizes often were smaller.

Face-to-face and online networking

  • NCNSP has encouraged and facilitated networking and collaboration by various means, including embedding it in face-to-face PD events, furnishing online collaboration tools, and providing coaching services. Currently, face-to-face meetings have been the most successful networking channels.

  • Networking among schools in the STEM network is still in the early stages. Some schools have been networking with other schools outside of the RttT network.

  • NCNSP provides multiple opportunities for online collaboration. Edmodo, the original online network for STEM schools, has not been actively used.

Infrastructure developed for schools and their partners to share resources

  • As part of the RttT initiative, the NC STEM Learning Network was created and provided a number of services and products, though some of the main products and services have not been finished and require additional sustainable funds to continue in operation.

  • There has been little collaboration between the NC STEM Learning Network and the NCNSP STEM network.

Recommendations:

  • Leadership coaches should consider making increases in advanced math and science courses a possible emphasis for conversations with administrative teams in RttT STEM schools.

  • Implementers should consider various strategies for increasing the appeal of and incentives for visiting a virtual networking hub, including moving some PD elements for content and instruction into the online space, and encouraging instructional and STEM coaches to create online groups for following up on face-to-face visits.

  • In order to increase the effectiveness of sharing best STEM practices and resources, the NCNSP Affinity Network and the North Carolina STEM Learning Network should consider a better coordination of their activities. Additionally, creating a central hub (or portal), with access to content resources, professional development, and assessment and lesson planning tools that could serve both networks, might increase the utility and effectiveness of online collaboration for both networks.

II. Professional Development

  • Schools are receiving the PD and coaching services outlined in the scope of work.

  • Most of the coaching visits to comprehensive schools that joined the network in 2011 happened in 2012, and the number of visits per school was unevenly spread among schools.

  • Overall, PD and coaching were seen as valuable and of high quality. Staff at the anchor schools hoped for continuing PD and coaching in the upcoming year.

  • Professional development was most appreciated when participants understood its direct application to their classroom.

  • The vast majority of coaching time was spent on changing instruction in the classrooms.

  • The fact that coaches engaged with schools over an extended period of time gave coaches, teachers, and principals the opportunity to develop trusting relationships that likely increased the coaches’ impact.

  • Challenges and barriers related to PD included:

    • Sending teams to off-site PD during the school year for schools with small staffs;

    • Balancing the competing demands of different RttT initiatives; and

    • Getting buy-in from teachers around changing instruction.

Recommendations:

  • Much of the professional development was perceived by recipients as relevant, but NCNSP may want to explore ways of increasing the relevance of the lowest-rated sessions.

  • Because the impact of the coaches increased the longer they worked with teachers, implementers should consider having coaches in larger schools focus initial efforts on working intensively with a sub-set of teachers, instead of working with the entire faculty.

  • To better leverage professional development and coaching resources and to create incentives for using online networking, the Implementation Team should consider blended professional development.

III. Development and Implementation of Project-Based Curricula

  • A new contract was awarded to the North Carolina School of Science and Mathematics (NCSSM) by NCDPI to design STEM curricula with project units. Between July and August 2012, NCSSM delivered the outlines for all 16 year-long courses and the first units for the four freshman courses in each of the four themes (Aerospace, Security and Automation, Biotechnology and Agriscience, Energy and Sustainability, and Health and Life Sciences).

  • NCNSP provided multiple opportunities for teachers to engage in professional development focused on the four themes and on project design and implementation.

  • Themes are being incorporated in anchor and affinity schools in a number of different ways, including special sequences of courses on a theme, integrating a theme in all core subjects, and blending two or more courses.

  • Three existing anchor schools started to incorporate both cross-curricular projects and projects within individual subjects.

  • Scheduling and teacher knowledge on project-based learning (PBL) were identified as challenges for project implementation.

Recommendations:

  • The initiative leads should consider identifying additional resources and supplementary funds to support piloting and revisions of and professional development for the 16 year-long STEM courses. NCNSP should consider identifying schools from each of the Affinity Networks that are willing to pilot the courses and provide feedback to the developers. In addition, NCSSM should share the units with Affinity Network schools at scheduled professional development events.

  • Based on teacher feedback, incorporation of themes and project design and implementation should be emphasized both in professional development and in resource development efforts.

  • Based on principal feedback, the Implementation Team should consider providing schools with tips and examples of schedules that allow for integration of themed and cross-curricular projects in the context of a regular school day.

IV. Partnerships

  • Industry Innovation Councils (IICs) for each of the four themes met quarterly to plan and provide support for the networks.

  • Industry and IHE partners provided expertise to school staff on themes and on relevance to local community economic development, and they also planned partnership activities with schools.

  • NCNSP, with the help of business partners, is developing a sustainable and replicable prototype model partnership to be implemented in the four themed networks.

  • Ongoing challenges for schools: building partnerships in rural areas; making partnerships more collaborative and hands-on; and developing teacher content knowledge in the theme and in teaching career-ready skills.

Recommendations:

  • The model for partnership building is currently being developed in one of the urban schools; the Implementation Team should consider examining specific issues faced by rural schools.

  • There are still a number of questions and issues related to partnerships that anchor schools need to resolve, such as the anchor school’s role in providing partners to other schools in the network, or in communicating between schools. The Implementation Team should devote more time both face-to-face and online to the anchors or other groups of schools with common issues and work together to resolve these issues.

V. Intermediate Outcomes for Students and Staff in Anchor Schools

  • In all three anchor schools, the initiative remains in the beginning stages of implementation.

  • Given the large number of the early college/STEM design features that schools have to implement, the anchor schools each start with different priorities, which are affected by their context and by principals’ preferences.

  • There is not yet universal buy-in into the STEM initiative among staff in the anchor schools.

  • All anchor schools added additional STEM courses, such as engineering, technology, science, and health sciences; some schools are adopting more innovative math and science textbooks.

  • Technology is a high-priority area in all three schools, both as a subject of study and as an instructional tool for learning content across subjects.

  • Many teachers report that they improved their instruction and implemented instructional strategies emphasized by NCNSP professional development, such as collaboration, classroom talk, inquiry and project-based learning, and higher order questioning.

  • Interviews with staff and students indicated that students in anchor schools enjoy personalized attention and exhibit high motivation, engagement, and passion for learning.

  • Staff identified a number of challenges to overcome during implementation, such as better defining and understanding the STEM model, improving teacher qualifications, increasing student preparedness, and addressing logistical issues.

Recommendations:

  • In acknowledgement of the struggles faced by many participating schools to define what this initiative means for them and how to integrate multiple initiatives from the state, district, and NCNSP, the Implementation Team should consider providing more differentiated help to schools by staggering emphasis on different Design Principles and STEM features, depending on each school’s context.

  • To help schools faced with logistical issues related to their conversion or start-up, the Implementation Team should create resources and an online blog or discussion devoted specifically to those issues.

  • Implementing the STEM initiative’s more innovative components such as thematic and cross-curricular projects requires that teachers gain substantial new knowledge about both content and instructional strategies. The Implementation Team should consider differentiating ways of providing professional development devoted to these issues.

Next Steps

  • Continue to track changes in the demographic, financial, and academic measures of RttT STEM schools through the administrative data, identify the degree to which any changes are related to efforts connected to RttT, and use this evidence to determine progress toward the stated goals of the North Carolina RttT STEM initiative.

  • Continue qualitative data collection and analyses.

  • Analyze responses to staff and student surveys that were collected in Spring and Fall of 2012 to provide baseline data.

  • Provide a more detailed report about RttT-funded NCSSM curriculum development activities.

View Resource

Full Report PDF Full Report PDF

Projects

Evaluation of Race to the Top

This evaluation was designed to provide formative feedback for program improvement and determine impact on the target goals of each initiative and on overall state-level outcome goals.

Published

November 1, 2012

Resource Type

Report

Published By

Consortium for Educational Research and Evaluation–North Carolina