Skip to main content
William & Ida Friday Institute for Educational Innovation

NCVPS Blended Learning STEM Courses: First-Year Formative Assessment, Part II

Executive Summary

The Consortium for Educational Research and Evaluation–North Carolina is evaluating North Carolina’s use of Race to the Top (RttT) funds to develop a series of STEM-based courses to be delivered to underserved students through the state’s Virtual Public School (NCVPS) via a blended-learning model. The evaluation’s goals are to assess the extent to which this initiative contributes to: (a) the enrollment of underserved students targeted by the initiative; (b) the success of those students in the STEM courses offered; and (c) an increase in the availability of effective STEM teaching to students in high-need schools.

Purpose and Structure of the Report

This report—the second part of a two-part report on the first year of implementation—presents participant feedback from the second semester of implementation (January-June, 2013). This feedback supplements baseline measures from the first semester, provides data for additional first-year formative feedback to NCVPS in support of the growth and development of this initiative, and informs future evaluations of the NCVPS blended learning STEM courses.

As with Part I, this second report provides a formative review of preliminary results for a still-developing initiative in order to inform ongoing initiative improvements; it is not intended to serve as a statement about the anticipated quality of the final form of this initiative.

The NCVPS Blended Learning STEM Course Initiative

For this initiative, blended learning refers to a course that is taught by a local teacher in a traditional setting with the aid of a virtual co-teacher and the support of online materials. The overarching goal for the initiative is to increase the number of highly-qualified STEM teachers in low-income rural areas and low-performing urban schools by pairing current face-to-face STEM teachers in target schools with online STEM mentor co-teachers. Over the course of the initiative, NCVPS will pilot nine blended-learning STEM courses, beginning with the three courses first offered in School Year 2012-13. Each blended learning course consists of a sequence of project-based learning (PBL) units that focus student energies on solving challenging and complex problems that incorporate concepts from the curriculum of the course. Each course also is designed to align with one of the National Academy of Engineering’s Grand Challenges of Engineering.[1]

Updated Initial Observations and Findings: Spring 2013

Capacity

  • Spring 2013 participants and participation rates were similar to those from Fall 2012.
  • The three participating Local Education Agencies (LEAs) enrolled 135 students in three blended learning STEM-focused courses (Earth and Environmental Sciences, Integrated Mathematics I, and Forensics) in Spring 2013, a decrease of 12 students from Fall 2012.
  • Participants’ demographic data indicate that, collectively, the courses continued to enroll students from groups traditionally underrepresented in STEM fields (i.e., females and minorities). In Spring 2013, the percentage of females remained fairly stable overall. However, the percentage of ethnic/racial minorities decreased slightly.
  • Most participants continue to be 9th graders (73%). In Fall 2012, only 2% were enrolled in more than one RttT-funded blended courses, but in Spring 2013, 21% (n=29) of the participants were enrolled in more than one of the RttT-funded blended courses. Only 5%(n=7) of the Spring participants also were enrolled in a non-blended NCVPS courses.

Course Quality

  • Course Content, Grand Challenges, Pedagogy, and Technology Integration:
    • Content and Grand Challenges. Teachers and students expressed concern about the rigor of the courses, especially in terms of content coverage. Helping students to understand the relevance of what they were learning and the connections between content and the Grand Challenges (especially in the mathematics course) remained a challenge.
    • Pedagogy and Technology. Teachers reiterated concerns from the Fall about the quality and frequency of the pedagogically-focused professional development available to them.[2] Teachers and students noted that at least some degree of the courses’ success depended on students’ comfort with and proficiency in PBL and their facility with the provided technology. Teachers and students indicated that progress had been made in reducing the technology issues experienced in Fall 2012 but that there was still work to be done. Students were more open and provided longer interview responses in the Spring about the degree to which the availability of the technology could be a distraction, but observations suggested that technology use typically was on-task, and teachers believed that appropriate course-related communication via technology was expanding. Student-to-student project-related interactions were common for some but infrequent for others.
  • Student and Teacher Participation in the Courses
    • Teachers felt more empowered in Spring to shape and update course content and organization, based on their Fall teaching experiences. Face-to-face and online teacher pairs interpreted their respective roles in the courses in multiple ways, but almost all established some form of regular communication to share planning responsibilities.
    • Students communicated with both of their teachers, but for the most part still relied more heavily on their relationships with their face-to-face teachers and actively expressed a desire to interact more with online teachers.

Program Effectiveness

  • Teacher and Student Growth in the Student-Centered, Project-Based Environment: Most face-to-face teachers grew in their acceptance of the value of student-centered and project-based instruction, but their success in effectively implementing these strategies was mixed, and there was some concern about their appropriateness for the mathematics course. Some students embraced the approaches while others struggled with them.
  • Impact on Other Areas of Student, Teacher, and School Capacity-Building: Some teachers and students noted positive growth in some students’ academic toolkits (e.g., time management; future educational aspirations), though self-direction remained a challenge. Teachers also indicated that their participation in the initiative was beginning to impact their approach to teaching in their other classes. Some suggested (with cautious agreement from non-participating teachers) that the initiative’s impact was traveling to other teachers, and a few even considered the initiative to be a stepping-stone toward broader, school-wide changes.

Conclusions and Formative Recommendations[3]

  1. Continue to clarify roles and expectations for face-to-face and online teachers. Teachers and students alike continue to report uncertainty about face-to-face and online teacher roles and responsibilities in the blended setting. Latitude with respect to how different teacher teams engage in course-planning and delivery can be helpful, but provision of at least some initial additional clarity about the ideal distinction between face-to-face and online teacher roles can help reduce the uncertainty and strengthen the co-teaching relationship.
  2. Move from a focus on course content and delivery to a focus on teacher development. Much of the first year of the initiative was spent refining the content and delivery of the three initial courses, but one overarching goal of this initiative—as with other initiatives outlined in the same section of the state’s RttT plan—is to enhance equitable distribution of effective teachers across LEAs. This initiative ultimately contributes to that goal when participating face-to-face teachers experience growth as they work with online master teachers and explore the potential of a PBL approach. Currently, the initiative provides a space for that growth and the materials to support it, but there are as yet few formal mechanisms in place for direct mentoring from online master teachers to the face-to-face teachers. Face-to-face and online teachers work together, and some have developed collegial relationships, but these relationships (and any resulting knowledge transfers) are not yet supported by a formal and clear plan that ensures targeted growth for all face-to-face teachers.
  3. Enhance student orientation to and preparation for the blended, problem-based learning setting. The more time spent focusing on ensuring that students know how to operate in this environment, the greater the likely benefit in terms of final student outcomes. NCVPS might want to consider providing: (a) PBL training not only for teachers but also for students; (b) infrastructure for an explicit introduction between online teachers and students so both can navigate the blended setting more effectively; (c) formal progress monitoring mechanisms for 9th grade students above and beyond that currently provided by the teacher teams; and/or (d) an application or screening process for LEAs to help them gauge student readiness for the courses.
  4. Seek out and incorporate student and teacher feedback. The best arbiters of course success are those who are actively participating in the courses. Find ways to formalize teacher and student feedback into the course development process, as well as opportunities for both groups to provide feedback in real time for mid-course corrections.
  5. Revisit Grand Challenges integration. Particularly in the introductory courses (Integrated Mathematics I and Earth and Environmental Science), the inclusion of problems and projects based on Grand Challenges has met with mixed success. Consider incorporating Grand Challenges in later, upper-level classes and include instead more discrete and attainable engineering projects related to the formal Grand Challenges in earlier courses that may better foster initial student engagement and provide more immediately relevant and meaningful applications of the course material.
  6. Continue planning for life after Race to the Top. In addition to the current plan to offer the courses in three different formats (full blended model, online-only model, provision of course materials only) and at different price-points (from full subscription to free) when access to the initial courses is extended to all LEAs, also consider strategizing ways to make the courses financially supportable after RttT funding ends.

1 The Grand Challenges of Engineering are a set of 21st-century challenges identified by members of the National Academy of Engineering and other groups worldwide to serve as a framework for focusing engineering efforts at all levels of education and innovation: http://www.engineeringchallenges.org/
2 In response to these concerns, in Spring 2013, NCVPS began revising and extending the professional development available to teachers, with a particular focus on provision of more face-to-face professional development opportunities related to project-based learning.
3 Note: A response from North Carolina Virtual Public School to both the Updated Initial Observations and Findings: Spring 2013 and the Conclusions and Formative Recommendations sections is included in Appendix D of the main report.

View Resource

Full Report PDF Full Report PDF

Projects

Evaluation of Race to the Top

This evaluation was designed to provide formative feedback for program improvement and determine impact on the target goals of each initiative and on overall state-level outcome goals.

Published

October 1, 2013

Resource Type

Report

Published By

Consortium for Educational Research and Evaluation–North Carolina