Shaun Kellogg

The Friday Institute for Educational Innovation (The FI) at NC State University??????????????????s College of Education proposes to partner with The ASSISTments Foundation (TAF) in the ????????????????Recovering from COVID-Learning-Loss with a Platform to Support Human Tutoring??????????????? proposal to the US Department of Education??????????????????s Education Innovation and Research (EIR) Program - Mid-Phase Grant program. The proposal aims to scale a cost-effective way for schools to provide high dose human tutoring by matching volunteer tutors with students whose teachers are using ASSISTments mathematics instruction in the classroom - The TutorASSIST Program. In the first 2.5 years of the project, The Professional Leading and Learning Collaborative (PLLC) Team at The FI will develop, execute, and refine the TutorASSIST tutor training and credentialing and the scalable TutorASSIST program administration framework, materials, and administrator training for participating schools. Also, in the first 2.5 years of the project The Program Evaluation and Education Research (PEER) Team at The FI will conduct a formative evaluation of the TutorASSIST training and administration work, using a mixed methods approach based on the CIPP Evaluation Model and Chris Dede and Deborah Coburn's scaling educational innovations framework. In the final 2.5 years of the project The PLLC team will execute small revisions and refinements to the program design and will support hand-off of the program execution to TAF; and The PEER Team will hand-off relevant evaluation instruments and materials to the external evaluator for an impact evaluation study and will support publication and dissemination efforts.

The overarching goal of this project is to broaden the pool of researchers that leverage Learning Analytics (LA), a computational research methodology that transforms digital data into actionable insights for the purpose of understanding and optimizing learning, thus advancing our knowledge about STEM learning and learning environments. To achieve this goal, LASER BEAM will build upon the existing NSF-funded LASER Institute by refining, expanding, and repackaging current instructional resources ,and separately at the University of Pennsylvania (including the MOOC Big Data and Education, accessed by over 100,000 students), into ???turnkey??? curriculum materials that can be used and adapted by faculty in higher education to prepare the next generation of STEM scholars.

The primary goal of this work is to develop a comprehensive and responsive evaluation plan that details programmatic strategies and intended outcomes, measures for assessing and improving program impact, a timeline of evaluation activities, and an estimated budget. Working collaboratively with TIP leadership, North Carolina Public School Units (PSU), and the North Carolina Department of Public Instruction (NCDPI), the Friday Institute Program Evaluation and Education Research (PEER) Group will ensure evaluation efforts for the NC HTLA are: 1.Aligned to the program goals and priorities of the TIP leadership team; 2.Inclusive of legislative and funder-specified reporting requirements; and, 3.Responsive to the context and needs of North Carolina Public School Units. A secondary goal is to document design, development, and implementation efforts during the first year of the initiative using an ethnographic approach. Reporting of these efforts is intended to inform program stakeholders and external audiences of the progress made towards program objectives, design decisions and rationale, and successes and challenges experienced by program leadership and participants.

Minimizing crop loss and increasing output, across the food supply chain, will increase the economic viability of US growers and the global economic competitiveness of industry and stakeholder partners. We have assembled a diverse team across different National and International Universities with faculty that have track records of convergent research, education, and outreach. We will be well positioned to implement a Networks of Networks with diverse backgrounds, ethnicities, genders, experiences, and disciplines to drive research and innovation. Students and postdocs will be exposed to hands-on learning, on-farm technology training, cooperative extension, commercialization, industry engagement, and transdisciplinary education to create a highly trained workforce that is equipped to address food security and safety challenges.

The purpose of the Track 3 Noyce Master Teaching Fellows program at NC State is to support and retain master teachers in mathematics by partnering with four high-need LEAs (Cabarrus County Schools, Mount Airy City Schools, Rockingham County Schools, and Wayne County Schools), and the non-profit agency, The Innovation Project (TIP) and NC State. This project dovetails with our collaborations with TIP to place highly qualified beginning STEM teachers from NC State in high-needs districts in the state of North Carolina. The project also builds on our experiences from two prior Noyce scholarship programs, to prepare students to become mathematics teachers, and a state NC Quest grant to support practicing high school mathematics teachers to implement new state standards for mathematics. Teacher shortages have been growing in North Carolina and across the country, especially in STEM fields and in high-needs school districts (Cowan, Goldhaber, Hayes, & Theobald, 2016; Podolsky, Kini, Bishop, and Darling-Hammond, 2016; Sutcher, Darling-Hammond & Carver-Thomas, 2016) Podolsky, Kini, Bishop, and Darling-Hammond (2016) identify two factors contributing to these shortages: the number of students choosing to pursue a career in teaching and the number of teachers leaving the profession. Sutcher, Darling-Hammond and Carver-Thomas, make three recommendations to address the teacher shortage: offer competitive salary packages, enhance the supply of new teachers, and improve teacher retention, especially in high-needs schools, through induction programs, mentoring, improved working conditions and professional development. While some efforts are currently underway in the state of North Carolina to encourage more students to become STEM teachers, little has been done to address issues related to teacher attrition.

The evaluation questions developed for this proposal are explicitly aligned with the strategic objectives of the NCEC. Collectively, the evaluation questions are intended to help NCEC better understand: Q1.How, and to what extent, does NCEC foster effective partnerships with schools to implement research-based programs that support teachers and help students thrive? Q2.How, and to what extent, does NCEC activate untapped talent by connecting Corps members to partner schools?????????????????? needs? Q3.How effectively does NCEC train corps members for high-impact tutoring, and how can training better meet the needs of corps members and partner schools? Q4.How effective are ongoing supports provided to corps members, and how can NCEC better support members in-service and in pursuit of public service careers? Q5.To what extent does NCEC implement high-impact interventions that improve early literacy and social-emotional outcomes for students? Q6.How can NCEC improve and grow program services to better serve public schools and children in need?

In 2018, the North Carolina Department of Public Instruction (NCDPI) was awarded a Public Charter Schools Program (CSP) grant from the U.S. Department of Education of approximately $26.6 million. The Department was awarded an additional $10 million in 2019. The CSP grant will be used to implement the North Carolina Advancing Charter Collaboration and Excellence for Student Success (NC ACCESS) Program to: 1. Increase the number of educationally disadvantaged students attending high-quality charter schools and expand the number of high-quality charter schools available to educationally disadvantaged students; 2. Develop a cohort of charter school leaders who can develop and demonstrate best practices in serving educationally disadvantaged students; and 3. Broadly disseminate best practices in serving educationally disadvantaged students and foster collaboration in the charter school community and between charter schools and traditional public schools.

The goal of the Learning Analytics in STEM Education Research (LASER) Institute is to increase the capacity of researchers to understand and improve STEM learning and learning environments through the use of new sources of data and analytical approaches. Specifically, the LASER Institute will focus on preparing scholars from underrepresented groups in STEM to: 1) understand the methodologies, applications, and ethical issues of Learning Analytics as it relates to understanding and improving STEM education, and 2) gain proficiency in R, a popular free software environment for statistical computing and graphics, to apply computational analysis techniques (e.g. network analysis, text mining, machine learning, and data dashboards) using real-world data from STEM contexts. The LASER Institute will be anchored by a week-long intensive training program consisting of hands-on learning labs, presentations from experts in the field, support for research planning and analyzing their data, and opportunities for networking and collaboration.

The Friday Institute Research and Evaluation Team will provide evaluation services for a proposed project entitled, The Undergraduate Network for Increasing Diversity of Ecologists (UNIDE) project. This research coordination network project aims to build a sustainable and interdisciplinary network of ecologists, educators and social scientists to address the need to address how cultural and social barriers impact human diversity in ecology and environmental disciplines (EE). The research network coordination project seeks to build cultural competence into ecology and environmental biology education.

The purpose of the teacher compensation models and advanced teaching roles program is to allow highly effective classroom teachers to teach an increased number of students by assuming accountability for additional students, either by becoming a lead classroom teacher accountable for the student performance of all of the students taught by teachers on that lead classroom teacher's team, or by leading a larger effort in the school to implement new instructional models to improve school-wide performance. In addition, the program enables local school administrative units to provide salary supplements to classroom teachers in advanced teaching roles to create innovative compensation models that focus on classroom teacher professional growth and student outcomes. The purpose of this evaluation is to assess the impact of the teacher compensation models and advanced teaching roles program on teaching and learning as described in North Carolina General Assembly Session Law 2020-78, Section 2.6(b). In addition, this evaluation includes a comparative analysis of program implementation in order to help identify and scale the most effective components of these programs. To provide a more complete understanding of the implementation and impact of these programs, the evaluation will use a mixed methods convergent design, in which different but complementary data are collected concurrently or sequentially. The evaluation will be conducted in several phases and will make use of secondary data collected by NCDPI to provide quantitative estimations of the program???s impact on select teaching and learning outcomes, as well as primary data collected directly from Public School Units in order to examine the similarities, differences, and patterns across program implementation.

The proposed implementation project will enable Department of Information (DIT) Broadband Infrastructure Office (BIO) and ORH to develop a comprehensive, innovative, multi-faceted program with regional and local partners to increase access to healthcare for vulnerable populations and disadvantaged groups while simultaneously increasing the economic viability and digital skills for residents seeking to re-enter the workforce and those currently employed. DIT will contract with the Friday Institute of Educational Innovation at North Carolina State University (NCSU) to conduct a formative evaluation of the project implementation and provide data to inform the final playbook development and track outcomes and measures throughout the project.The Friday Institute Evaluation Team will work collaboratively with the project partners to develop a comprehensive evaluation plan to provide formative and summative feedback about the implementation and impact of the grant that align to the goals and objectives. A real-time evaluation approach will be used to assess the quality and effectiveness of the project in rural North Carolina. As the name suggests, this approach to evaluation will offer formative, in-the-moment feedback at critical decision points in the program with an initial goal of improving instead of proving effectiveness (Nolan and Fontane, 2012). This allows the program to use evaluation data to guide and improve its strategies in real time, rather than waiting to make important changes after the program??????????????????s conclusion.

The Friday Institute for Educational Innovation, in partnership with the Social Science Research Institute (SSRI) at Duke proposes an evaluation on behalf of the Office of Learning Recovery and Acceleration (OLR). The aim of the evaluation is to provide actionable information to the OLR and the larger North Carolina education community to assess the implementation and impact of the summer programs designed to support student re-engagement and academic recovery. In alignment with the call for qualitative research partners, data and analyses generated by this evaluation are intended to support decision-making around the targeting, design, and continuation of future programs.

The purpose of this study proposed by WestEd International and its partners is to evaluate the efficacy of the use ASSISTments, an online tutoring system that provides coached practice problem-solving support for students and cognitive diagnostic reports to teachers, to support students?????????????????? mathematics homework completion, differentiated instruction, and ultimately students?????????????????? understanding of mathematics. The study will leverage individual student?????????????????? access to computers to implement the ASSISTments homework support program.

To inform collective understanding of STEM learning in informal environments and to ensure that the program is implemented as intended, the team proposes a robust process-focused evaluation since outcome analyses conducted in isolation may not provide sufficient information to determine which program characteristics contributed to success or failure. Process evaluation also can inform efficient implementation at new sites. Ongoing program evaluation will align with AISL program requirements for evaluation.

The PEER Group proposes to work closely with SMT Center leadership and staff to edit the NC STEM Education Strategic Plan for 2030 into versions targeted for public consumption. These public audiences may include policymakers, the general public, educators, etc

The Friday Institute for Educational Innovation (FI) at North Carolina State University (Friday Institute), will collaborate with the University of North Carolina (UNC) System, the North Carolina Independent Colleges and Universities (NCICU) and the North Carolina Department of Public Instruction (NCDPI) to develop a rubric to be used to review and assess the professional development modules submitted to DPI by the NCICU and UNC Work Groups. In addition, the FI will provide training and support for assessors who will be using the rubric to evaluation the professional development modules.

The purpose of the proposed development and evaluation work conducted by the Friday Institute will be to support the Innovative Assessment system through the provision of professional learning for educators and conduct applied research to support continuous improvement. The professional learning and continuous improvement approach will be implemented with the pilot schools and districts and then with the broader group of districts and schools. Goals for the project include: 1. Professional Development. To develop a blended professional development program that will support teachers, coaches, principals and district leaders in implementing the Innovative Assessments, including why the assessments are important for instruction and student learning and how to use data in a systematic way to inform teaching and learning. This will include developing capacity in coaches and district leaders in supporting teachers, including strategies for implementation. 2. Strategies and Tools. To work in partnership with DPI to create a multi-year implementation plan for delivering and supporting the professional learning and communication resources to support outreach to stakeholders. 3. Regional Supports. To share and review deliverables with the DPI Regional Case Managers to identify possible gaps or anticipated needs prior to the implementation with the pilot cohorts. 4. Program Evaluation. To develop a comprehensive evaluation plan to support the continuous improvement of professional development efforts and gather stakeholder feedback and data that will be used to guide development of the assessment system, professional development program, and other aspects of the project. 5. Reporting & Dissemination. To deliver timely, valid, actionable feedback to guide innovative assessment and professional development efforts and to inform internal and external stakeholders of the program??????????????????s progress, anticipated challenges, and opportunities.

The purpose of this work is to provide support for districts, schools, and educators in NC during this rapid transition to remote learning through the creation and implementation of guidelines for remote instruction, professional learning opportunities, and model lessons. The goals of this work will be to develop tangible guidelines for remote learning to support educators in their development of lessons and activities for students, including options that are with or without devices and the Internet (online, blended, non-digital). -To work with DPI to publish and share the comprehensive set of professional learning opportunities, including the work being led by the Remote Learning, Professional Learning, and Teacher Teams. -To develop a series of professional learning opportunities that help teachers build capacity in transitioning their face-to-face instructional approaches to remote learning. This will include taking strategies used in a traditional classroom and creating options that will work from home, new strategies that they may not have used previously, and specific approaches to building SEL from a distance. The professional learning will consider different grade bands, students with diverse needs, and different subject areas. This will also include a small number of opportunities for school leaders in their development of school-wide SEL approaches. -To develop model lessons that can be used in the professional learning opportunities to demonstrate the guidelines in action and various strategies and approaches. This effort will focus on a small set of high quality model lessons rather than a large curriculum or lesson plan bank of resources.

The University of North Carolina at Chapel Hill (UNC) is collaborating with the FIRE Group to provide evaluation services for a proposed project entitled, Targeting Equity in Access in Mentoring (TEAM) ADVANCE. The project seeks to improve the quantity and quality of mentoring provided to early- and mid-career underrepresented (people of color and women) STEM faculty members in an effort to bolster their rates of success. PROJECT ACTIVITIES The TEAM ADVANCE Project Evaluation??????????????????s activities are intended to provide formative and summative feedback to program implementers. At a high-level, the activities include: - Developing a detailed evaluation plan - Meeting with TEAM ADVANCE quarterly - Administering project surveys electronically [Fall and Spring] - Conducting interviews [Mentor Coordinator(1), Mentors (2-4), Mentees(6-8)] - Collecting project artifacts (mentoring plans, mentoring professional development content/agendas, attendance rosters, etc.) - Completing data analysis - Crafting evaluation reports DELIVERABLES ??????????????? Electronic survey reports and qualitative data summaries based on the project timeline. ??????????????? Formative evaluation memos which synthesize multiple data sources will be generated twice per year (Fall and Spring) ??????????????? Summative annual report (2019, 2020, and 2021)

With the large majority of schools likely to continue remote learning well into the fall, this project aims to build upon the success demonstrated by Phases I of the At Home Learning initiative, which began by airing two blocks of NC standards-aligned PBS content each weekday on UNC-TV's North Carolina Channel. Phase II expanded upon these efforts to include both UNC-TV produced and teacher generated content featuring messages aired between programming, as well as printable and online materials and activity ideas curated by NCDPI to complement the UNC-TV programming. The purpose of this project is to expand the At Home Learning initiative into a third phase focused on the development of a more comprehensive set of remote learning curriculum materials and resources. Specific goals for Phase II of this initiative include: 1. Lesson Development. Recruit and support teachers to develop NC standards-aligned literacy lessons to be aired on UNC-TV??????????????????s stations (and DPI??????????????????s OER repository and DPI??????????????????s YouTube Channels). 2. Learning Supports. Work with participating teachers to develop one-page extension lessons that correspond to each video lesson filmed to be used by North Carolina teachers and families to support and extend broadcast and internet-based video lessons. 3. Research & Evaluation. Support continuous improvement and gather information on the implementation and impact of the At Home Learning Initiative. 4. Reporting & Dissemination. Report to DPI and UNC-TV on the activities, impacts, and lessons learned both formatively to help guide decision-making and summatively once all program activities have ended. ????????????????