Student Attitudes Toward STEM (S-STEM) Survey: Tips for Using Your Data
What the S-STEM Survey Measures
The S-STEM Survey invites students to give information about their attitudes toward science, technology, engineering, and mathematics subjects, post-secondary pathways, and career interests.
The Research/Evaluation/Continuous Improvement Process
These 5 steps are a useful process by which individuals/teams can organize efforts to learn about the impact of their program on participating students. (1) Develop Research Questions: determine important questions you want to know about program activities and impacts. (2) Identify Data: data that can help answer those questions, anything from a set of informal conversations with participants to student test scores. (3) Collect Data: administer instruments and gather data. (4) Analyze Data: Figure out the themes or patterns, or results, in the data. (5) Interpret Results: Decide what those themes or patterns mean about the program.
Asking Important Questions About Your Program
If you do not already have question(s) that are important to you, or if you would like to brainstorm other possible questions, consider reflecting on your program’s mission statement, logic model, or questions that other groups have (parents, school boards, funders, etc.). What do you want to know about your program?
Keep in mind that it can be much more informative and critical to uncover the outcomes of your program, instead of the outputs. Outputs are easily measurable products of a program’s activities, often expressed in units such as hours of training provided, numbers of people participating, numbers of completed actions, etc. Outcomes are the impacts of a program’s activities, often expressed in terms of changes in behaviors, beliefs, or practices. Outcomes are usually measures of the broader, deeper goals of a program. For example, the number of middle school students enrolled in a summer engineering education program is an output, the change in those students’ interest in engineering is an outcome.
Collecting Data
Results from the S-STEM Survey gives information on student attitudes toward STEM from only one angle: a survey. Consider collecting other information about student attitudes to see if the results are similar – if they point to a common conclusion. Interviews or observations, for example, are other angles from which you can try to understand student attitudes toward STEM.
NOTE: If you are receiving S-STEM Survey data from the Friday Institute, it will not contain any identification of students. This is because the Friday Institute is an organization outside of the school system and cannot identify any individuals who responded to any questions; it can only report grouped/aggregated results.
“Cleaning” Data
Don’t forget to allow time for this. If you have raw data, especially survey data, test score data, or other sets of numbers, you will most likely have to make sure all of your numbers are correct and in the right place in your spreadsheet before you analyze them. NOTE: save copies of all original datasets and spreadsheets before doing any cleaning.
Common data cleaning steps include: fixing formatting when downloading data from one program into another (e.g. from Survey Monkey into Microsoft Excel); deleting empty rows or columns for people who may not have usable information (e.g. if they started but did not complete the survey); removing any personally identifying information or combinations of information, like name or school ID number; removing any data you won’t use in your analysis.
Looking for Themes/Patterns in the Data, or “Analysis”
The S-STEM Survey has 6 sections (see companion document “Student Attitudes toward STEM Survey – Development and Psychometric Properties”). To get the most thorough measure of student attitudes, the questions in 4 of the sections of the survey – Math Attitudes, Science Attitudes, Engineering and Technology Attitudes, and 21st Century Learning Attitudes – should be summarized together, by section. For example, to get the most thorough understanding of a single student’s attitudes toward, say, “math,” their responses to all 8 questions from the section “Math Attitudes” should be averaged. Assign the value of “1” every time a student responds “strongly disagree;” “2” for “disagree;” “3” for “neither agree nor disagree;” “4” for “agree;” and “5” for “strongly agree.” Average those numbers together for all of the student’s responses in the Math Attitude section and you’ll have a “score” for that student for their Math Attitude – the higher their score, the more positive their attitude toward math. You can average all students’ scores together to get a score for a whole class, grade, or school (as depicted below).
| STEM Attitudes | Upper Elementary (n=100) | Middle/High (n=200) | All Students (n=300) |
|---|---|---|---|
| Math Attitudes | 3.6 | 3.3 | 3.5 |
| Science Attitudes | 3.5 | 3.4 | 3.4 |
You can also compare answers to individual survey items that you find interesting, however strong conclusions about at student’s attitude should not be made from their responses to just a single question. The surveys have been validated at the construct-level, not at the item-level; therefore it is most recommended that comparisons be made at the construct-level.
NOTE: Most survey questions are positively worded, like “I am good at math.” A few, however, are negatively worded, like “Math is hard for me.” The negatively worded questions must be assigned values in the reverse order of all the other questions (“5” for “strongly disagree,” “4” for “disagree,” etc.), since agreement to those questions represents an attitude opposite of the attitude for agreement with the other questions. Refer to the companion document mentioned above to see a list of negatively worded S-STEM Survey questions.
You can also combine response options and report percentages of students who were generally positive (e.g. responded either “Interested” or “Very Interested”), for example.
| Career Area | Proportion “Interested/Very Interested” | ||
|---|---|---|---|
| Female (n=150) | Male (n=150) | All Students (n=300) | |
| Veterinary Work | 66.6% | 36.4% | 51.4% |
| Medicine | 61.6% | 38.8% | 50.2% |
More Analysis Tips
- Consider common ways to summarize data points – mean, range, frequency counts, percentages, median, and mode.
- Report the number of responses related to summarized numbers (“n” for subsample; “N” for the entire population).
- All analyses discussed in this guide can be done in the common software application Microsoft Excel.
- Create levels of units that reflect what you want to know – do you want student-, classroom-, or school-level results?
- Focus on comparisons – by sub-groups of students (gender, race/ethnicity) or time (beginning vs. end of year).
Reporting Data
Reporting data is a balance between showing every single point of data and combining them (averages, combined response options like agree/strongly agree, etc.). You have to decide what is best. It is important to give the audience a clear picture and not to be misleading, while also summarizing the data so the main themes can be easily understood and the presentation is not overwhelming. Do not be afraid to point out limitations in the data.
Pie chart: Show smaller groups in one larger group (e.g. the number of males and females in a program).
Bar chart: Show different findings, organized by categories, within a group or across groups.
Line graph: Show how results change over time, within a group or across groups.
Using and Sharing Data
Results – the patterns and themes found – can be used to decide to take new action in your school or program; to assess the degree to which goals are being met; to inform others about impacts of the program, including program participants, program staff, parents, school boards, etc.; to complete grant proposals; etc. Importantly, remember to share results with those who provided the information (who responded to surveys, etc.) – this helps promote buy-in and future collaboration.
View Resource
PDF Related Resources
The Middle/High School (6-12th) S-STEM Survey is intended to measure changes in students’ confidence and efficacy in STEM subjects, 21st century learning skills, and interest in STEM careers. The survey is available to help program coordinators make decisions about possible improvements to their program.
The Upper Elementary School (4-5th) S-STEM Survey is intended to measure changes in students’ confidence and efficacy in STEM subjects, 21st century learning skills and interest in STEM careers.
Projects
Maximizing the Impact of STEM Outreach through Data-Driven Decision-Making (MISO)
MISO is a campus-wide project that seeks to determine the collective STEM impact of NC State University through its pre-college outreach and extension programs.
Published
September 20, 2012
Resource Type
Survey Instrument
Published By
Friday Institute for Educational Innovation