Student Attitudes Toward STEM (S-STEM) Survey: Development and Psychometric Properties
The S-STEM survey invites students to give information about their attitudes toward science, technology, engineering, and mathematics subjects, post-secondary pathways, and career interests. Two versions have been developed: one for 4-5th graders (the “Upper Elementary S-STEM”), and one for 6-12th graders (the “Middle/High School S-STEM”).
| Survey Section | Psychometric Profile | Negatively Worded Items | Measurement Application |
|---|---|---|---|
| Math Attitudes | construct | Item # 1, 3, 5 | attitudes toward math – consists of items measuring self-efficacy related to math and expectations for future value gained from success in math |
| Science Attitudes | construct | Item #8 | attitudes toward science – consists of items measuring self-efficacy related to science and expectations for future value gained from success in science |
| Engineering and Technology Attitudes | construct | none | attitudes toward engineering and technology – consists of items measuring self-efficacy related to engineering and technology and expectations for future value gained from success in engineering and technology |
| 21st Century Learning | construct | none | attitudes toward 21st century learning – consists of items measuring student confidence in communication, collaboration, and self-directed learning |
| Your Future | items | none | interest in 12 broad categories of STEM career fields |
| More About You | items | none | predication of future academic performance, plans to take advanced classes, interaction with STEM professionals, postsecondary plans |
The math attitudes, science attitudes, and engineering and technology attitudes constructs were adapted from a survey developed for a female, middle-school-student, engineering program (Erkut and Marx, 2005). The careers section was further developed using the Bureau of Labor Statistics’ Occupational Outlook Handbook (2010-11). The 21st century learning attitudes construct was adapted from the Friday Institute’s Student Learning Conditions Survey (2010).
Validity and Reliability
Phase I. The Middle/High School S-STEM was piloted on 109 students. Several analyses were conducted on the items measuring attitudes toward science, math, and engineering and technology to identify constructs. Exploratory factor analysis was used, applying principal axis factoring and promax rotation to allow factors to be correlated. Item loadings above .40 were classified as significant. Five subject matter experts rated each item as “Essential,” “Useful but not
Essential,” or “Not Necessary,” and Lawshe’s content validity ratio was calculated for each item. Survey-takers’ open- ended responses to “do you have any suggestions for how we can improve this survey?” were collected and analyzed for themes. Finally, engineering education experts assisted in rewriting the engineering and technology attitudes section. New items that measure technologist roles were added and items containing gender bias were revised or removed. In addition to these changes, seven other survey items were dropped and several others were rewritten. Improvements were made to the “Your Future” section as well. Originally consisting of 43 items measuring student interest in as many specific STEM professions, the pilot section was edited based on factor analysis, expert feedback, participant feedback, and literature review. This revision process resulted in a “Your Future” section measuring student interest in 12, broad, STEM career fields. In addition to these survey improvements, an Upper Elementary version of the S-STEM Survey was developed based on the revised Middle/High School S-STEM survey. Cognitive interviews were conducted with five 5th graders and information gathered was used to revise wording for improved decoding and comprehension.
Phase II. The revised Upper Elementary (4-5th) and Middle/High (6-12th) S-STEM Surveys were administered to 799 fourth through fifth grade students and 9,081 sixth through twelfth grade students. Results from another exploratory factor analysis (significant item loading at .40) indicated that two items in the “Math Attitudes” construct needed to be dropped. Four additional questions were dropped to improve survey length. Overall, results showed a clear factor structure with each survey section acting as a single construct. The constructs’ reliability levels, measured with Cronbach’s Alpha, are:
| Construct | Number of Items | Cronbach’s Alpha | |
|---|---|---|---|
| Upper Elementary | Middle/High | ||
| Math Attitudes | 8 | 0.85 | 0.90 |
| Science Attitudes | 9 | 0.83 | 0.89 |
| Engineering and Technology Attitudes | 9 | 0.84 | 0.90 |
| 21st Century Learning Attitudes | 11 | 0.87 | 0.92 |
Student reading levels were analyzed using both tests for differential item functioning and expert feedback. Ten upper elementary teachers rated each of the upper elementary items as “Too Easy (below grade level),” “Just Right (at grade level),” or “Too Hard (above grade level).” Seven middle and high school teachers did the same for the middle/high school survey. Both sets of experts uniformly indicated that the surveys were at an appropriate length and difficulty for students. Differential item functioning tests were conducted to assess the internal validity of the surveys. Results indicated that students in the different grade levels targeted separately by the upper elementary survey (4th and 5th graders) and the middle/high school survey (6th through 12th graders) comprehend the survey in a similar manner. For example, a 6th grader and a 10th grader similarly comprehend the Middle/High School S-STEM Survey. Results from these tests show that measurement invariance held at all five levels. Responses from male students and female students were also analyzed using differential item functioning. Measurement invariance held at the first three, most essential, levels. Lack of factor covariance invariance indicated that males and females view the relationships between STEM subjects differently.
For further details on the S-STEM survey validation, please see Unfried, Faber, Stanhope, & Wiebe (2015).
References
Bureau of Labor Statistics, U.S. Department of Labor. Occupational Outlook Handbook, 2010-11 edition.
Erkut, S., Marx, F. (2005). 4 schools for WIE: Evaluation Report. Wellesley Centers for Women.
Unfried, A., Faber, M., Stanhope, D. S., & Wiebe, E. (2015). The Development and Validation of a Measure of Student Attitudes Toward Science, Technology, Engineering, and Math (S-STEM). Journal of Psychoeducational Assessment. doi:10.1177/0734282915571160
The William and Ida Friday Institute for Educational Innovation. (2011). Governor Perdue’s North Carolina Student Learning Conditions Survey (SLCS): Survey Implementation Study. Raleigh, NC: Author.
View Resource
Related Resources
The Middle/High School (6-12th) S-STEM Survey is intended to measure changes in students’ confidence and efficacy in STEM subjects, 21st century learning skills, and interest in STEM careers. The survey is available to help program coordinators make decisions about possible improvements to their program.
The Upper Elementary School (4-5th) S-STEM Survey is intended to measure changes in students’ confidence and efficacy in STEM subjects, 21st century learning skills, and interest in STEM careers.
Projects
Maximizing the Impact of STEM Outreach through Data-Driven Decision-Making (MISO)
MISO is a campus-wide project that seeks to determine the collective STEM impact of NC State University through its pre-college outreach and extension programs.
Published
September 20, 2012
Resource Type
Survey Instrument
Published By
Friday Institute for Educational Innovation
Suggested Citation
Friday Institute for Educational Innovation (2012). Middle and High School STEM-Student Survey. Raleigh, NC: Author.