Schools are kind of like Congress. Most people claim they hate Capitol Hill, but they like their own representative. Similarly, people say the U.S. education system is broken, but they like the school that their kids go to. I’ve been doing alumni interviews for Brown for more than 15 years and my first question is always, “So, how do you like your high school?” One would think this is an opportunity to show off some critical thinking. But the answer is invariably something like, “I love it. My school is great.”
Nonetheless, more and more school districts around the country are spending gobs of money and time on surveying parents, students and teachers on what they think about their schools. In education lingo, they’re called school climate surveys. The federal government is helping funding them in 11 states. The idea is to have another data point besides test scores by which to assess schools and, hopefully, help identify areas that need improvement.
But how useful are these surveys really?
The Research Alliance for New York City Schools at the New York University Steinhardt School of Culture, Education, and Human Development took a hard look at the NYC survey to see what lessons could be learned.
The NYC School Survey is the second largest survey in the country, after the US Census. All parents, students and teachers in grades 6-12 are asked to complete it. Last year, in 2012, 476,567 parents, 428,327 students, and 62,115 teachers did.
The Research Alliance found that the survey results tell you more about the “differences between individuals within a school, and less information about how that school differs from other schools.” In other words, parents tend to give similar scores to whatever school their kid is attending. Students respond differently than parents, but also they tend to give equally high marks to bad schools as they do to good schools.
Teachers were much better at distinguishing which schools are higher quality. The Research Alliance suggests that teachers’ scores should be weighted more heavily.
For those who are interested in constructing these opinion surveys, the Research Alliance found that the NYC survey is too long and reports the results in an unnecessarily complex way. So many of the satisfaction questions were so highly correlated with one another that the survey could be cut in half and still produce as reliable a result. The authors also recommended that NYC just report a single “school environment” measure and stop reporting scores in four different categories: academic expectations, communication, engagement and safety. The four categories were deemed to be “statistically indistinguishable” from each other.
