On May 7, 2014 I argued that the stagnation of U.S. high school seniors’ test scores can’t be blamed on demographic changes because the scores of top students, the 90th percentile, have been flat too.
Fredrik deBoer wrote on his website that we should never be surprised by stagnant performance at the top because “there are likely ceiling effects at play.” That is, top students might already be acing the test and there’s not much more room for the students to grow. They can’t show gains. I posed that question to Arnold Goldstein, a statistician at the National Center for Education Statistics who works on the reporting of the NAEP test results. He said there’s no indication that top students are bumping their heads against any test ceiling. “The students aren’t scoring anywhere near the top of the NAEP scale,” he said.
Of course, there are more subtle aspects to ceiling effects. It’s possible that top students are acing all the algebra, geometry and calculus questions that they could possibly be expected to answer. Perhaps the topics they’re not getting right are ones that aren’t taught in high school, such as ring and field theory. You’d have to analyze the more difficult NAEP questions one by one, but I doubt that the exam makers have included advanced math topics intended for college math majors.
Liana Heitin at Education Week correctly points out that it’s not exactly apples-to-apples to compare percentile scores over time. Because of the demographic changes in the overall student population (more minorities, fewer dropouts, etc.) , it may be a lower threshold to make it into the top 10 percent. It’s quite possible that top 10 percent of students is a weaker 90th percentile today than it was in 1992.
Heitin, citing Morgan S. Polikoff, an assistant professor at the University of Southern California’s Rossier School of Education, called this kind of percentile analysis “misleading”.
Having lower scores at the 90th percentile doesn’t necessarily mean the top performers weren’t doing any better–it could just as well mean there was an influx of lower performers who brought all the scores down. So it wouldn’t take as high a score to be considered a high performer.
I’ve been paying extra attention to percentile comparisons since I started this blog. I’m quite interested in why top U.S. students appear to be falling further behind internationally as students in other countries make great leaps forward. Is the 90th percentile today a much weaker group of students who needed a far lower score to make it into the top tier? Is the top 10 percent of 30 years ago more like the top five percent today?
Related stories:
Top US students lag far behind top students around the world in 2012 PISA test results
