The New Tech Network issued a press release on Oct. 2, 2013, trumpeting that its high school seniors outperformed 68% of 4-year college freshmen with similar backgrounds and abilities on key indicators of higher order thinking skills according to the College and Work Readiness Assessment (CWRA). The New Tech Network employs a lot of technology in its classrooms. On its website, there are a lot of pictures of students smiling in front of laptops and desktop computers. So it would be amazing to learn if New Tech’s unusual mix of computerized instruction and project-based learning was really producing such results.
I clicked on the underlying report and the numbers were more confusing. Instead of the “68%” figure above, the report said that “NTN seniors outperform 77% of college freshman (sic) and 60% of other high school seniors when controlling for academic ability.” I added the italics for emphasis.
I’d never seen test scores controlled for “academic ability” before. Usually I see controls for socio-economic status, income or race so that you’re comparing kids with similar backgrounds. But isn’t it bizarre to say that NTN kids got higher test scores than other kids with similar test scores?
It was also strange to me that NTN seniors performed so much better than college students, but not quite as well against other students their age. (77% outperformance vs. 60% outperformance).
A footnote explained, “On average, students in New Tech Network schools have lower academic skills than those in the comparison groups; this is possibly explained by the fact that the CWRA sample of high schools consists largely of private schools and CWRA does not control for ethnicity or socio-economic status in their analysis.”
So if I understand correctly, NTN seniors are testing WORSE than comparison groups. But when you cherry pick the test questions that deal with “higher order thinking skills”, then NTN students do better on those questions than other students who did as badly on the overall test as they did. Do I have something wrong here?
