PISA math score debate among education experts centers on poverty and teaching

I’ve been enjoying the posts on how to interpret the PISA math scores. Everyone agrees that average math performance among American 15 year olds is disappointing with the US ranking 36th among 65 nations and subregions.

Michael Petrilli wrote a piece, PISA and Occam’s Razor, arguing that poverty might not the reason the US fares so poorly and thinks, perhaps, there’s a problem with teaching. “Maybe we’re just not very good at teaching math, especially in high school.” 

There’s been an emotional, impassioned rebuttal, arguing that poverty is what is dragging the US down and we would otherwise be excellent. Do not blame the teachers.

On Diane Ravitch’s blog Daniel Wydo Disaggregates PISA Scores by Income makes the case that poverty is to blame by isolating rich schools in the US. If you looked at only schools in which fewer than 10% of the population is poor, the US would rank #1 in reading, #1 in science and #5 in math.

Bruce Baker’s School Finance 101 blog takes on Petrilli’s in this post, showing clearly that poverty affects math scores.

Of course poverty matters. The US has a real problem educating poor children. The gaps are clearly worse in high school than in elementary school. The rich-poor gap in the United States is bigger than the rich-poor gap in many other countries.  But we also have a real problem with our top students.

The flaw in the Wydo analysis is that it’s silly compare students from the richest schools in America with the entire mass of another country. The PISA report clearly states that there are more variations within countries than between countries. (In Lichtenstein, a top performing country, roughly 250 points separate top and bottom students. That’s more than the point difference between top ranked Shanghai and bottom ranked Peru). Every country’s mean score is weighed down by its poor students and its unfair to compare your best students with the average student elsewhere.  The fair comparison would be to compare rich schools in the United States with rich schools in Switzerland and the Netherlands. And I think you would find (I don’t immediately see the income data on the PISA website to do this number crunching) that the US would NOT compare favorably.

I think this because PISA serves up data on the 90th percentile in each nation and top American students are also below average. So poverty alone cannot explain mediocre math performance.

My conclusion is that it’s not an either or. Perhaps the US has two separate problems. One is poverty. Two is high school math education.

Related story:

Top US students lag far behind top students around the world in 2012 PISA test results


POSTED BY Jill Barshay ON December 12, 2013

Comments & Trackbacks (8) | Post a Comment

X

How about you actually spend a week in a classroom and find out. Education is one of the areas that people love to conjecture about and propose a ton of theories, throw in their own agendas, and mix up with their own childhood (mis) memories. Instead of debating, get in a classroom for a week. Then another week in another one. Soon the answers you avoid will be clear.

David

Studies are bound to inherent flaws when comparing disparate systems of education. Although studies conducted by organizations, such as the International Association for the Evaluation of Educational Achievement (IEA), report on results of students taking the same tests, they overlook in their analyses how and when students are taught different concepts. American students, as note Berliner and Biddle for instance, are “more broadly educated than are students elsewhere” (p. 53) and as a result will not have “as much detailed knowledge of specific academic subjects” at certain phases of their educational experience. American students typically stay in school longer than students do in other countries. As a result students in the US acquire detailed knowledge over time and typically “end up with a knowledge base that is uniquely broad as well as deep” (Berliner & Biddle, 1995, 53).

Berliner & Biddle (1995) provide a clear example of how student results on comparative studies are misleading by reporting and commenting on the IEA’s Second International Mathematics Study. It found that the achievement of 8th grade Americans “lagged behind that of students in many other countries, notably Japan” (p. 55). The author’s express however that the test measured in the study emphasized algebra and that Japanese students take courses that stress algebra in the 8th grade, whereas American students take courses that stress algebra a year or two later. American students scored lower on the test because they were not yet taught algebra in depth.

Jill Barshay

@X I personally spend many hours in classrooms, as do my colleagues. It’s one of the things we take pride in at The Hechinger Report.

Jill Barshay

@David It would be fascinating to see how American students compare internationally at age 17, just before high school graduation, with 17 year olds abroad. My understanding is that PISA sets 15 years old as the testing age because that is the final age of compulsory schooling in some nations. Nonetheless 15 seems like a fair age for the questions that I saw on the PISA website sample. Indeed, even the most advanced questions on the sample didn’t require much formal algebra. But they do require clear, logical thinking and comfort translating word problems into expressions or equations.

Jim

“The flaw in the Wydo analysis is that it’s silly compare students from the richest schools in America with the entire mass of another country.”

Not when the child poverty rate of countries are completely engulfed by others.

For example based on OECD standards, Finland has a child poverty rate of 5% while the U.S. at 23%. It would be silly to compare across countries without disaggregating. Our middle class kids’ and even our upper class kids, are the norm in Finland, especially given the fact that our elite go to private schools and are not included in the PISA sampling while there are not private schools in Finland. All kids go to public schools, even their very elite.

Free/reduced lunch rates allow us to roughly compare apples to apples.

It’s not silly, it’s very logical.

Jim

“Every country’s mean score is weighed down by its poor students and its unfair to compare your best students with the average student elsewhere.”

Yes, but we have many more poor kids, relatively speaking.

Our best have a comparable socioeconomic status as their normal students or even less-than-normal students (in terms of childhood poverty).

Jim

“The fair comparison would be to compare rich schools in the United States with rich schools in Switzerland and the Netherlands. And I think you would find (I don’t immediately see the income data on the PISA website to do this number crunching) that the US would NOT compare favorably.”

No, that’s not the fairest way to do it. First, in may European countries, and elsewhere, there are not private schools enlisting the elite. For instance, even in Finland, there are no private schools. This means that even their richest students attend public schools. Obviously, in the U.S. this is not true. Our 10% richest, most elite kids attend private schools in the U.S. some charging upward to 30K to 40K a year. Some obviously don’t charge anywhere near that, but you get the point – even a private school that charges 5K a year is going to attract a wealthier family than what we get in U.S. public schools where 50% of students across the U.S. qualify for free/reduced lunch based on the federal guidelines.

Second, regardless of these facts, I think we get fairly close just by pulling out America’s richest kids in schools receiving less than 10% free/reduced lunch remembering that these kids typify whole countries of kids like Finland.

Jim

“I think this because PISA serves up data on the 90th percentile in each nation and top American students are also below average. So poverty alone cannot explain mediocre math performance.”

Wrong on two counts. First, OECD calculates percentiles based on all of a country’s sampling. So just because the U.S. has a lower number of students scoring in the highest percentile, remember that this is a percentage and it is calculated out of the whole. So the fact that the U.S. has so many poor kids, which are counted in the “whole”, of course we will have less and less students scoring in the 90th percentile.

Second, and I’m not sure you exactly alluded to this fact, but the OECD also attempts to dissagregate PISA scores based on poverty, but it is a crude measure based on what they call an economic index. This index is highly unreliable and considers some very subjective things such as goods and other ‘things’ that students have at home. So any measurement the OECD uses is much rougher than disaggregating based on free/reduced lunch rates which is strictly defined by income/wealth of parents, not subjective items that may/may not indicate relative poverty of students.

Your email is never published nor shared.

Required
Required