The already muddy research on whether it’s better to hold back struggling students or promote them to the next grade just got muddier. A new study ,“The Scarring Effects of Primary-Grade Retention? A Study of Cumulative Advantage in the Educational Career,” by Notre Dame sociologist Megan Andrew, published Sept. 26, 2014, in the journal Social Forces is an empirically solid analysis that adds more weight to those who say retention — what education wonks call repeating a grade — is ultimately harmful.
Andrew mined two large data sets in a way no researcher has done before and concludes that kids who repeat a year between kindergarten and fifth grade are 60 percent less likely to graduate high school than kids with similar backgrounds, and even 60 percent less likely to graduate high school than siblings in the same family.
Before I discuss Andrew’s paper in more detail, it’s helpful to understand some history. Most early research overstated how harmful it is to be held back a grade. It tended to point out that the struggling kids who repeat a grade don’t fare as well as kids who stay with their class, most of whom are not struggling. But that’s shoddy research. These studies didn’t compare the held-back kids with the kids who were also failing, but were promoted nonetheless.
Related story: Why Los Angeles sends failing students on to the next grade
This article also appeared here.
In data analysis terms, this early research conflated the bad effects being held back with the bad effects of the underlying issue that led a school (or a parent) to hold the child back in the first place. Consider a child who has trouble paying attention, can’t read by the end of fourth grade and is held back. Say, this child continues to get bad grades, tests poorly and eventually drops out of high school. Did the stigma of repeating fourth grade cause the child to become demoralized and to perform worse at school? Or was it his ongoing struggle with attention deficit disorder? If he had been promoted, would his academic career turned out differently? These early studies don’t say.
Even as the low quality research kept showing that holding kids back was bad, a growing chorus of critics urged schools to end “social promotion,” the practice of passing failing students onto the next grade. As my Hechinger Report colleague Molly Callister wrote here, 15 states and the District of Columbia have adopted policies requiring third-grade reading proficiency before a student can move to fourth grade. Two big cities, Chicago and New York City, undertook ambitious experiments in ending social promotion.
Those urban experiments attracted sophisticated researchers. Brian Jacob and Lars Lefgren studied students in Chicago, where the decision to hold a student back was based on a test score. The researchers were able to compare the experience of students who scored just below the threshold for passing with the experience of students who scored just above the threshold. Because of test measurement errors, these students were effectively testing at the same level — academically identical. But half were held back and half were promoted. In a 2009 paper, Jacob and Lefgren found that the harmful effects of retention largely melted away when comparing these two groups of students. Students held back in older grades still suffered a bit, but there was no decrease in high school graduation for students who’d been held back young. (Jacob, Brian A., and Lars Lefgren. 2009. “The Effect of Grade Retention on High School Completion.” American Economic Journal: Applied Economics, 1(3): 33-58.)
Four years later in 2013, a RAND study looking at New York City’s experiment with ending social promotion came to a similar conclusion — retention isn’t harmful. It also found that the kids who repeated fifth grade were better off than kids who just squeaked by and passed the test and moved on to sixth grade. (Study: “The Academic Effects of Summer Instruction and Retention in New York City.” Educational Evaluation and Policy Analysis, v. 35, no. 1, Mar. 2013, p. 96-117)
So a growing consensus was emerging in the research community that holding a kid back in younger grades isn’t harmful and sometimes helpful if accompanied by support services, such as summer school, tutoring and advising.
And now Andrew’s paper — contradicting the new consensus — lands. It’s a quantitatively rigorous study finding harmful effects for younger children. She looked at more than 37,000 children across the United States from two older multi-year surveys (NLSY 1979 and NELS 1988) and found that about 10 percent had been held back at school, most of them during the 1980s. The surveys included details of the family characteristics of the children. That allowed Andrew to create 6,500 matched pairs of students, where the retained and non-retained students had similar backgrounds. Their mothers had attained the same level of education and their families had the same household income. The students had scored the same on a pre-school cognitive test. (In layman’s terms, they started school with similar IQs). The matched students also had similar behavioral problems, as reported on the surveys. Home environment, gender and race were factored in, too. In other words, Andrew matched the held-back students with students who were equally “at risk” for being held back, but weren’t.
Related story: India data show test scores rise when students are automatically promoted to the next grade
Then Andrew looked at whether these matched students eventually graduated from high school. And that’s where she found that the held-back children were 60 percent less likely to have graduated from high school than their matched “partners” who stayed on grade level. Andrew went one further to see if she could reproduce the results in a different way. Using the 1979 data survey, which included sibling information, she compared children who were held back with their siblings who weren’t held back. Again, she found the same result. Even in the same family, held-back kids were 60 percent less likely to graduate high school than their brothers and sisters. Astonishing!
Andrew acknowledges that held-back students often show a short-term boost in their grades and test scores, but she believes this boost “disappears” after just a few years. A sociologist by training, Andrew hypothesizes that being held back is so psychologically scarring that many students fail to regain their confidence in the long-term. In her paper Andrews argues that being held back is a one of the biggest negative events of a child’s life. “In surveys, students rank being retained in grade second only to a parent’s death in seriousness in some cases,” Andrews wrote.
At first blush, the data seem to defy common sense. (Data have a way of doing that!) Kids, especially boys with fall birthdays, are commonly held back in kindergarten as they get another year to mature. I have a hard time believing that they’re 60 percent less likely to graduate from high school than the kid who stayed with his class and moved on to first grade.
Unfortunately, Andrew wasn’t able to test whether kindergarten retention was less scarring than say, fourth grade retention. But by email she explained that the majority of the students were held back in the earliest grades, confirming that she found even held-back kindergarteners less likely to graduate from high school.
How much you buy Andrew’s conclusions depends on how similar you think her paired children are. If there were a characteristic that prompted a parent to hold back one child that his statistical “partner” doesn’t have, then the analysis isn’t clean. Her control group (the promoted partner) isn’t otherwise identical to the treatment group (the retained child). Andrew’s data sets didn’t list every behavioral problem and learning disability, so she couldn’t control for Attention Deficit Hyperactivity Disorder (ADHD) and other many other conditions. It’s quite possible that some of the held-back children had behavior issues or a mild learning disability and the promoted partner child didn’t. Years later, when Andrew found that the held-back child didn’t graduate from high school, it’s possible that factors related to the student’s behavior or learning issues — being placed in an alternative academic track, for example — impeded his academic career and not the psychological scarring of being held back in first grade.
I don’t want to suggest that ADHD makes it hard to graduate from high school, but I am trying to explain how Andrew’s research can fall into the same trap that the early research on retention fell into. It can accidentally conflate the bad effects connected to a behavioral or learning problem with the bad effects of the retention.
I asked Andrew how a parent should factor in her research when deciding whether to hold a student back. “My study is not a parent’s how-to guide on retention,” she replied by email, explaining that holding a child back is a very personal decision. The most important thing is to address your child’s underlying academic problems, whether you’re holding him back or passing him on to the next grade.
She explained her study is aimed at education policy officials who are deciding whether to have high-stakes tests that determine who moves on and who is held back. “My study is an argument about how a very expensive policy, grade retention, may actually undermine our shared goals of ensuring even child gets a quality education,” she replied. “I would argue that my study is evidence that we might take funds used for an expensive and likely deleterious policy and use them for earlier, pre-school interventions and …supplemental services… to help get a student up to speed.”
