Shelf space for books at home predicts educational outcomes

A fascinating blog post, “Does Poverty Cause Low Achievement?“, by Richard Rothstein of the Economic Policy Institute cautions researchers against using poverty or family income when crunching numbers to come up with education policies. He argues that poverty in and of itself doesn’t cause low achievement. And flawed educational research conclusions have been made by using poverty in data analyses. For example, a famous Heritage Foundation No Excuses study, found that low income kids can do extraordinarily well at school. But these low-income children were mostly the kids of Harvard and MIT grad students who just happened to qualify for free lunches while their parents were getting PhDs.

Instead of poverty, Rothstein says that education researchers should be trying to encapsulate a more complicated mixture of factors, such as multigenerational low income, that we think of as low social class. Rothstein thinks a good statistic would be mother’s educational attainment, but that information isn’t always available. In Rothstein’s own research, he has used “number of feet of shelf space devoted to books” at home. Using that standard, he found that kids from families without books in the U.S. perform just as badly as kids without books in other countries.

“…there is a social class achievement gap in every country, and it is of roughly similar size, even in countries which purportedly have much superior school systems. This should suggest that school improvement alone will not narrow that gap in the absence of addressing its social and economic causes.”

NYC high school graduates read by third grade

A new data study shows that third graders who can’t read proficiently are unlikely to graduate from high school in New York City. Only 2.7% of students who failed to meet a basic third grade English Language Arts (ELA) standard went on to meet or exceed the benchmark in eighth grade. Only one in three of these students ultimately graduated from high school. That’s a stark statistic to prove that early childhood education matters. I wonder if there’s anything a high school teacher can do to make up for huge deficits in the early years.

The Columbia University study, The Experiences of One New York City High School Cohort: Opportunities, Successes, and Challenges, analyzed elementary and middle school records for all 77,501 high school students who entered ninth grade in 2005 and hoped to graduate in 2009. The researchers were looking for things that happen before students even enter high school that might determine long-term academic success, including college performance. While overall high school graduation rates in New York City are improving (up by about 15 percentage points over the past decade to 65%) , some demographic groups, from special needs to Black and Hispanic students, lag considerably behind the average. The data report did not offer clear answers on what should be done.

I plan to take some more time later this week to study the results and the tables and write more. Full disclosure: two of the study’s authors were my instructors in a Columbia Teachers College data analysis course I took back in 2010-11, Professor Doug Ready and Miya Warner, who was the class teaching assistant.

The study’s release coincided with the formation of a new Education Funders Research Initiative a project of Philanthropy New York to study the results of education reform strategies over the last decade. The new data-driven initiative is funded by a large group of non-profits including the Ford Foundation, JP Morgan Chase Foundation and the Leona M. and Harry B. Helmsley Chartiable Trust.

Dissecting New Tech Network Numbers

The New Tech Network issued a press release on Oct. 2, 2013, trumpeting that its high school seniors outperformed 68% of 4-year college freshmen with similar backgrounds and abilities on key indicators of higher order thinking skills according to the College and Work Readiness Assessment (CWRA). The New Tech Network employs a lot of technology in its classrooms. On its website, there are a lot of pictures of students smiling in front of laptops and desktop computers. So it would be amazing to learn if New Tech’s unusual mix of computerized instruction and project-based learning was really producing such results.

I clicked on the underlying report and the numbers were more confusing. Instead of the “68%” figure above, the report said that “NTN seniors outperform 77% of college freshman (sic) and 60% of other high school seniors when controlling for academic ability.” I added the italics for emphasis.

I’d never seen test scores controlled for “academic ability” before. Usually I see controls for socio-economic status, income or race so that you’re comparing kids with similar backgrounds. But isn’t it bizarre to say that NTN kids got higher test scores than other kids with similar test scores?

It was also strange to me that NTN seniors performed so much better than college students, but not quite as well against other students their age. (77% outperformance vs. 60% outperformance).

A footnote explained, “On average, students in New Tech Network schools have lower academic skills than those in the comparison groups; this is possibly explained by the fact that the CWRA sample of high schools consists largely of private schools and CWRA does not control for ethnicity or socio-economic status in their analysis.”

So if I understand correctly, NTN seniors are testing WORSE than comparison groups. But when you cherry pick the test questions that deal with “higher order thinking skills”, then NTN students do better on those questions than other students who did as badly on the overall test as they did. Do I have something wrong here?

Government shutdown makes education data hard to come by

Granted, there are far more dire consequences to the federal government shutdown that this. But it’s really hard to write for the blog this week. The Census website, where you can find school district and funding data, is down.

As is the National Center for Education Statistics.

I was hoping to write a brilliant post about a new release of data linking the U.S. NAEP scores with the international TIMSS scores. But alas, that’s been postponed too. What’s an ed data girl to do? Another latte might help.

Data analysis discredits widely used TERC math curriculum

One of the most widely used math curricula in elementary schools, Investigations in Number, Data, and Space, also known as simply  “Investigations” or by its developer’s name, “TERC,” was found to underperform three other elementary school curricula. The three that performed better were Math Expressions, Saxon Math and Scott Foresman-Addison Wesley Mathematics (SFAW), also known as “enVision” Math.

The study, sponsored by a unit of the U.S. Department of Education to figure out what works in classrooms, looked at student results in 58 schools over the course of two years. It builds upon a previous one-year study that showed that students using the Saxon curriculum outperformed others, which I previously wrote about here.

Here are the key findings, according to a press release by Mathematica, which conducted the study for the Institute of Education Sciences.

At public school tours this past Spring in New York City, educators were openly criticizing the TERC curriculum and suggesting that parents should do supplemental drills to make sure that their kids learn how to add, subtract, multiply and divide. It’s a well-intentioned curriculum that encourages kids not simply to memorize formulas but to express their own mathematical ideas and make sense of numbers. It’s music to the ears of anyone who prefers progressive approach to education. I’d like to understand a little more about what this curriculum is doing wrong.

Still widespread use of restraint and seclusion at schools

I was surprised to see this article by Christopher Cousins in the online version of the Bangor Daily News, “Data from schools show widespread use of restraint and seclusion, but validity of numbers debated.” Cousins reports that 800 of Maine’s 185,738 students were restrained at schools during the 2012-13 academic year in order to deal with their emotional outbursts. Often, the students have special needs. The disciplinary practice is controversial and Maine even changed its state laws in 2012 to clarify what teachers can and cannot do. This May 2013 paper, How Safe Is The Schoolhouse? An Analysis of State Seclusion and Restraint Laws and Policies, by Jessica Butler lays out the confusing patchwork of seclusion and restraint laws in each state.

Butler makes the argument that “Seclusion and restraint are highly dangerous interventions that have led to death, injury, and trauma in children. The GAO (The Government Accountability Office, a U.S. Congressional watchdog organization) collected at least 20 stories of children who died in restraint. Neither practice should be allowed when there is no emergency posing a danger to physical safety.”

College board moans stagnation of SAT scores

The College Board, which administers the SAT, issued a report yesterday (September 26, 2013) bemoaning that only 43 percent of SAT takers in the 2013 graduating class were college and career ready. That means 57% are not ready. What does that mean? The College Board set an arbitrary cut off, that is a 1550 SAT score, above which students have a 65% probability of obtaining a college grade point average of a B- or above.  Those that scored a 1550 or higher were deemed “ready”. Those that scored lower were deemed “not ready”. What is so important about a B -, I wonder? Don’t grades fluctuate by university. I’d imagine that a C student at Yale might be able to snag a B – at SUNY. Is the lazy Yale student not college ready? If I recall correctly, President George W. Bush received a number of those gentleman’s C’s.

None of this is a sudden change in SAT scores. The number hasn’t budged since 2009. The big change in SAT scores happened in the late 80s and early 90s when thousands more students, many of them not as academically ambitious, began taking it. The average score plummeted and that’s why the administrators of the SAT had to “recenter” the exam and jack everyone’s points up by about 100. Thus, the 1300 SAT score of my day is now equivalent to 1400.

The College Board is planning to revamp the SAT in 2015 to test more of the kinds of things that students actually learn in high school.

Pre-K spending down but not everywhere

The New America Foundation on Sept. 24, 2013 released 2012 state and school district pre-kindergarten data, which the think tank says has never been published before. Their Funding Per Child widget allows users to see which districts in a specified state spend most and least per child on pre-K.

“The National Institute for Early Education Research, from which FEBP compiles state-level data, found that in 2012, state pre-K funding fell for the first time in a decade. The FEBP data can go a step further to show how those cuts were implemented in many states at the district level. For example, Texas’ state pre-K funding fell from $844 million in 2011 to $727 million last year. Houston pre-K funding fell from more than $61 million to nearly $53 million over the same span, while Denton School District saw increased spending of more than $700,000 and 100 more children enrolled.”

I just tried using the widget and it’s awkward to use. When I clicked on New York, too many districts popped up and I didn’t immediately see New York City among them.

To view the pre-K data for your state or school district, visit febp.newamerica.net and use the PreK-12 search box.

The full data release is on New America Foundation’s Federal Education Budget Project website.

Q&A with KIPP’s Dave Levin about Technology in Education

Dave Levin, co-founder of the KIPP (Knowledge is Power Program) network of charter schools, has a lot to be proud of. His schools, which focus on inner-city minority students, are now operating in 20 states and producing admirable test results and impressive numbers of college graduates.

One of his Los Angeles schools, which uses a lot of technology in the classroom, recently posted test scores so high (API = 991*) that it not only ranked as the highest performing school in the Los Angeles school district, but also the 10th highest performing elementary school in the State of California. This is a school in a tough neighborhood in South Los Angeles where nearly all the students are poor minorities.

At a September 24, 2013 dinner where Levin was awarded the Harold W. McGraw Jr. Prize in Education, I asked Levin how these results were influencing his thoughts on how much we should use technology in the classroom.

Barshay: What do these high test scores from KIPP Empower, where they’re using a lot of computerized instruction in the classroom, tell you?

Levin: It opens up the idea that you can individualize and differentiate instruction in real time, at the exact right level for each student, in a classroom, at home. You are able to accelerate student mastery. We’re in the very early days of what technology can do. But you can already see the promise.

Q: Do you see this happening at every one of your 140 KIPP schools?

A: A number of our principals are experimenting with technology. Mike (Kerr) in LA, Kate (Mazurek) in Chicago and Danny (Swersky) in New York.

All of KIPP’s success derives from having great teachers and great principals. What makes a great teacher? They’re constantly looking for the tools that will allow students to succeed and technology is one of those tools. Our teachers are going to find their way.

Q: Do you have concerns about using technology?

A: There’s no reservations whatsoever. Technology is a piece of the solution right now. It no way is the entire solution.

In his dinner remarks, Levin made the point that there are no clear answers to our educational debates. He is neither pro nor anti testing. He is neither an advocate for standardized curriculum nor a proponent of giving teachers complete autonomy to develop their own innovative lessons. “The answer is in the messy middle,” Levin said. “You need both”

* I first wrote about this KIPP Empower School back in October 2011 here. And it is interesting to see sustained high test scores after three years. I wanted to draw a little time series chart, but the school is using different tests in different years. 

2013: On California State tests, 95% of 2nd graders were proficient or advanced in English language arts and 98% were proficient or advanced in math. Its API score was 991

2012: On Stanford Achievement Test Series, usually referred to simply as the “SAT 10”, 96% of 1st graders were at or above national average in reading and 97% were at or above the national average in math. For kindergarteners, 97% were at or above national average in reading and 98% were at or above the national average in math.

On the STEP (Strategic Teaching and Evaluation of Progress) test, developed for high-risk children by the University of Chicago, 91% of kindergarteners were at or above grade level. These students began the year with only 39% at where they should be.

2011: On the Measures of Academic Progress test, developed by the Northwest Evaluation Association, 95 % scored at or above the national average in math, while 96 percent scored at or above it in reading.

Only nine percent arrived in kindergarten ready, according to the STEP (Strategic Teaching and Evaluation of Progress) pre-reading test, developed for high-risk children by the University of Chicago. By the end of the year, 96 percent of kindergarteners reached or exceeded the proficient mark on the same STEP test.

We have no idea what helps disabled children lead meaningful, productive adult lives

Anyone who cares for or knows a disabled child has likely wondered how to educate that person to lead a productive adult life. Is it best to educate the child in a conventional classroom, mixing disabled with non-disabled together? Should parents be more involved in a disabled child’s education? Depending on the severity of the disability, should the child be pushed to prepare for college or be tracked into a technical career? The questions go on and on. And now, nearly four decades after the Individuals with Disabilities Education Act (IDEA) granted access to public education for students with disabilities in the United States, we still have no idea what works.

That’s the conclusion of a large team of researchers from  Colorado State University, Mathematica Policy Research, University of South Florida, University of Montana, and Concordia University, with support from the National Library of Education, in an August 2013 paper, “Improving Post-High School Outcomes for Transition-Age Students with Disabilities: An Evidence Review,”  for the Institute of Education Sciences  under the Department of Education. The team looked at 43 studies of interventions for disabled students and found that none of the studies were designed well. Many lacked control groups who did not get the treatment. Others, for example, tested their treatment only once and could not replicate the result again. Others lumped children with very different types of disabilities together and it was unclear which types of disabilities were responding to the treatment.

“Finally, perhaps because of pressures to keep costs low and turnaround of results quick, many researchers do not follow students beyond high school to ascertain whether secondary school interventions have the desired ultimate effects after the transition from high school,” the authors wrote.

The studies covered a wide range of disabilities, including attention deficit hyperactivity disorder; ataxia; cerebral palsy; deafness, visual impairment, Down syndrome; emotional or behavioral disability; epilepsy; intellectual disability; learning disability; physical disability; seizure disorder; sensory impairment; Tourette syndrome; and traumatic brain injury.

Desperate to give some guidance to policy makers, parents and educators, the authors highlighted some of the imperfect studies as guides for further research. They saw some potential in further study of these hypotheses:

* Participation in career and technical education may be important for promoting employment outcomes.

* Employment in at least one job before students with disabilities leave high school may be an integral part of transition support.

* Inclusive education settings may be a key dimension of transitioning to postsecondary education.

* Computer-based instruction and prompting, may help students with intellectual disabilities to live more independently by increasing their functional skills.