The accuracy of federal education data

Correcting mistakes may be an essential part of a good education, but that doesn’t apply inside the branch of the U.S. government that compiles and keeps education statistics. Indeed, the National Center for Education Statistics (NCES) knowingly leaves in errors that are discovered two to three years later. And then this error-ridden data is used by education policy makers to make decisions.

I recently learned about these revision deadlines from the person in charge of the education data, NCES Commissioner Jack Buckley. Buckley explained that the Integrated Postsecondary Education Data System data set, a.k.a. IPEDS, allows for one year of revisions (after the initial collection year) and then “locks” that year’s data forever. That’s been a frustration for for-profit universities who’ve been clamoring to retroactively revise their graduation rates upwards so that their students can remain eligible for federal student loans.

Another major data collection, the Common Core of Data, is kept active for only three years, effectively cutting off revisions afterwards.

The IPEDS data is self-reported and it may be wise to limit the ability of schools to game the data to their benefit. But the “locking down” of the data also means that more innocent mistakes are made that can never be fixed — and then used for analysis by the public. At a May 2, 2013 session of an Education Writers Association conference, California Watch investigative reporter Erica Perez recounted how she repeatedly found mistakes in the federal databases when she phoned schools to verify the numbers.

Contrast the Department of Education’s data practices with the Bureau of Labor Statistics inside the Department of Commerce, which never locks its data and allows for infinite revisions of its jobs and other economic data. (I verified this with the BLS). Why shouldn’t the education folks, like the economists, always correct an error when one is found?

Commissioner Buckley, in an email, said that he sees a “tradeoff between getting it right the first time (since these data are used by policy makers and the public for annual decision making)” and “ allowing some flexibility for inevitable errors. Allowing schools to submit revisions for any year, any time is confusing for users and difficult for data collectors trying to maintain a complex process and keeping it timely and accurate.”

It’s hard to make a blanket assessment of the quality of the federal education data. The Department of Commerce rushes data out to the public as quickly as possible, making it more error prone. The Department of Education takes a much longer time to verify and clean up the data before its first release. “We spend a lot of time and money on data quality. Beware of the fastest person to put data out,” said Buckley at an April 30, 2013 session of the American Educational Research Association’s annual meeting.


POSTED BY Jill Barshay ON May 9, 2013

Comments & Trackbacks (1) | Post a Comment

SDR

Having worked in tumor registries for years, with millions of data points, I can assure you that data there is always found to be not-clean and needing attention. And it happens. I once found several dog’s tumors listed in the human database. Of course they were removed. Why would you not strive to make the data as clean as possible? It is a frank data-collections error not to. The point is to try to tease a correct analysis from the data, but if it is inaccurate, any conclusions will be as well.

There is, however, an important difference between fixing errors – and being permitted to do so – and remeasuring data. Schools should not be allowed to change their reports indefinitely; there must be a reporting cutoff for the purposes of counting. But that is a different matter from fixing errors and the two should not be confused.

Your email is never published nor shared.

Required
Required