I wrote this piece earlier this summer, looking at the use of technology in education. But our conversation wandered into the domain of data and I wanted to excerpt those parts of the interview for Education By The Numbers.

MIT technology trailblazer is a critic of computerized learning

Q: What do you think of using data to influence instruction? Using big data sets to change how schools teach kids?

A:  To be honest, being at a place like MIT, people here are focused a lot at looking at data and treat data in a very privileged way. I’m often on the side of saying, “Wait a minute. We shouldn’t be designing everything just on the data.” Yes, we should take advantage of the data. But there are other ways of trying to get information as well. For example, if we want to understand how and what children learn, sitting down and talking to one of the students can also be very useful.

Q: Then why are you collecting data on Scratch usage?

A: Now that Scratch is online, we can have access to lots more data about what kids are doing. And that can be useful. If we see that certain of the programming blocks are not used, it might make us wonder, should they even be there? Or is it confusing for some reason? Should we change them to make it less confusing? It could influence our design.

If we want to see, how is it that students start using a certain concepts? When do they start using variables? Are there certain experiences that people have that are more likely to lead into using variables? There are things like that that might change some of our pedagogy.

Looking at the data might change some of our design and some of our pedagogy. But I want to be careful not to make all of our decisions based on that.

Q: What do you think of so-called adaptive learning, where computers tailor instruction for each student?

A:  Clearly there are some advantages at having certain things personalized for you. As long as it’s some options, choices and suggestions, then it’s okay. But I wouldn’t want to be limited only to what a machine suggests for me. If it’s central to my experience, if I’m categorized in a certain way and pushed down a certain path, it could make a much worse experience for me.

The machine could have students avoid things they might have been interested in. If the machine is trying to make a guess, based on how I answered one question, what would be appropriate to show me next, even if you and I answer a question the same way, it could be for different reasons. Even if we make the same mistake on the same question, it might be for different reasons. When a machine tries to make suggestions for you, a lot of time it’s wrong. It can be more frustrating than it’s worth. I personally tend to be somewhat skeptical when the machines try to be too intelligent.

One other caution would be, it’s great to have things that are specialized for me, but it’s also great to be part of a greater community.

I sometimes worry [that] it’s very easy for computers to give feedback these days. It’s seen as this great thing. Students are filling out answers to problem sets and exams. Right away it shows them if they’re right or wrong and they can get feedback right away, which can influence what they do next. Getting feedback is great. I’m all for feedback.

My concern, it’s only easy to give feedback on certain types of knowledge and certain types of activity. I think there’s a real risk, that we as a society, are going to end up giving too much privilege to the types of knowledge and the types of activity that are most easily evaluated and assessed computationally.

Q: Are you worried about more multiple-choice worksheets in our schools?

A: If that’s the result, then it’s a really bad result.

(edited for length and clarity)