A big selling point for a lot of educational software is that it’s engaging like a video game. Students are motivated to progress from level to level and so they pay attention and work hard. Some kids, and this is particularly true with competitive boys, are so motivated that they race against each other to see who can hit a level first.
I first noticed this racing phenomenon in a so-called blended learning classroom in Los Altos, California back in 2011. They were using online worksheets produced by Khan Academy to learn math in fifth grade. During my classroom tour, I saw boys craning their necks to check out each others’ computer screens to see where their neighbors were. Some were on 10 and some were on 11. The boys were having a blast. But there was no time to click on a video and hear an explanation for something they didn’t understand well.
Even then Sal Khan, the website’s founder, was concerned that students weren’t retaining the math they were learning online in long term. Six months later, he said, a lot was forgotten. But it was unclear whether speed was a factor. Perhaps even the slow, methodical students weren’t retaining the material very long.
So I was fascinated when data scientists at Applied Predictive Technology (APT) specifically looked at speed when they analyzed the experience of 1200 students at D.C. Prep, a charter school network in Washington D.C., that was using two educational apps in the classroom this past fall. One was a reading program, Raz-Kids, for kindergarten through third grade. The other, Typing Club, is aimed at second through seventh grade. It turned out that speed is a complicated thing and, depending what you’re trying to learn, going fast might be beneficial. And other times, it’s detrimental.
First the data. 504 kids were on the Raz-Kids reading program this past fall where they earn stars (or points) and progress from level to level as they listen to, read and take quizzes on clusters books. Students used the app in the classroom and could access it at home if they wanted to. The app itself retains a lot of data on how much time kids spend on it, what order kids do the tasks, how many books each kid reads, etc. Children’s reading levels were assessed twice, at the beginning of the school year and again in December to seem how much each child progressed. The school also provided APT with children’s school records.
Researchers found that for first and second graders, the more kids used the reading app, the more their reading scores increased. First graders who read over 30 books were correlated with a 1.06 level increase in STEP scores. Second graders who read over 20 books were correlated with a 1.2 level increase in STEP scores. There was a much weaker correlation for kindergarten students . And the app didn’t make a difference at all for third graders. First graders who listened to as many books as possible, even if they skipped the quizzes that checked for comprehension, did the best. In other words, exposure to large quantities of words matters more than accuracy in first grade reading. However, by second grade, it mattered more to follow the listen, read, quiz steps in order.
Sarah Hinkfuss Zampardo, a principal at Applied Predictive Technologies, said that the app was most useful for kids with behavioral issues. The kids with the most behavioral actions on their student records saw a lot of improvement. But she also pointed out that these tended to be some of the weaker students in the class and “it’s easier to improve if you’re starting from a lower place.” Meanwhile, the strongest readers in the classroom tended to use the app more at home. “But Raz-Kids wasn’t necessarily making them stronger,” Zampardo clarified.
Meanwhile, in typing, quality mattered more than quantity. At DC Prep, 573 students used the Typing Club app. And like Raz-Kids, the app retains all sorts of data on usage. APT researchers found that boys and girls tended to use the app in two distinct styles. Some students, particularly boys, tried to advance through levels as quickly as possible. Others, particularly girls,, tried to achieve the highest score possible on a level before progressing.
APT data analysts found that students took the time to “fill in,” or go back and complete levels to achieve the highest score, advanced in reaching their words-per-minute goal more rapidly than students who would “fly through” the program, or get through each level as quickly as possible. The methodical students who went slower had a 36% improvement in typing. The “fly through” students had only a 17% improvement.
So is competitive “flying through” behavior good in reading, but not typing? “I don’t know if we know enough to say students are flying through in the same way. Maybe in Raz-Kids, the kids are more excited to read the next book,” said Zampardo, and not just seeking the bragging rights of the next level.
