Researchers cast doubt about early warning systems’ effect on retention

CS researchers have long been interested in what predicts success in introductory computing, e.g., the “camel has two humps” paper, and the Bennedsen and Caspersen review of the literature.  Would knowing who might succeed or fail allow us to boost retention?  A new system at Purdue was claimed to do exactly that, but turns out, isn’t.

Michael Caulfield, director of blended and networked learning at Washington State University at Vancouver, decided to take a closer look at Signals after Purdue in a September press release claimed taking two Signals-enabled courses increased students’ six-year graduation rate by 21.48 percent. Caulfield described Purdue research scientist Matt Pistilli’s statement that “two courses is the magic number” as “maddening.”

Comparing the retention rates of the 2007 and 2009 cohorts, Caulfield suggested much of what Purdue described as data analysis just measured how many courses students took. As Signals in 2008 left its pilot and more students across campus enrolled in at least one such course, Caulfield found the retention effect “disappeared completely.”

Put another way, “students are taking more … Signals courses because they persist, rather than persisting because they are taking more Signals courses,” Caulfield wrote.

via Researchers cast doubt about early warning systems’ effect on retention | Inside Higher Ed.