Challenges of using Big Data to inform education

The story below is interesting, but not too surprising.  Researchers are having trouble using MOOC data to inform our understanding of student behavior and learning.  Lots of data doesn’t necessarily mean lots of insight.

I watched Charlie Rose interview the Freakonomics guys (view here), Dubner and Levitt, and found Levitt’s comments about “big data” intriguing.  He’s concerned that we don’t really have the methods for analyzing such large pools of data, and there’s a real chance that Big Data could lead us to Big Mistakes, because we might act in response to our “findings,” when we don’t really have good methods for arriving at (and testing) those “findings.”

Coursera isn’t the only MOOC provider to leave researchers longing for better data collection procedures. When Harvard University and the Massachusetts Institute of Technology last week released student data collected by edX, some higher education consultants remarked that the data provided “no insight into learner patterns of behavior over time.”“It’s not as simple as them providing better data,” Whitmer said. “They should have some skin in it, because this is their job. They should be helping us with this.”

via After grappling with data, MOOC Research Initiative participants release results @insidehighered.

An FTC commissioner (see article) just pointed out the possibility of big data to lead to discriminatory practices.  How much more is education at risk?

During a conference held yesterday in Washington, DC, called “Big Data: A Tool for Inclusion or Exclusion?” FTC Commissioner Julie Brill declared that regulatory agencies should shift their critical lens to what she described as the “unregulated world of data brokers.” According to Brill, there is a “clear potential” for the profiles of low-income and racialized consumers built with personal data “to harm low-income and other vulnerable consumers.”