It all sounds so good, until you study it

I’ve recently started reading Everything is Obvious: *Once you know the answer which points out that just about every answer that can result from a social science study can seem obvious, and that’s why data is so important.  Our “common sense” when it comes to questions of social science is murky and easily biased.  (In particular, the author critiques Gladwell, which is pretty interesting since I do enjoy reading Gladwell books.)

I was reminded of that lesson yesterday, sitting on the advisory board  for an NSF-funded education project.

First, let me set the trap.  A few weeks ago, a colleague had recommended to me this interest RS Animate YouTube video on education by Sir Ken Robinson.  One of the claims made in the video (starting about 7:40 into the video) is that kindergarteners do really well on measures of creativity, but those same students do worse-and-worse on those same measures the longer that they’re in school.  The hypothesis is that schooling sucks the creativity out of students.

The NSF project, whose advisory board I was serving on, seeks to measure the potential impact on creativity of their intervention.  On the advisory board were several experts on creativity (from both psychology and education), and they told the project to just give up.  We spent a good amount of time reviewing measures of creativity and literature on creativity.  First, there are very few good (e.g., validated, reliable, you get the same results regularly) measures of creativity, and second, all good measures of creativity are based on expertise.  You have to know something to be creative with it.  The project is unlikely to measure much scientific creativity in 8th graders.

I realized that I had been drawn into the it’s-obvious trap by Sir Ken Robinson.  What does it mean for kindergarteners to be creative?  What do they know?  They might be creative in the playing-with-fingerpaints sense, but not in the innovative thinking sense.  And how exactly did they measure creativity in kindergarteners, given that the valid existing measures assume significant knowledge?  It may well be true that schooling sucks creativity out of kids, but I don’t think that the study described by the RS Animate video is providing significant evidence of that.  But it all sounded so good…