The economic benefits of reducing drop-outs

Alan Kay just asked in another comment thread about the motivations for getting more students through CS education courses, perhaps with less than threshold understanding of computing.  I’ve had this piece queued up to post in my blog for some time — it’s an article arguing that there are enormous economic impacts of getting more kids through high school.  David Brooks has argued similarly that there are enormous impacts for getting more kids through undergraduate.  I think Alan’s point is well-taken — what is the threshold below which graduating someone is just dumping someone with a sophomoric level of knowledge on society?  Or is graduating someone with some level of knowledge better than graduating fewer (who are above threshold) and flunking out those who are below?

Few people realize the impact that high school dropouts have on a community’s economic, social, and civic health. Business owners and residents—in particular, those without school-aged children—may not be aware that they have much at stake in the success of their local high schools. Indeed, everyone—from car dealers and realtors to bank managers and local business owners—benefits when more students graduate from high school.

via The Economic Benefits from Halving the Dropout Rate: A Boom to Businesses in the Nation’s Largest Metropolitan Areas | Alliance for Excellent Education.

Fighting Engineers: Robots, Anecdotes, and Data

Got into a fight yesterday with an engineer.  We were both at an advisory board meeting for a new project here at Georgia Tech about teaching middle school science with Lego robotics.  It’s a great project — very careful in assessment, using Janet Kolodner’s Learning By Design classroom structures and rituals, with a national advisory board including several engineering faculty who work with Lego robots in outreach programs.  I raised the issue of girls and Lego at several points during the day, and at the end of the day, I asked, “Is it too late to consider something other than Lego?  Or are we bought in now?  Could we use another form of robotics?”

At that point, one of the engineers said, “Wait, I have to call him on this.” Then turned to me, “You seem to be under the misperception that girls don’t like Lego robots.  You’re wrong!”

Something in “misperception” and “wrong” got my dander up.  I snapped back, “And I’ve got an n of 1500 that says that you’re wrong!”  At this point, people between us literally stepped back to avoid the cross-fire.

Janet inserted a comment here, “Well, wait a minute, it all depends on what you do and your approach…” I interrupted her and explained.  “We’ve been doing Lego Robots with girls for four years now with Georgia Computes! We find that Lego Robots don’t change girls attitudes about computing, doesn’t make them more interested in STEM fields, and doesn’t make them see themselves as doing computing.  We find that PICO Crickets and Scratch do. ” It’s all about increasing the odds of success.

After the exchange, several engineers came up to me to tell me how their outreach program really works, and how their FIRST Robotics team has 30% females.  I asked how they knew it was working.  “You can just tell!” or “The girls keep coming back!”  How can you tell?  Why do they keep coming back?  Maybe it has nothing to do with the robots?

As I mentioned earlier, I’m the School of Interactive Computing.  We do a lot of qualitative analysis around here.  A commonly heard phrase in our hallways is, “The plural of anecdote is not data.”  We’re pretty careful with our data collection in “Georgia Computes!” We do surveys before and after each activity.  We use an external evaluator, to try to avoid the kids telling us what they think we want to hear.  We regularly use focus groups to spot check our findings.

Janet’s right, of course.  Some of the engineers are probably right, and their programs are really working.  From what I can tell, none of them that I spoke with know that for sure because they are not trying to evaluate things carefully.  They have a bunch of anecdotes.

I do believe that one can create an excellent Lego Robotics outreach program that engages girls.  Our data suggest that it doesn’t happen automatically with the default curriculum.  You have to do something special, and it would be great if someone would do the studies to figure out what that something special is so that we could all be doing it.  Let’s change the default curriculum.

The question I’m raising here about Lego Robotics for girls is the same one that I raised earlier about CS1.  How would you know if it’s a bad idea? How do you know that it’s just not working, given any random set of students, teacher, classroom, textbook, and curriculum? The data on CS1 support the claim that any particular offering of CS1, with any popular and well-used textbook and curriculum, will probably result in a 30-50% failure rate.  You have to do something unusual to beat those numbers. Further, I think that the data support the claim: “Given the number of studies, and the large number of subjects, the probability of engaging girls with Lego Robotics is low.”  Is it possible? Sure!  It would be great to do it right at scale!  But any one person’s experience with their outreach program or FIRST Robotics team is not going to change that statement.