What if “CS1” is just a really bad idea?

At the AP CS Commission meeting this last weekend, we addressed the question, “How would we know if we’ve done our job right?  How would we know if  this is a good class?”  It’s an important question to ask about anything, but I realized from the discussion that it’s a particularly hard question to ask about a class or a class definition.

We talked about the five pilots of this class which are going to take place in Fall 2010. (Yes, the teachers have been picked, and no, I’m not allowed to say yet who they are — they’ll be announced at SIGCSE 2010, I understand.)  These are all excellent teachers.  What could go wrong?  I started listing the problems that could occur.  Sure, those things could go wrong — but what would you attribute a failure to?  What evidence would you need to convince you that the idea of the class is bad, e.g., that the learning objectives are too big and too broad and can’t be covered in a one-College-semester or one-High-School-year course?

In any given offering of the class, outcomes might suggest that something went wrong — a high withdrawal or failure rate, students do not perform well on a post-test, or the loss of more women and minorities than white or Asian males.  Would you say that that was just bad luck? Maybe it was students in the class?  Maybe they were under-prepared, or unmotivated, or had a bad mix of personalities so that they were always annoyed with each other or with the teacher?  Maybe it was a bad teacher?  Maybe the approach to teaching the learning objectives was bad — a lousy book, an inappropriate use of project-based or inquiry-based learning, boring lectures, or bad homework?  Maybe the room was at fault — uncomfortable, poor HVAC, faulty A/V? Maybe the school didn’t support  the class well, or the graders were bad, or there were rumors about the class going around the school creating a set of weird expectations?

But at what point would you decide, “You know, I don’t think that this class works?”  What would convince you?

Now, let’s consider Jens Bennedsen and Michael Caspersen’s paper surveying CS1 failure rates across North American and Europe.  These failure rates range between 30-50% in the US, and even higher (on average) in Europe.  If you have a class that hundreds of schools can’t teach without flunking out half the students, maybe the whole class is a bad idea?  Maybe the set of learning objectives for CS1 are just too big and it’s actually the rare case (a correct combination of teacher, book, approach, room, school, etc.) where the class actually results in learning gains for most if not all students.

I’m not making the argument here that CS1 is a really bad idea.  I am raising the question of “How would you know?”