What if “CS1” is just a really bad idea?

February 2, 2010 at 11:25 am 12 comments

At the AP CS Commission meeting this last weekend, we addressed the question, “How would we know if we’ve done our job right?  How would we know if  this is a good class?”  It’s an important question to ask about anything, but I realized from the discussion that it’s a particularly hard question to ask about a class or a class definition.

We talked about the five pilots of this class which are going to take place in Fall 2010. (Yes, the teachers have been picked, and no, I’m not allowed to say yet who they are — they’ll be announced at SIGCSE 2010, I understand.)  These are all excellent teachers.  What could go wrong?  I started listing the problems that could occur.  Sure, those things could go wrong — but what would you attribute a failure to?  What evidence would you need to convince you that the idea of the class is bad, e.g., that the learning objectives are too big and too broad and can’t be covered in a one-College-semester or one-High-School-year course?

In any given offering of the class, outcomes might suggest that something went wrong — a high withdrawal or failure rate, students do not perform well on a post-test, or the loss of more women and minorities than white or Asian males.  Would you say that that was just bad luck? Maybe it was students in the class?  Maybe they were under-prepared, or unmotivated, or had a bad mix of personalities so that they were always annoyed with each other or with the teacher?  Maybe it was a bad teacher?  Maybe the approach to teaching the learning objectives was bad — a lousy book, an inappropriate use of project-based or inquiry-based learning, boring lectures, or bad homework?  Maybe the room was at fault — uncomfortable, poor HVAC, faulty A/V? Maybe the school didn’t support  the class well, or the graders were bad, or there were rumors about the class going around the school creating a set of weird expectations?

But at what point would you decide, “You know, I don’t think that this class works?”  What would convince you?

Now, let’s consider Jens Bennedsen and Michael Caspersen’s paper surveying CS1 failure rates across North American and Europe.  These failure rates range between 30-50% in the US, and even higher (on average) in Europe.  If you have a class that hundreds of schools can’t teach without flunking out half the students, maybe the whole class is a bad idea?  Maybe the set of learning objectives for CS1 are just too big and it’s actually the rare case (a correct combination of teacher, book, approach, room, school, etc.) where the class actually results in learning gains for most if not all students.

I’m not making the argument here that CS1 is a really bad idea.  I am raising the question of “How would you know?”

Entry filed under: Uncategorized. Tags: , , .

Context matters, not just my context NSF BPC and CPATH end

12 Comments Add your own

  • 1. Alan Kay  |  February 2, 2010 at 12:54 pm

    Forgive me for iterating a few lessons learned ….

    It takes about 3 years to normalize enough things (including the observers) to get some sense of what is actually going on with a new curriculum. Attempts at global assessments (other than to help with what should be modified for year 2 and then year 3) are a really bad idea.

    Another hugely important question is to find out what students can learn under the very best conditions, and what it takes. Even if this is not realistic to scale, it must be done to get some kind of cognitive anchor on the distribution of abilities and needs. (This is almost never done, even when the eventual plan is a huge national curriculum and deployment.)

    There is more that needs to be done, but these two are critical.

    Cheers,

    Alan

    Reply
    • 2. Mark Guzdial  |  February 2, 2010 at 1:00 pm

      Hi Alan,

      I appreciate you reminding us of these points! The first one is highly relevant for the new APCS effort, and I do recognize that it’s a multi-year effort to be able to make a global assessment. The definition of CS1 dates back to the 1968 ACM Curriculum Report, I believe — which suggests that we’ve had enough experience to make a judgment.

      Your second point is particularly interesting to me. How do we establish this “cognitive anchor,” that in the best conditions, it is possible to cover this much material in the given time? I agree that we rarely do this for any new class.

      Cheers,
      Mark

      Reply
  • 3. Alan Kay  |  February 2, 2010 at 5:05 pm

    Hi Mark

    I actually meant 3 years per a classroom with the same teacher (which you might also have meant) under controlled observations.

    The second point is partly a matter of being willing to spend money to find out something about distribution of “readiness/abilities” over a big enough sample to be significant. Pick 5 or 10 categories of students (maybe there is a principled way to do this) and see what they need to get some sense of what kinds of prep have to be done by both partners in the enterprise, what kinds of effort and cost it takes to achieve (or not) above threshold results.

    I can imagine why they don’t want to do this (what it seems to imply), but it’s what any decent music teacher would do student by student.

    Cheers,

    Alan

    Reply
  • 4. Raymond Lister  |  February 2, 2010 at 5:33 pm

    “How would we know if this is a good class?” … I think this is a question central to computer science education research.

    I’m afraid my answer is disappointing … “You can’t reliably assess the quality of a whole-of-course change”.
    (But I’ll be very happy to be shown that I’m wrong.)

    I’ve come to think that we focus too much on assessing whole-of-course change. Instead, I think we shoulld develop a more patient, incremental approach to change, and only change a small portion of a class each semester … perhaps as little as one or two weeks.

    When you only change a portion of a course, you can use the part that remains unchanged to match students of equivalent performance across semesters. You can then test staitistically if a student who experienced the new portion of the course did better on that portion than an equivalent student, from an earlier semester, who experienced the old portion of the course.

    I realize that my answer is totally irrelevant to Mark’s original question, as his AP context requires an evaluation of a whole-of-course change. But to people who are teaching their own course, and can control the rate of change in their course, I suggest that you slow down, change a little bit each semester,
    and test whether that change resulted in an improvement.

    Reply
  • 5. Alfred Thompson  |  February 2, 2010 at 8:17 pm

    When I think about the amount of material that was covered when I took CS1 35+ years ago and what we tend to try to cover in APCS A (programming) now I see a large increase. In fact what is attempted in a 4 year college program is much more than was attemted in 4 years 35 years ago. The field has expanded that much. As a result more and more gets pushed into the first course and that is not, in my opinion, a good idea. I suspect that this overload is part of the high fail rate. Given the tools we have for teaching today which are so much better than what we had “back in the day” one would expect more sucess than we are seeing.
    There was a time when we welcomed gate keeper courses to narrow down the students who went on. Overdoing it seemed to make sense. Today we need a course that is a gateWAY course that leads students to continue. The requirements are different.

    Reply
  • 6. Alan Kay  |  February 2, 2010 at 8:40 pm

    To all,

    I’m curious about the prerequisites (if any) for CS 1. Is the idea that this is the class to take if you have had essentially (or actually) no experience in computing?

    And is there a more or less agreed on outcome for this class? Is it more about programming than about surveying the field? Is it about building some skills for later courses (such as learning to program in a language that later courses think is important)?

    Are there “killer side-conditions” for this course?

    What is it?

    Cheers,

    Alan

    Reply
    • 7. Mark Guzdial  |  February 3, 2010 at 11:06 am

      My take:
      – No pre-requisites in the definition of the course. At many institutions, a pre-requisite is a breadth course (“CS0”), a light programming course (sometimes called “CS0.5”), or a discrete math course.
      – The outcomes are prescribed in the curriculum documents. I used CC2001 as my guide when I built the Media Computation Python course.

      Reply
  • 8. Briana Morrison  |  February 3, 2010 at 12:02 pm

    I believe it’s easier to convince ourselves that a course is *working* instead of not working. As educators we usually tend to think I can change how I present this topic to increase student knowledge, the test question(s) I used to assess this outcome were too difficult, etc. The question you pose, in essence, is when do we know when to declare the course unsuccessful. Given the difficulty of measuring computing knowledge independent of programming language (and since right now that’s just about the only mechanism we have of measuring computing knowledge), I don’t know that we are in a position to answer that question.
    But I also don’t know that computing is unique. Look at the changes in math and physics education. What prompts their seemingly endless (especially math at the primary grade levels) pedagogical changes? Usually performance in standardized test scores. CS has no equivalent (unless you consider the AP test…which is again programming language specific).
    The underlying question (to which I’m sure you’ve discussed in great depth in your meetings) is: What should an introductory computer science course cover? And that’s the one to which I don’t think we have a good answer…at least none with strong evidence.
    Briana

    Reply
  • 9. Jeff Graham  |  February 3, 2010 at 12:12 pm

    At my institution, there is no prerequisite for CS 1 and we expect that nobody has programming experience. Our course is about the programming. We hope to get enough syntax and skill so they can handle a data structures course the next semester. We have an unusual mix of students in our class. About 80% are business majors who need the class to complete an Information Systems Minor. The business students only need two courses past their common business core for this minor. For what it is worth, I’ve found a lot of good programmers majoring in accounting and finance. I don’t know where the person who commented above teaches, but I have been shedding material in the course I teach for awhile now. We still lose about half the class as soon as we start talking about functions.

    Reply
  • 10. Darrin Thompson  |  February 4, 2010 at 3:35 pm

    Do other disciplines have it easier? How do they know when their class is a bad idea?

    If I were to revamp Lit. 101, how would I know it’s bad? Or Philosophy or Physics or Welding?

    Reply
  • 11. Erik Engbrecht  |  February 4, 2010 at 4:09 pm

    I think CS1 may very well be a bad idea. It flunks half the students, and there is probably a non-trivial minority (maybe 15-25%?) that are bored out of their mind. So for the majority of the class, the class just isn’t working.

    But I think there is risk in focusing in solely on the people who it fails because it is too much for a semester.

    Reply
  • […] question I’m raising here about Lego Robotics for girls is the same one that I raised earlier about CS1.  How would you know if it’s a bad idea? How do you know that it’s just not working, […]

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed


Enter your email address to follow this blog and receive notifications of new posts by email.

Join 9,003 other followers

Feeds

Recent Posts

Blog Stats

  • 1,875,117 hits
February 2010
M T W T F S S
1234567
891011121314
15161718192021
22232425262728

CS Teaching Tips


%d bloggers like this: