The long tail may not hit a target: High school teachers

January 31, 2012 at 9:28 am 12 comments

I’m attending the NSF CE21 Community meeting this Thursday and Friday.  I have been asked to lead a session on Friday afternoon on distance education in CS for teachers.  I was encouraged to talk about just a couple concrete examples, then leave the session open for discussion.  The question is which examples?

Here’s a more specific question that leads to this blog post: Are the on-line Stanford CS classes a good example to talk about?  Clearly, they are a highly innovative example of distance education for computer science.  But is it relevant for teaching high school teachers for the CS10K effort?

First of all, was the audience for the Stanford CS classes like the audience of potential CS10K teachers?  I’m not convinced.  First, when I read the comments to posts about the the Stanford classes, or Fred Martin’s post, I’m struck by how many people took the courses who already knew the content.  They were curious about the course, or wanted a refresher.  I wonder how many of the students who finished were novices to the content, and how many were old-timers?  My guess is that the average completer in the Stanford classes was a lot more CS-savvy than a business teacher who had never taken a CS class.

Second, was the method of teaching right for reaching in-service high school teachers?  I don’t think that the medium of the Stanford CS classes would work, at least as-is.  I read the comments to my post about the effort required in classes like these, and I think about Klara Benda’s study.  The people who dropped the course aren’t saying it was too hard.  They’re saying it took too much time, the pace was too demanding.  I can’t imagine that the technology behind the Stanford classes demands a rapid pace, but it’s clear that the pace was an issue for some of those who dropped out.  High school teachers don’t have the spare cycles for that rapid pace — Klara’s study has us realizing that we get small chunks of an in-service teacher’s time in which we can provide learning opportunities.

What I’ve come to realize is that the Stanford classes were successful as a long tail effect.  They enrolled a couple hundred thousand students, and some 20% finished. When you look at the big number of finishers, which is way more than probably all other students in all other AI classes in the world combined, it’s really quite remarkable.

On the other hand, 80% didn’t finish, and it may be that the students we most need to succeed for CS10K were in that 80%.  A long tail effect can get you large numbers, but perhaps, none of the numbers that you might be targeting.  A long tail covers a wide swath of the distribution of people, but those that you hit (who complete the course) are not necessarily randomly distributed.  More likely, the course acts as a filter on the long tail and filters everyone who doesn’t meet a particular set of criteria.  It may be possible to use a long tail approach and hit the target population you want to reach.  But it’s not a sure thing.

I am not claiming that the Stanford AI classes were trying to reach teachers for CS10K.  I am looking at that innovative work with a different filter.  I’m exploring the question of how well that innovation meets the CS10K goals.  As part of my talk preparation, I’m revisiting John Daniel’s book Mega-Universities and Knowledge Media.  It’s an older book now (1999), but they report that the UK Open University with its reliance on printed books had over a 50% completion rate on average across their classes.  I hope that advanced Internet technologies would lead to even better completion rates.

Entry filed under: Uncategorized. Tags: , , , , .

Openness is influenced by cognitive abilities: Self-efficacy too? Everyone needs to understand computing, even (especially?) Congress

12 Comments Add your own

  • 1. Cecily  |  January 31, 2012 at 9:55 am

    My vote: skip the Stanford online classes as an example. Ideally, you have some sort of online lesson pilot study that you’ve used with HS teachers that you can talk about, or if not, maybe you have some kind of outreach activity you’ve used with them, and you can talk about your plans to move material online. If neither of those is an option, try talking about w3schools or something similar- I actually used w3schools when I was teaching High School to learn PHP, CSS, and a number of other things that I thought students in a web development class should know. Of course, I was quite a bit more CS savvy than the standard high school teacher; many of the other teachers around the state didn’t know how to use a loop.

    I am a little skeptical that using online education to help teachers is going to work. My experience is that the new teachers are the most flexible and desperate to find curriculum, but they have the least amount of time to look for it as they are usually stuck in jobs with lots of classes to teach and lots of prep time. The more experienced teachers that have fewer preps have usually solidified their curriculum and they aren’t looking actively for new ideas to try.

  • 2. Alan Kay  |  January 31, 2012 at 10:08 am

    Hi Mark

    Adult learning has been studied extensively, and generalizations formed (they seem to have a lot of agreement, but maybe they are just sourcing each other).

    If close to accurate, these provide lots to be worried about with regard to plans proposing to seriously retrack large numbers of existing teachers into areas that require quite a bit of thinking and practicing to learn fluently.

    It is more likely that solutions involving the combination of partially fluent teachers with visitations and planning a few times a week with a fewer number of highly fluent expert teachers will work much better. This has been used in other subjects with real effectiveness, including both music and science.

    Over the next 20 years it is even more likely that a wide variety of STEM subjects (including computing) will be mostly taught by “smart media” and the roles of most of the human teachers will be social and motivational.

    Great teachers are rare and wonderful, but for most children, something better than no teacher or an average teacher would be an incredible improvement.

    A funding effort and exertion of will of the size and scope of the 60s ARPA funding that caused personal computing and pervasive networking to be invented over a period of about 12 years would most likely be successful.

    This kind of funding and national will is not so easy to come by these days, but it is interesting that the current state of computing technologies allow this to be thinkable and feasible.



  • 3. Alfred Thompson  |  January 31, 2012 at 10:36 am

    The more I try to leave an intelligent comment here the more I realize I don’t know enough to do so. I keep coming up with questions. Like, who will be taking these online courses and why will they be signing up? Will they be interested and excited about the new courses? Will they be taking the course just to get a stipend or will they be upset about not getting a stipend for the training? Will they be taking it on release time or “their own time?” Will the teachers know for sure that they will be teaching this material to full classrooms? It seems as though the training itself is the easy part. The online method a less expensive way to scale but one that only matters if the right students are taking the course.

  • 4. gasstationwithoutpumps  |  January 31, 2012 at 10:38 am

    I suspect that on-line courses will have a much higher drop-out rate than in-person classes. There is simply less social and emotional investment in finishing an on-line course—particularly a free one.

    I think that an on-line effort to reach kids is more likely to work than an on-line effort to reach high-school teachers. The ‘gifted children’ mailing lists are full of parents looking for computer science courses (on-line, textbook, summer camp, … ) for their kids. So are the homeschool mailing lists. There is a large under-served market—the best known player is iD tech camps ( and their material tends to be rather light on content and unsatisfying to the brighter students.

    Once you train the high-school teachers to be good enough programmers to teach CS1, there is a strong chance that they will leave teaching for the higher pay and lower workload of programming, so the CS 10K effort may be doomed from the start, unless you get most of the teacher-training colleges to include CS as a required course so that there is a lot longer tail of teachers who think that they might be able to teach programming.

    • 5. Cecily  |  January 31, 2012 at 10:54 am

      Tell me more about these ‘gifted children’ and ‘homeschool’ mailing lists- how does one go about getting on them and marketing materials to them?


      • 6. gasstationwithoutpumps  |  January 31, 2012 at 12:40 pm

        The best lists I’ve found for parents of gifted kids are the “tag” lists at but marketing is prohibited on the lists.

        The best clearinghouse for info about gifted students is Hoagies (and they will take advertising, I think, though most of their material is non-commercial). Their list of on-line communities is at

        Listing resources at Hoagies is a great way to reach a lot of parents and teachers of gifted kids, as it is the most commonly referred to resource site. (So far as I know, listing stuff at Hoagies is free—they are interested in being as inclusive as possible of resources.)

        One of the three home-school lists I’m on is tagmax (one of the lists)—the other two are local and not appropriate to list here. There is a lot of discussion of curricula and on-line courses, and endorsements of particular classes are ok (as long as they are real endorsements by people on the list whose kids have taken the courses—advertisers are quickly banned).

  • […] Mark Guzdial’s Computing Ed blog, I commented The ‘gifted children’ mailing lists are full of parents looking for computer […]

  • 8. Laura  |  January 31, 2012 at 2:03 pm

    One word: Summer. I have taken several courses in order to be ready to teach CS 1, which I am teaching this year. First, I went through a textbook. Then I took a six-week online course. Then, I basically took my own course. The six-week online course was a disaster for me. The content was good, but I had no time to complete all the assignments. Those were all first-level courses and now that I’m ready for CS 2 essentially, it’s harder to find things. Everything I see is geared toward the complete novice.

    I’ve looked an online courses through the community colleges, but they often don’t offer what I want in the summer, which is when I have time. And places that do offer good courses in the summer are ridiculously expensive. I’m not going to pay $3k to take a course, even if it would be “good for me”.

    One thing I’ve done that’s been successful is the weekend workshop. I’ve done a couple of these to pick up, for example, Arduino programming or Scratch. But these are usually focused around a particular technology, and geared toward beginners. If you have time to continue playing around with what you’ve learned, then it’s great. I took both of the workshops in the summer. See? Summer!

    • 9. gasstationwithoutpumps  |  January 31, 2012 at 3:22 pm

      The UCSC summer courses (which include the 2nd programming course and the main data structures and algorithms course) cost about $1500. That’s a couple hundred dollars more than Open Campus enrollment for non-students during the school year.

      Cabrillo College (like other community colleges) offers similar classes (some on-line) for about $175 (for California residents, out-of-state students would pay an extra $708). They’ve not released their summer schedule yet, so I don’t know which courses they will offer over the summer. The way the state budget is looking, it is likely that there will be far fewer courses available than in previous years.

  • 10. Lloyd  |  February 1, 2012 at 12:44 am

    The Stanford courses offered in the fall were on relatively advanced topics. The more relevant test, for training high school teachers, is not success rate (however measured) on the AI or ML courses, but on the CS 1 class scheduled for the spring.

  • 11. Comparing MOOCs and books « Computing Education Blog  |  December 7, 2012 at 8:26 am

    […] I just came back from a visit to Stanford where John Mitchell, vice-provost for on-line education at Stanford, explained to me the value of MOOCs over textbooks.  Textbooks don’t provide much of a feedback mechanism to the author — you write the book, and you get feedback from your class and maybe a few teachers who adopt your book and provide you comments.  But MOOCs let you try out ideas at scale, even do A/B testing on how to present something, and get feedback for the next design iteration.  I pointed out to him that that’s true, but only if you can separate out the signal from the noise.  Which MOOC students do you really want to get feedback from?  The 80% of “students” who are re-taking a course they’ve taken before?  The 90% of enrollees who never planned to finish? […]

  • […] journal article on the research that Klara Benda, Amy Bruckman, and I did finally came out last month the ACM Transactions on Computing Education.  The abstract is below. […]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 9,052 other followers


Recent Posts

Blog Stats

  • 2,030,790 hits
January 2012

CS Teaching Tips

%d bloggers like this: