Udacity’s CS101: Who are you talking to?

April 20, 2012 at 8:19 am 26 comments

When Carnegie Mellon University’s cognitive tutors first went into the classroom, Janet Schofield went with them.  She’s a social scientist who wanted to see what happened when this advanced technology got into real classrooms.  One of her findings was that, as predicted, the students would work individually with the technology, giving the teacher the opportunity to wander the room and give one-on-one time to individual students. The teacher could provide personalized instruction.  That did happen, but not in the way that you might hope.  The teacher spent most of the time with the best students.  Why not?  Those were the students most interesting to talk to.

Sebastian Thrun and Dave Evans of Udacity came to Georgia Tech this week, and talked about the completion of their CS101 course.  100,000 people signed up for the course, but that was just providing an email address — no cost, no commitment.  50,000 visited the site before the first assignment, and 30,000 completed the first assignment — one of those is probably a better measure of who was serious about taking the course.  10,000 completed the course. There are blog posts around from both completers and non-completers.  3,000 got a perfect score, which is great for Udacity and their business model. (Thanks to Dave who vetted these results for me.)

Sebastian was exuberant.  He says that he can’t go back to lecture teaching anymore, since on-line courses reach so many student.  “You move the needle so much!”  I asked how he knew that he moved the needle.  He admitted, “I don’t. We don’t know what they knew coming in. But we get told about the effect we have on these students. I get these great emails!”  He talked about how empowering it was for students to complete their programming assignments, how much the students said that the course changed their lives.  “It’s a thrill ride for the instructors!”

I believe Sebastian when he says that.  I bet that a Udacity class is great fun to teach.  Key to Udacity and similar online course platforms is a rapid feedback loop.  They know what’s going on in the class all-the-time, from all the instrumentation on every problem, and from all the message boards and email traffic.  They hear a lot, from really good students.

Is the Udacity (and maybe Coursera) model effective for more than reaching the best students, beyond the ones who are willing to put in the big effort? It’s an occupational hazard, of being a professor in a state university and of being a computing education researcher who studies how non-CS majors learn CS, that I worry about those who start but struggle.  I am a state employee, and ultimately, work for the state taxpayers who want to have their children educated.  I measure my success (and failure, too) on how well I serve the whole class.  Retention matters to me. I care about motivating students to care about computer science. And I hear from the students who drop out or fail: in terms of course reports, in terms of failing grades on assignments, in terms of tears at office hours.  If I only talked and listened to the top students, the job would be easier.

From hearing Dave and Sebastian, I don’t think that they’re arguing that they are replacing the state university, nor that they are reaching everybody.  They have a new kind of educational technology that speaks to a particular audience, and they are exploring it.  I don’t worry about Sebastian and Dave.  I more worry about those who don’t see the students that Udacity isn’t talking to.  There are lots of stories on the Internet about how Udacity represents the future of university education.  If you want to have more well educated students, if you want to improve graduation rates, you have to speak with the students that might not be so much fun to talk to — the ones who make you invent new approaches to motivate and engage, who question what you teach and why you teach it.  Udacity solves a particular problem.  It’s not necessarily the answer to the problems facing higher education.

About these ads

Entry filed under: Uncategorized. Tags: , , .

Massive open, on-line courses: With the faculty, or against the faculty? Do free and open learning technologies help the rich more than the poor?

26 Comments Add your own

  • 1. nickfalkner  |  April 20, 2012 at 8:29 am

    I couldn’t agree more.

    Udacity is great and is an excellent part of the solution, but we have to remember that a lot of our intake aren’t self-starting, don’t know what they want to do and may not even realise what they could get out of something like Udacity until they’ve spent some time in another institution and got over their initial hurdles.

    Reply
  • 2. Betsy DiSalvo  |  April 20, 2012 at 8:41 am

    These concerns with free online learning echo some of the findings and musings of Justin Reich on digital Equality, http://www.gse.harvard.edu/news-impact/2011/11/thoughts-on-digital-equity-justin-reich/. His observations indicate that the free online learning may create greater inequities, rather than closing the gap.

    Reply
    • 3. Cecily  |  April 20, 2012 at 11:28 am

      This sounds like a finding from a Chas Murray(I think) paper at AIED 2005 that showed an ITS actually widens the gap, especially if students are gaming the system.

      Reply
  • 4. Andy Ko  |  April 20, 2012 at 10:51 am

    It seems clear from the comments and rhetoric about Udacity and similar efforts that the priorities are scale and access first and learning second (“You move the needle so much!” and not “Nearly everyone passed!”). This might be inherent to the fact that these learning opportunities are discretionary. Perhaps It’s really only in compulsory settings that we can reach (and have the incentive and attention to reach) the whole class.

    In discretionary settings, I wonder what successful retention would even mean. It’s probably not some absolute proportion, since there are always going to be people who stop for reasons unrelated to the class. Instead, it’s probably something about learners hitting barriers, but overcoming them with the help of instructors, instructional materials, and classmates.

    Reply
    • 5. Jason  |  April 24, 2012 at 12:43 am

      Yes, I think this gets at the real issue. In the traditional university classroom, it is also easy (unfortunately) to find oneself talking more to the students who are nodding along, asking questions, and coming to office hours.

      Struggling or disengaged students are nonetheless fairly likely to stay in a traditional course (where they require some kind of attention) while they’re fairly likely to drop out of a Udacity course. But isn’t that difference mostly about the incentive structure rather than the delivery mechanism? If students were trying to accumulate credits toward some kind of Udacity degree program, then they’d be less likely to drop out, especially if their Udacity transcript (like a traditional transcript) listed the courses they’d dropped as well as the ones they’d completed.

      Reply
  • 6. Cecily  |  April 20, 2012 at 11:41 am

    I am intrigued by this post. I agree that it is definitely more fun to teach the bright students; we are doing final projects now, and they are definitely getting more attention. I also agree that it is important to reach all students. However, you claim ” I measure my success (and failure, too) on how well I serve the whole class.” For some definition of class, this is essentially what Sebastian is doing too, right? The major difference is how you define class. In neither case, do I tend to believe you mean all of society as a class. Nor do I believe that either of you is dealing with students at extreme risk of dropping from college. In Sebastian’s case, I would not be surprised if a number, maybe even a majority, of his student have already graduated from college. In your case, the admissions standards to Georgia Tech are so high, that it is not really fair to compare to the average state school, even if you are taxpayer funded. I suspect that teaching at Georgia Tech would be roughly comparable to teaching at BYU or Duke, based on admission standards. Both of those are way more competitive than any of the state schools in our state or region.

    Reply
    • 7. Mark Guzdial  |  April 20, 2012 at 11:49 am

      Cecily, can you explain a bit more? Agreed, GT students are better than the national average of College students. We do flunk students out of classes and school. It looks like I’m going to keep two seniors from graduating this semester, which is disappointing to me, and does cause me to reflect on what I could have done to have prevented that. My point is that I see that. I know who I’m flunking out and talk to them and hear their complaints. If I was teaching somewhere else, I would have different constraints and student quality, but I would have similar goals.

      Reply
  • 8. gasstationwithoutpumps  |  April 20, 2012 at 12:44 pm

    I don’t believe that retention is the most important goal for a university, even a state-funded one. Too often our politicians and administrators confuse a marker (like a bachelor’s degree) with what it is supposed to mean (a certain level of education). One can increase the number of people with the marker much more easily than one can increase the number of people with the education it is supposed to represent.

    The current focus seems to be on making college degrees more common and cheaper, which might be good if it weren’t being achieved mainly by lowering quality standards (both for the instruction and the level of performance needed to graduate).

    One consequence of many faculty who “measure my success (and failure, too) on how well I serve the whole class” is that they often measure their success mainly by how well they reach the bottom third, assuming that the ones near the top will somehow educate themselves. Honors classes and sections were the first victims of budget cutting on our campus—with the result that we have a bimodal retention problem, losing students at the top and bottom of the achievement spectrum.

    Even the regular classes for students in the middle of the spectrum suffer from gradual lowering of standards, as departments get resources based on the number of students they teach and easy courses tend to get higher enrollments.

    Reply
    • 9. Mark Guzdial  |  April 20, 2012 at 1:01 pm

      I’d love to be able to figure out how to measure that, the bimodal retention problem. I don’t see it, but you may be right — I may be looking too much at the bottom third. When I look at withdraw and failure stats, I mostly see low GPA students leaving, but maybe the drop-out (i.e., those just not re-registering) numbers have more high GPA?

      Reply
      • 10. gasstationwithoutpumps  |  April 20, 2012 at 6:58 pm

        You may not have a bimodal retention problem at Georgia Tech—most schools have a unimodal problem, losing students mainly from the bottom. We lose top students to more prestigious schools that appear to freshmen to have more opportunities for advanced work (in fact, I think we have more research opportunities for juniors and seniors, but the loss of almost all honors courses in the first 2 years hides this from the students).

        Reply
        • 11. Bonnie  |  April 24, 2012 at 7:06 am

          This is a huge problem for us too. We definitely lose from both the bottom and the top.

          Reply
  • 12. udacitydave  |  April 20, 2012 at 12:51 pm

    Thanks for the post, Mark! It was great visiting Georgia Tech, and I agree with much of what you say here.

    You raise some good points about the subset of students for whom open education works well. It requires a lot of self-motivation for someone to work hard in an open class without the external pressures that apply to most traditional university students. On the other hand, selecting based on motivation might be more fair to most people than selecting students based on the kinds of privileges students typically need to be able to attend even public universities today (e.g., no need to work for 4 years, being born in a wealthy country, having parents who value education, etc.) As wonderful as institutions like Georgia Tech are, only a tiny fraction of the world’s population has an opportunity to attend them.

    We do want to make our courses as accessible as possible to everyone, and realize there were some things about the way cs101 was designed the first time that were detrimental to that, and will be experimenting with alternatives to learn more about what works and doesn’t work.

    I also think the on-line social environment can provide a community that, for many students, can be very supportive and conducive to learning. Many students who find it difficult to participate in an in-person class become the vocal leaders and major contributors to on-line social communities.

    Reply
    • 13. Mark Guzdial  |  April 20, 2012 at 1:03 pm

      On the other hand, selecting based on motivation might be more fair to most people than selecting students based on the kinds of privileges students typically need to be able to attend even public universities today (e.g., no need to work for 4 years, being born in a wealthy country, having parents who value education, etc.)

      It’s a great point, Dave. Motivation is probably more equitably distributed than is socioeconomic status.

      Reply
  • [...] looked up Justin Reich based on Betsy DiSalvo’s comment last week.  Justin argues that the affluent benefit more from free and open learning technologies [...]

    Reply
  • 16. What’s the “problem” with MOOCs? « EdTechDev  |  May 5, 2012 at 1:38 am

    [...] These courses are clearly putting the traditional college course model online, and the problems are the same as with traditional college courses.  They are a big step above opencourseware sites, which just have notes or long recordings of class lectures online with no guidance or learning support, but as with traditional college courses, there is often a lack of active learning or effective instructional design, and a lack of interactivity or scaffolding of the learning experience for beginners.  Here are some comments from some folks who have attempted these courses (the vast majority of people drop out of these courses): [...]

    Reply
  • [...] gave me a new way of thinking about the results from Coursera and Udacity courses.  It’s not a problem that these systems are mostly attracting the 10-30% of students at the [...]

    Reply
  • 18. The Udacity student | run( ) {  |  June 1, 2012 at 9:04 am

    [...] reminded of something relevant previously mentioned both by Audrey Watters  at InsideHigherEd and Mark Guzdial at Computing Ed: that such online learning courses may be making a larger dent on experienced or gifted students [...]

    Reply
  • [...] real expertise. The second post is agreeing, but pointing out that maybe that’s the role of Udacity. I’m not arguing that Udacity or Coursera is dealing with teaching novices to code well [...]

    Reply
  • 20. Cheating Or Mastering? « Gödel’s Lost Letter and P=NP  |  August 21, 2012 at 12:27 pm

    [...] Tech who is an expert on many things including theory of education. Last April he wrote a a great post on his “Computing Education Blog” that discussed the drop rates: Sebastian Thrun and [...]

    Reply
  • [...] (to use Rich DeMillo’s term): An E-Ivy, or an ubiquitously-accessible Stanford.  The low pass rates aren’t a problem, then.  Rather, it’s using motivation and willingness to put in the effort as the filter, [...]

    Reply
  • 22. La revisión p2p del MOOC | exetia  |  February 22, 2013 at 1:33 pm

    [...] cual se plantean hasta hoy, los MOOC no acaban de funcionar; las altas tasas de abandono lo indican claramente. Para ser enseñanza y aprendizaje satisfactorios tienen que, por un lado, [...]

    Reply
  • 23. Research Questions About MOOCs » CCC Blog  |  February 22, 2013 at 2:15 pm

    [...] of the completers in edX’s first circuits course already had taken a circuits course.  30% of the completers in the Udacity CS101 course got perfect scores on the final exam.  In Tucker’s course, over 40% had their [...]

    Reply
  • 24. La revisión p2p del MOOC | exetia  |  May 23, 2013 at 6:04 am

    […] cual se plantean hasta hoy, los MOOC no acaban de funcionar; las altas tasas de abandono lo indican claramente. Para ser enseñanza y aprendizaje satisfactorios tienen que, por un lado, […]

    Reply
  • 25. What’s the “problem” with MOOCs? | InfoLogs  |  October 5, 2013 at 5:17 pm

    […] These courses are clearly putting the traditional college course model online, and the problems are the same as with traditional college courses.  They are a big step above opencourseware sites, which just have notes or long recordings of class lectures online with no guidance or learning support, but as with traditional college courses, there is often a lack of active learning or effective instructional design, and a lack of interactivity or scaffolding of the learning experience for beginners.  Here are some comments from some folks who have attempted these courses (the vast majority of people drop out of these courses): […]

    Reply
  • […] MOOCs represent the extreme approach of a ‘single learner only’ environment where the student is entirely on his/ her own. Due to their large scale and high number of subscribers MOOCs cannot offer individualized guidance, support or qualitative feedback on assignments. MOOCs have been criticized for supporting predominantly passive learning, conceptually replacing the ‘sage on the stage’ simply by an anonymous automated system. Attrition rates are subsequently very high (Tyler-Smith, 2006) with a vast majority not completing courses, e.g., in one of Udacity’s latest course ‘Introduction to Computer Science’ only 1 out of 10 participants completed their studies. The record-breaking ‘Introduction to Artificial Intelligence’ mentioned earlier had only a 7% completion rate (Guzidal, 2012). […]

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Recent Posts

April 2012
M T W T F S S
« Mar   May »
 1
2345678
9101112131415
16171819202122
23242526272829
30  

Feeds

Blog Stats

  • 967,706 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 3,145 other followers

CS Teaching Tips


Follow

Get every new post delivered to your Inbox.

Join 3,145 other followers

%d bloggers like this: