Lessons Learned From First Year College MOOCs at Georgia Tech (and SJSU)

September 21, 2013 at 1:29 am 14 comments

Karen Head has finished her series on how well the freshman-composition course fared (quoted and linked below), published in The Chronicle. The stats were disappointing — only about 238 of the approximately 15K students who did the first homework finished the course. That’s even less than the ~10% we saw completing other MOOCs.

Georgia Tech also received funding from the Gates Foundation to trial a MOOC approach to a first year of college physics course.  I met with Mike Schatz last Friday to talk about his course.  The results were pretty similar: 20K students signed up, 3K students completed the first assignment, and only 170 finished.  Mike had an advantage that Karen didn’t — there are standardized tests for measuring the physics knowledge he was testing, and he used those tests pre-post.  Mike said the completers fell into three categories: those who came in with a lot of physics knowledge and who ended with relatively little gain, those who came in with very little knowledge and made almost no progress, and a group of students who really did learn alot.  They don’t know why nor the relative percentages yet.

The report from the San Jose State University MOOC experiment with a remedial mathematics course came out with the argument:

The researchers also say, perhaps unsurprisingly, that what mattered most was how hard students worked. “Measures of student effort trump all other variables tested for their relationships to student success,” they write, “including demographic descriptions of the students, course subject matter, and student use of support services.”

It’s not surprising, but it is relevant.  Students need to make effort to learn.  New college students, especially first generation college students (i.e., whose parents have never gone to college), may not know how much effort is needed.  Who will be most effective at communicating that message about effort and motivating that effort — a video of a professor, or an in-person professor who might even learn your name?

As Gary May, our Dean of Engineering, recently wrote in an op-ed essay published in Inside Higher Ed, “The prospect of MOOCs replacing the physical college campus for undergraduates is dubious at best. Other target audiences are likely better-suited for MOOCs.”

On the freshman-composition MOOC, Karen Head writes:

No, the course was not a success. Of course, the data are problematic: Many people have observed that MOOCs often have terrible retention rates, but is retention an accurate measure of success? We had 21,934 students enrolled, 14,771 of whom were active in the course. Our 26 lecture videos were viewed 95,631 times. Students submitted work for evaluation 2,942 times and completed 19,571 peer assessments (the means by which their writing was evaluated). However, only 238 students received a completion certificate—meaning that they completed all assignments and received satisfactory scores.

Our team is now investigating why so few students completed the course, but we have some hypotheses. For one thing, students who did not complete all three major assignments could not pass the course. Many struggled with technology, especially in the final assignment, in which they were asked to create a video presentation based on a personal philosophy or belief. Some students, for privacy and cultural reasons, chose not to complete that assignment, even when we changed the guidelines to require only an audio presentation with visual elements. There were other students who joined the course after the second week; we cautioned them that they would not be able to pass it because there was no mechanism for doing peer review after an assignment’s due date had passed.

via Lessons Learned From a Freshman-Composition MOOC – Wired Campus – The Chronicle of Higher Education.

Entry filed under: Uncategorized. Tags: , , .

CS National Curriculum in England Released Knowing more doesn’t necessarily lead to correct reasoning: Politics changes problem-solving

14 Comments Add your own

  • […] Guzdial, in his post Lessons Learned From First Year College MOOCs at Georgia Tech (and SJSU), points out the astonishingly low completion rates for some MOOCs (about 1% of registered students, […]

    • 2. mgozaydin  |  September 21, 2013 at 3:09 am

      That low retention rate does not bother me .
      It is free
      Even a primary school graduate can take it
      It is from an elite university
      Every body on internet signs up.
      Not bad .
      But if result is . Only 1 % I do not get sad .

      When MOOCs are for degrees every thing will be in place .

      • 3. gasstationwithoutpumps  |  September 21, 2013 at 10:30 am

        But the cost to put on the MOOCs was higher than the cost for conventional courses for the number who completed, and the quality probably lower. So what is the point? To give thousands of people a taste of failure?

        • 4. mgozaydin  |  September 21, 2013 at 11:18 am

          That is where the secret is .
          Today MOOCs are just experiments, market and opponents checking.

          When edx starts providing degrees even 1.000 students per semester can amortise $ 1 million development cost .
          Assume edx charges only $ 100 for 10 semesters.

          Collected revenue is
          1,000 x 10 semesters x $ 100 = $ 1,000,000
          Sure development is less than $ 1 millon ,
          probably enrollment is more than 1,000 , you may charge even $ 50 per course.

          2,000 students x 10 semesters x $ 50 = $ 1,000,000
          Not bad at all .
          These numbers are not unreachable numbers .
          I believe about 10,000 students globally in case of degrees.

          Let us do not miss the boat .

          We should be indebted that elite schools agreed to provide online courses.

          Sure they know they can make Money out of it .

          Look now MIT started to provide XSeries Certificates for 7 courses in a row . That is the sign of degrees now .

  • 5. mgozaydin  |  September 21, 2013 at 11:25 am

    Look up also
    Georgia Tech MS in Cs program .
    They expect 10,000 and will ccourses for 12 harge $ 7,000 of MS program .
    With degree everything will change .
    they do not want to scare people .

    I hope only elite ones will provide degrees .

  • 6. Garth  |  September 21, 2013 at 11:52 am

    Has there ever been any evidence that a MOOC would be successful? It would seem a little thought on the basics of how and why students learn would be enough to understand a MOOC will always get very poor completion results.

    • 7. mgozaydin  |  September 21, 2013 at 1:58 pm

      It is not easy to understand what is the policy of MOOCs.
      It is a long story .

      MIT started to capture the global markets with online in 2001 with OCW. They have now 150,000,000 students and teachers in the World following them free.

      MIT started first online in December 2001. There were no Coursera at that time .

      Coursera as a for profit company started in April 2012.
      Daphne Kollet of Coursera was very successful in marketing Coursera at New York Times .

      Then MIT and / or Harvard got the idea to work together like Coursera and they did and set up EDX.

      To me intention of EDX is to provide degrees eventually and globally to 150,000,000 students best courses in the World and still make Money .

      EDX providing degrees will be very successful.
      They do not need to be massive at all .

      At the most pessimist estimate
      Enrollment in a course 1,000 per semester very pessimist.
      For 10 semester
      Charge only $ 100
      Then revenue is 5 years is
      1,000 x 10 x $ 100 = $ 1,000,000

      I am sure their global enrollment will be around 10,000 per semester per course . That means lots of Money.
      Therefore they will charge less than $ 100,may be $ 50 or $ 20 per course.

      They will be successful since
      For the last 20 years online by even not so good schools with
      $ 1,500 per course were successful. The made billion dollars profits in only one year. There are 7 million online students now .

      Therefore I say and I conclude that
      Elite schools online with low fees must be successful .

      MOOCs are now experiment only .
      So retentions etc does not mean anything .

      Look up Georgia TECH $ 7,000 for an MS in CS . They even commit themselves to 10,000 students . ATT supports them.
      They are not naive to support such a big project .

  • 8. Mike  |  September 21, 2013 at 12:37 pm

    We’re all wondering how to evaluate MOOCs, and what the criteria for their success is. Given that they’re free (to students, not to the institutions that make them) a lot of people keep saying that the absurdly high drop-out rates aren’t relevant.

    Your mention of pre/post testing got me thinking – maybe MOOCs should be evaluated based on how much actual learning takes place? All those people that sign up for a MOOC despite already knowing the material aren’t really a success. They’d never sign up for a normal course because they’re not going to throw their money away on something they already know, but they’ll piddle around on a MOOC because it doesn’t cost them anything.

    Maybe a metric like:

    Sum of (each person who had a high posttest and a low pretest)
    divided by:
    cost to produce the course

    It seems like this would produce a number that reflects the true cost of educating people using this (or any other) method.

    • 9. mgozaydin  |  September 21, 2013 at 2:04 pm

      Please read my comment above.
      MOOCs are just experiment now .
      To me edx can replace traditional education soon and enroll 150,000,000 students in 10 years . Very optimistic.

      When they start providing degrees real learning will start.
      That can be done by only edx . Non profit and elite .

  • 10. Erik Engbrecht  |  September 21, 2013 at 11:16 pm

    I’m surprised that so many MOOCs have such rigid schedules. I’ve completed one and stopped maybe a quarter of the way through on another. The reason I stopped was that life interfered with assignments and given the forward march of due dates there was no way to catch up. This came really close to happening in my first one. I’m not sure about other people, but for me I think it would help alot if material was available a couple weeks prior to the week it was associated with (so it’s possible to work ahead if you have a vacation coming up) and have the class stretched maybe 1.5x to 2x the actual duration.

    But I don’t know if I’m in the target demographic. I work full time (and sometimes a lot more), am in the middle of my career, and at this point don’t see additional formal education as something that would provide me with any advantage. I just want to learn. So far I’ve basically used MOOCs as replacements or supplements to text books I would have bought to learn things on my own. The MOOCs provide some helpful structure and pacing. This all means that (1) I’d be willing to pay $50-100 for a MOOC (more if I had some pre-assurances that it was good), I would in most cases be willing to spend another $50-100 on an accompanying textbook, and I have an employer that given a little prodding would probably reimburse me and that is much less cost sensitive as long as there is some evidence of successful completion.

    Anyway, my point is that I think with the current structure of MOOCs you gain about what you venture. I doubt most people would hesitate much to fall behind in the event of a vacation, family emergency, or really busy time at their jobs. Charging thousands of dollars for a class makes people prioritize completing it. Charging nothing encourages people to sign up just to see what it’s about, and creates little sunk cost. Few, if anyone, really formally recognize completion of a MOOC so all you get out of it is what you learned, and you don’t lose what you learned if you quit early or keep on going but stop submitting assignments.

    Also, it’s pretty much impossible to measure the effectiveness of something if you don’t know what that something is supposed to achieve. Right now I have no idea what MOOCs are actually trying to achieve. Perhaps the offerors are just figuring out what their goals should be based on experimentation.

  • 11. Debbie Morrison  |  September 23, 2013 at 5:22 pm

    The issue is that MOOCs are geared to people that know how to learn, and are highly motivated to learn about a specific topic or subject.

    When reading several reports that have been published on the demographics of MOOC students, time and again the majority of active MOOC participants already have an undergraduate degree, and in some cases a graduate or terminal degree.

    First or second year undergraduate students do not possess the skills (in most cases), and motivation to complete a MOOC course.

  • […] students completed the first year college MOOC on physics at Georgia Tech. In an interesting blog post published last September, Karen Head from the School of Interactive Computing at Georgia Institute […]

  • […] don’t use MOOCs for intro courses (as we learned at GT with English composition and physics) where students don’t know that they really need this knowledge to go on, and the completion […]

  • […] terrific series or articles in The Chronicle of Higher Education about the experience (you can see my blog post on her last article in the series here). Her experience is the basis for her new book Disrupt This! (link to Amazon page here). There is […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 9,005 other followers


Recent Posts

Blog Stats

  • 1,878,616 hits
September 2013

CS Teaching Tips

%d bloggers like this: