Lessons Learned From First Year College MOOCs at Georgia Tech (and SJSU)
Karen Head has finished her series on how well the freshman-composition course fared (quoted and linked below), published in The Chronicle. The stats were disappointing — only about 238 of the approximately 15K students who did the first homework finished the course. That’s even less than the ~10% we saw completing other MOOCs.
Georgia Tech also received funding from the Gates Foundation to trial a MOOC approach to a first year of college physics course. I met with Mike Schatz last Friday to talk about his course. The results were pretty similar: 20K students signed up, 3K students completed the first assignment, and only 170 finished. Mike had an advantage that Karen didn’t — there are standardized tests for measuring the physics knowledge he was testing, and he used those tests pre-post. Mike said the completers fell into three categories: those who came in with a lot of physics knowledge and who ended with relatively little gain, those who came in with very little knowledge and made almost no progress, and a group of students who really did learn alot. They don’t know why nor the relative percentages yet.
The researchers also say, perhaps unsurprisingly, that what mattered most was how hard students worked. “Measures of student effort trump all other variables tested for their relationships to student success,” they write, “including demographic descriptions of the students, course subject matter, and student use of support services.”
It’s not surprising, but it is relevant. Students need to make effort to learn. New college students, especially first generation college students (i.e., whose parents have never gone to college), may not know how much effort is needed. Who will be most effective at communicating that message about effort and motivating that effort — a video of a professor, or an in-person professor who might even learn your name?
As Gary May, our Dean of Engineering, recently wrote in an op-ed essay published in Inside Higher Ed, “The prospect of MOOCs replacing the physical college campus for undergraduates is dubious at best. Other target audiences are likely better-suited for MOOCs.”
On the freshman-composition MOOC, Karen Head writes:
No, the course was not a success. Of course, the data are problematic: Many people have observed that MOOCs often have terrible retention rates, but is retention an accurate measure of success? We had 21,934 students enrolled, 14,771 of whom were active in the course. Our 26 lecture videos were viewed 95,631 times. Students submitted work for evaluation 2,942 times and completed 19,571 peer assessments (the means by which their writing was evaluated). However, only 238 students received a completion certificate—meaning that they completed all assignments and received satisfactory scores.
Our team is now investigating why so few students completed the course, but we have some hypotheses. For one thing, students who did not complete all three major assignments could not pass the course. Many struggled with technology, especially in the final assignment, in which they were asked to create a video presentation based on a personal philosophy or belief. Some students, for privacy and cultural reasons, chose not to complete that assignment, even when we changed the guidelines to require only an audio presentation with visual elements. There were other students who joined the course after the second week; we cautioned them that they would not be able to pass it because there was no mechanism for doing peer review after an assignment’s due date had passed.