Are MOOC Students Cheating Or Mastering the Material? « Gödel’s Lost Letter and P=NP
August 22, 2012 at 9:29 am 10 comments
I hadn’t heard about this form of cheating in MOOC’s. I knew that answers got passed around (as Dave Patterson reported in June), but was surprised to hear that students were creating multiple account in order to re-take exams. That changes one’s perception of the 100K registered users. The question raised here in Dick Lipton’s blog is: Is this “cheating” or simply “mastering” the material?
Here is what happens next. Bob signs up for the course multiple times: let’s call them Bob1, Bob2, Bob3, Bob4. Recall there is no cost to Bob for signing up multiple times—none. So why not sign up several times…
Bob’s insight is simple: he now can take the course multiple times and keep only the best grade. Say there is a graded exam. Bob1 takes the exam and gets a 70% on it. Not bad, but not great either. So Bob sees what he got wrong, sees what questions they threw at him. He studies some more, then takes the exam again as Bob2. Of course the exam is different, since all these on-line systems do some randomization. However, the exam covers the same material, so now Bob2 gets an 85% say.
Perhaps Bob is satisfied. But if he is really motivated he studies some more, retakes the exam, and now Bob3 gets 90%. You guessed right. He goes on and takes it one more time as Bob4 who—surprise—gets a perfect 100%.
via Cheating Or Mastering? « Gödel’s Lost Letter and P=NP.
Entry filed under: Uncategorized. Tags: cheating, computing education research, MOOC, undergraduates.
Computing Ed Research - Guzdial's Take 
1.
gfrblxt | August 22, 2012 at 9:47 am
Interesting. I wonder how this differs from the idea of retaking tests in school until mastery of the material is shown. Along the same lines, could this somehow be turned from a bug in MOOCs into a feature?
2.
alanone1 | August 22, 2012 at 9:57 am
“Mastering” is a good idea (see the rest of Dick Lipton’s blog).
To the extent that a test means something worthwhile (and it can), then the system should never offer the same test at the same level twice, but dynamically put together the test from a very large number of possible questions and parameters.
Using this scheme, the person who takes the course over and over is not learning how to take one test, but is arguably learning how to deal with a much larger range of examples, and this is starting to resemble getting good at the subject rather than with the one test.
Cheers,
Alan
3.
Krishna | August 22, 2012 at 10:44 am
I think this is one place where ‘social networks’ really help by providing a slightly higher barrier to cheating. An e-mail id is ‘cheap’ and sure it is easy to spend a few minutes creating bob1@gmail all the way to bob19@gmail. What we do at unclassroom.com is we require that students either upload a student id (we’re based in India so we know every college issues one to the student. also, we’re student facing as opposed to these MOOCs which seem to be aimed at any willing learner) or they should sign-in with a Facebook account and should have at least 50 friends. The id is really for people who don’t have a FB account and the FB sign-in provides a low friction alternative to those students that are active users of that network. While this isn’t 100% effective, it presents a tougher barrier and also there are straightforward improvements that can make this even more resilient.
4.
Sheila MacNeill (@sheilmcn) | August 22, 2012 at 10:56 am
I don’t think this is cheating really – it’s just drill and practice till you get 100% isn’t it? If this is really an issue then the design of the assessments should be changed to stop this happening.
5.
Dennis J Frailey | August 22, 2012 at 12:32 pm
This is an example of what I mean when I say you have to rethink your method of teaching to fit the distance education, on line paradigm. Here’s how I reduce cheating in this type of course: the main point is to make sure they do things in their own words rather than repeating something from the course notes or textbook or reference material. On student assignments I expect them to produce a unique version of a particular artifact (a program or a report or an analysis) and can readily spot whether two of them are the same or similar enough to suggest collaboration). [I can also pick out unique phrases and search my data base of assignments from prior semesters to see if they might have copied the answer from a previous student. I’ve caught a few this way!] On exams I require them to answer questions by explaining things in their own words, as to a colleague or supervisor or customer. For example: “explain to your boss why reused or “open source” software isn’t really free and show how to estimate the true cost in the following situation [situation described here].” I also sometimes give different students slightly different exams, especially when computation is involved, so that it is easy to see if they just copied from someone else. [In the above example, I might give different quantities or different percentages in describing the situation]. Yes, they can have someone else do it all for them, but simply copying from other students won’t work very well, and the person answering the question must understand the situation well to give an acceptable answer. Such questions test whether they really know the answer, not just whether they memorized something.
Not perfect, but generally effective.
6.
Alfred Thompson | August 22, 2012 at 12:40 pm
I think involved a confusion about the reasons for evaluations such as exams. Is the goal about helping students know what information they don’t understand for for faculty to understand what is not being covered well? Or is the goal of exams grades for the report card? All too often, especially in K-12, students learn more about how to get good grades than to actually learn. How often to students forget much of the material once the exam is over?
I like the idea of retaking an exam until there is mastery but unfortunately most systems are not set up for that sort of learning because we want students to take one exam and then give them a single grade based on that score. If a student is taking a MOOC in order to learn, which seems likely given that there is little in the way of credit beyond a certificate of completion that is fairly meaningless compared to a transcript or a degree, then this is more good than bad.
7.
KWRegan | August 22, 2012 at 10:34 pm
Note a sentence I included with reference to the slides by L.P. Norman(-Mackay): “The slides also discuss other students signing up as ‘sacrificial lambs’ and dropping at the end, which need not involve multiple accounts.” How does one combat this variation on the multi-take idea? This dates from 2006 already.
8. Musing on MOOCs « Nick Falkner | August 23, 2012 at 2:32 pm
[…] blog contains a number of posts where he looks at Massive Open On-line Courses (MOOCs) but a recent one on questionable student behaviour made me think about how students act and, from the link where students sign up multiple times so […]
9.
Sebastian Thrun bets education over driverless cars « Computing Education Blog | August 28, 2012 at 9:32 am
[…] asked him if he’s seen Dick Lipton’s blog on cheating vs mastery. He said that he had and that Udacity doesn’t work like that anymore. Students taking an […]
10.
How Computerized Tutors Are Learning to Teach Humans – NYTimes.com « Computing Education Blog | October 12, 2012 at 9:25 am
[…] a correct answer if the tutor suspects guessing” might be an excellent approach to deal with possible cheating in MOOC’s), but also liked the interview with Ken Koedinger later in the article which suggests a more […]