A Report on Worked Examples and Self-Explanations in Media Computation

I should give you a little report on how my worked examples/self-explanation intervention worked in my Media Computation class.  I have nothing close to real data, and you shouldn’t believe me if I offered any.  This is a rarified class: 22 students, meeting four days a week for 90 minutes, plus office hours for 90 minutes twice each week (that most of the students have come to), and the teacher (who is the author of the textbook) attends breakfast and dinner with the students.  I think it would be hard to get more student-teacher interaction than in this model.

That said, I would definitely do it again.  I was quite surprised at how seriously the students took the task of explaining these programs!  In retrospect, I shouldn’t have been surprised.  In most classes, aren’t students asked to analyze and explain situations, even asked to make sense of some text?  That’s exactly what I asked these students to do, and they really worked at it.  I had students coming to office hours to ask about their assigned programs, so that they could write up their one paragraph of explanation. There were things that I had to teach them about this process, e.g., teaching them to try a program with different data sets, to make sure that the odd result they got wasn’t an anomaly.  I gave them feedback (every single student, on every single program) about the quality of their explanations, and the explanations definitely got better over time.

The real benefit was that they were trying to understand some relatively complicated code before it was their own code that they were trying to understand (while also designing and debugging it, all before a deadline).   With the worked examples tasks, they were just trying to understand.  There clearly was a reduction in cognitive load.  Variations on the below program had lots of students coming to see me — combining sounds at different rates was a challenging idea, but students did a good job of getting a grasp on it:

def modifysound2(sound):
 retsound = makeEmptySound(2*getLength(sound))
 newsound = makeSound(getMediaPath("bassoon-c4.wav"))
 trgi = 0
 nsi = 0
 for i in range(getLength(sound)):
 value = getSampleValueAt(sound,i)
 if nsi < getLength(newsound):
 nsvalue = getSampleValueAt(newsound,int(nsi))
 else:
 nsvalue = 0
 setSampleValueAt(retsound,trgi,value+nsvalue)
 trgi = trgi + 1
 nsi = nsi + 0.5
 return resound

Because there four labs (that just involved explaining programs) and two homework’s (that involved typing in, executing, and explaining programs), the first real programming assignment was the collage assignment.  Everybody did it.  Everybody turned in a working program.  And some of these were huge.  This one (by Savannah Andersen) was over 100 lines of code:

This one, by Julianne Burch, is over 200 lines of code.  I’m posting shrunk versions here: Julianne’s is about 4000 pixels across, representing the travel portion of this study abroad program.

I suspect that the worked examples and self-explanations gave the students more confidence than they normally have when facing their first programs.  It’s unusual in my experience for students to be willing to write 50-200 lines of working code for their first programming assignment.

But some of these students were also getting it.  A few of my students realized that they could make their collages more easily by using a copy() method to reduce the complication of composing pictures.  I did prompt them to do that, and a few did — most just went with hard-coded FOR loops, because that was easier for them to understand.  When I described how to do that, one student asked, “Aren’t you just naming some of those lines of code?” Yes! Nice way to start thinking about functions and abstract: it’s about naming chunks of code.  One of my students, without prompting, also decided to create a copy() method for her sound collage.  They’re starting to grapple with abstraction.  Given that this is the third week of class, when none of them had any previous programming experience (all my students are liberal arts and management students), I think that they’re doing quite well at moving from notation into abstraction.

They’re working on their first midterm exam now, a take-home exam (to save classroom time.)  I think it’s significantly challenging for a first exam, but it doesn’t have much coding.  It has a lot of analysis of code, because that’s one of the key learning objectives.  I want them to be able to look at a piece of code and predict its behavior, to trace (if necessary) what’s going on.  For me, that’s a more important outcome from a first course than being able to write a lot of code.

Inventing a Worked Examples and Self-Explanation Method for CS Courses

I sent this idea to the mediacomp-teach mailing list, and got a positive response.  I thought I’d share it here, too.

I’m trying a worked examples + self-explanations approach in my Media Computation Python class that started Monday (first time I’ve taught it in seven years!) and in my “Computational Freakonomics” class (first time I’ve taught it in six years).  Whether you’re interested in this method or not, you might like to use the resource that I’ve created.

As I mentioned here, I’m fascinated by the research on worked examples and on self-explanations. The idea behind worked examples is that we ought to have students see more fully worked out examples, with some motivation to actually study them. The idea behind self-explanations is that learning and retention is improved when students explain something to themselves (or others), in their own words.  Pete Pirolli did studies where he had students use worked examples to study computer science (explicitly, recursion), and with Mimi Recker, prompted CS students to self-explain then studied the effect.  In their paper, Pirolli and Recker found:

“Improvement in skill acquisition is also strongly related to the generation of explanations connecting the example material to the abstract terms introduced in the text, the generation of explanations that focus on the novel concepts, and spending more time in planning solutions to novel task components. We also found that self-explanation has diminishing returns. “

Here’s the critical idea: Students (especially novices) need to see more examples, and they need to try to explain them.  This what I’m doing at key points in the class:

• Each team of two students gets one worked example in class. They have to type it in (to make sure that they notice all the details) and explain it to themselves – what does it do? how does it work?
• Each team then explains it to the teams on either side of them.
• At the end of the class, each individual takes one worked example, and does the process themselves: Types it in, pastes it into a Word document (with an example of the output), and explains what the program does.  I very explicitly encourage them to do with this others, and talk about their programs with one another.  I want students to see many examples, and talk about them.

Sure, our book has many examples in it, but how many students actually look at all those examples? How many type them in and try them?  Explain to themselves?

I’m doing this at four points in the MediaComp class: for images with getPixels, images with coordinates, sounds, and text and lists. For my CompFreak class, students are supposed to have had some CS1, and most of them have seen Python at least once, so I’m only doing this at the beginning of the class, and only on text and lists.  There are 22 students in my MediaComp class, so I needed 11 examples in class, then 22 examples one-for-each-person. Round it off to 35 examples. That’s 140 working examples. A lot of them vary in small ways — that’s on purpose. I wanted two teams to say, “I think our program is doing about the same thing as yours — what’s different?”

I did discover some effects that surprised me. For example, try this:

def changesound(sound):
   for sample in getSamples(sound):
     value = getSampleValue(sample)
     if value > 0:
       setSampleValue(sample, 4 * value)
     if value <= 0:
       setSampleValue(sample,0)

Turns out if you zero out all the negative samples, you can still hear the sound pretty clearly.  I wouldn’t have guessed this.

Whether you want to try this example-heavy approach or not, you might find useful all these examples.  I’ve put all 140 examples on the teacher MediaComp sharing site (http://home.cc.gatech.edu/mediacomp/9 — email me if you want the key phrase and don’t have it). I started creating these in Word, but that was tedious to format well. I switched to LaTeX, because that nicely formatted the Python without much effort on my part. I’ve uploaded both the PDF and the LaTeX, since the LaTeX provides easy copy-paste text.

My CompFreak students are doing their assignment now (due tonight), and we just did it for the first time in the MediaComp class today (the take-home portion due in two days).  I was pleased with the feedback.  I got lots of questions about details that students don’t normally ask about at the second lecture (e.g., “makeColor is doing something different than setRed, setGreen, and setBlue differently? What’s the difference between colors and pixels?”).  My hope is that, when they start writing their own code next week, they won’t be stymied by stupid syntax errors, because they will have struggled with many of the obvious ones while working with complete code.  I’m also hoping that they’ll be more capable in understanding (and thus, debugging) their own code.  Most fun: I had to throw the students out of class today.  Class ended at 4:10, and we had a faculty meeting at 4:30.  Students stayed on, typing in their code, looking at each others’ effects.  At 4:25, I shooed them off.

I am offering extra credit for making some significant change (e.g., not just changing variable names) to the example program, and turning that in, too (with explanation and example).  What I didn’t expect is that they’re relating the changes to code we’ve talked about, like in this comment from a student that just got turned in:

“I realized I made an error in my earlier picture so I went back and fixed it. I also added in another extra credit picture. I made a negative of the photo. It looks pretty cool!”

It’s interesting to me that she explicitly decided to “make a negative” (and integrated the code to do it) rather than simply adding/changing a constant somewhere to get the extra credit cheaply.

All my MediaComp students are Business and Liberal Arts students (and is 75% female — while CompFreak is 1 female and 9 males).  I got a message from one of the MediaComp students yesterday, asking about some detail of the class, where she added: “We all were pleasantly surprised to have enjoyed class yesterday!”  I take the phrase “pleasantly surprised” to mean that the expectations are set pretty low.