Inventing a Worked Examples and Self-Explanation Method for CS Courses

June 27, 2012 at 1:56 am 13 comments

I sent this idea to the mediacomp-teach mailing list, and got a positive response.  I thought I’d share it here, too.

I’m trying a worked examples + self-explanations approach in my Media Computation Python class that started Monday (first time I’ve taught it in seven years!) and in my “Computational Freakonomics” class (first time I’ve taught it in six years).  Whether you’re interested in this method or not, you might like to use the resource that I’ve created.

As I mentioned here, I’m fascinated by the research on worked examples and on self-explanations. The idea behind worked examples is that we ought to have students see more fully worked out examples, with some motivation to actually study them. The idea behind self-explanations is that learning and retention is improved when students explain something to themselves (or others), in their own words.  Pete Pirolli did studies where he had students use worked examples to study computer science (explicitly, recursion), and with Mimi Recker, prompted CS students to self-explain then studied the effect.  In their paper, Pirolli and Recker found:

“Improvement in skill acquisition is also strongly related to the generation of explanations connecting the example material to the abstract terms introduced in the text, the generation of explanations that focus on the novel concepts, and spending more time in planning solutions to novel task components. We also found that self-explanation has diminishing returns. “

Here’s the critical idea: Students (especially novices) need to see more examples, and they need to try to explain them.  This what I’m doing at key points in the class:

  • Each team of two students gets one worked example in class. They have to type it in (to make sure that they notice all the details) and explain it to themselves – what does it do? how does it work?
  • Each team then explains it to the teams on either side of them.
  • At the end of the class, each individual takes one worked example, and does the process themselves: Types it in, pastes it into a Word document (with an example of the output), and explains what the program does.  I very explicitly encourage them to do with this others, and talk about their programs with one another.  I want students to see many examples, and talk about them.

Sure, our book has many examples in it, but how many students actually look at all those examples? How many type them in and try them?  Explain to themselves?

I’m doing this at four points in the MediaComp class: for images with getPixels, images with coordinates, sounds, and text and lists. For my CompFreak class, students are supposed to have had some CS1, and most of them have seen Python at least once, so I’m only doing this at the beginning of the class, and only on text and lists.  There are 22 students in my MediaComp class, so I needed 11 examples in class, then 22 examples one-for-each-person. Round it off to 35 examples. That’s 140 working examples. A lot of them vary in small ways — that’s on purpose. I wanted two teams to say, “I think our program is doing about the same thing as yours — what’s different?”

I did discover some effects that surprised me. For example, try this:

def changesound(sound):
   for sample in getSamples(sound):
     value = getSampleValue(sample)
     if value > 0:
       setSampleValue(sample, 4 * value)
     if value <= 0:
       setSampleValue(sample,0)

Turns out if you zero out all the negative samples, you can still hear the sound pretty clearly.  I wouldn’t have guessed this.

Whether you want to try this example-heavy approach or not, you might find useful all these examples.  I’ve put all 140 examples on the teacher MediaComp sharing site (http://home.cc.gatech.edu/mediacomp/9 — email me if you want the key phrase and don’t have it). I started creating these in Word, but that was tedious to format well. I switched to LaTeX, because that nicely formatted the Python without much effort on my part. I’ve uploaded both the PDF and the LaTeX, since the LaTeX provides easy copy-paste text.

My CompFreak students are doing their assignment now (due tonight), and we just did it for the first time in the MediaComp class today (the take-home portion due in two days).  I was pleased with the feedback.  I got lots of questions about details that students don’t normally ask about at the second lecture (e.g., “makeColor is doing something different than setRed, setGreen, and setBlue differently? What’s the difference between colors and pixels?”).  My hope is that, when they start writing their own code next week, they won’t be stymied by stupid syntax errors, because they will have struggled with many of the obvious ones while working with complete code.  I’m also hoping that they’ll be more capable in understanding (and thus, debugging) their own code.  Most fun: I had to throw the students out of class today.  Class ended at 4:10, and we had a faculty meeting at 4:30.  Students stayed on, typing in their code, looking at each others’ effects.  At 4:25, I shooed them off.

I am offering extra credit for making some significant change (e.g., not just changing variable names) to the example program, and turning that in, too (with explanation and example).  What I didn’t expect is that they’re relating the changes to code we’ve talked about, like in this comment from a student that just got turned in:

“I realized I made an error in my earlier picture so I went back and fixed it. I also added in another extra credit picture. I made a negative of the photo. It looks pretty cool!”

It’s interesting to me that she explicitly decided to “make a negative” (and integrated the code to do it) rather than simply adding/changing a constant somewhere to get the extra credit cheaply.

All my MediaComp students are Business and Liberal Arts students (and is 75% female — while CompFreak is 1 female and 9 males).  I got a message from one of the MediaComp students yesterday, asking about some detail of the class, where she added: “We all were pleasantly surprised to have enjoyed class yesterday!”  I take the phrase “pleasantly surprised” to mean that the expectations are set pretty low.

Entry filed under: Uncategorized. Tags: , , , , , , .

Yay, College Board! 857 Desks Call Attention to Dropout Problem The changes the University of Virginia must make: Support for Board of Visitors

13 Comments Add your own

  • 1. Leigh Ann  |  June 27, 2012 at 2:40 am

    Mark, Have you read Michelene Chi’s Self-Explanations: How Students Study and Use Examples in Learning to Solve Problems? (Cognitive Science 1989) Its an awesome paper and really gets at what parts of the problem need to be explained to foster greater knowledge. The lit review at the beginning is one of the best I’ve ever read.

    Reply
    • 2. Mark Guzdial  |  June 27, 2012 at 6:09 am

      Yes, I did read it, back when it was new 🙂 Remember: I’m not a young’un. I started my PhD in 1989, and my first advisor (Pat Baggett) is an empirical cognitive psychologist. Kate Bielaczyc was a graduate student same time as me (and Ken Koedinger graduated soon after I started). Mimi Recker was a post-doc for Janet Kolodner when I got to Georgia Tech in 1993.

      I remember reading Cognitive Science for the first time, and wondering “What are they SAYING?!?” I was a computer scientist, and realized that “goals,” “plans,” and “scripts” meant something very different in those papers than in anything else I had ever read.

      Reply
  • 3. Deepak Kumar  |  June 27, 2012 at 7:53 am

    Mark, you are attending faculty meetings at Oxford???

    🙂

    Reply
    • 4. Mark Guzdial  |  June 27, 2012 at 1:26 pm

      For study-abroad faculty, yes.

      Reply
  • 5. Bonnie  |  June 27, 2012 at 8:11 am

    This is really interesting. I would love to see the examples. Did you generate them yourself, or have TAs do it?

    Also, how did you grade this? And did you show your students how to explain a program beforehand? I’ve tried to do some similar things a few times, and find that my students have far more trouble writing explanations than coding. Their explanations tend to be extremely shallow because they don’t have the language skills.

    Reply
    • 6. Mark Guzdial  |  June 27, 2012 at 1:36 pm

      Examples of the student work? I’ll see if anyone’s willing to let me share their turnins. I did them myself. I wanted them to explore the space of topics in the chapter(s) being covered, and to vary in small but significant ways.

      For example: Today, we discussed their homework in class. I asked if anyone thought their program was particularly interesting or confusing. One student had a program that he wanted me to demo that had constants of 150 (being used to generate RGB color values 0..255). Another student said, “My program was really similar, but where he has 150, I had 100. It was different, but not a lot.” Yup, that was the kind of insight I wanted them to get — what changes are small, and what changes are large.

      I haven’t graded them yet — that’s this weekend’s project. I expect to be very lenient. It’s struggling with the programs, comparing them, and discussing them that I most care about. I did do it for them in class, to give them an example. The ones I’ve seen (where a student brings it up to me and says, “Is this close?”) have been surprisingly good. I suspect that it’s because they compared and discussed with their friends. So far (and I’ve only seen four of 22 students), the students have nailed what made their program unique from others. I didn’t ask them to tell me that, but it’s coming out naturally when they try to figure out why their program is different from someone else’s.

      We’re going to do this three more times in the MediaComp class — tomorrow on using x,y coordinates for manipulating pictures (e.g., redeye removal, chromakey, mirroring, scaling), next week on sound, and the following week on text and list manipulations. I hope that the program explanations get more sophisticated as we go along.

      Today, I got feedback from my CompFreak course on what they thought about the worked examples. They were mostly “meh.” Last time I taught CompFreak, it was 12 liberal arts majors, 12 business majors, and one CS major. This time, all but one student is a CS major or minor. See what Pirolli and Recker say about diminishing returns from worked examples. I don’t think it’s so useful for more advanced programmers, even if they haven’t done much with Python.

      Reply
  • 7. Bonnie  |  June 28, 2012 at 8:35 am

    Thanks. I wasn’t interested in the student work, though if you grade to a rubric, that would be interesting. I just wanted to see the programs themselves. I think I could only do this if I walked through an example of an explanation first. My students tend to freeze up and do nothing at all if they don’t know exactly what I want. This kind of thing seems to be most useful in CS1 and 2. Athough, when I taught software engineering this year, we were working with a large existing codebase, so I asked my students to do a lab in which they answered questions explaining parts of the codebase. I was appalled at how bad their code reading skills were.

    Reply
  • […] should give you a little report on how my worked examples/self-explanation intervention worked in my Media Computation class.  I have nothing close to real data, and you shouldn’t […]

    Reply
  • […] my monthly Blog@CACM piece this last weekend, which was a synthesis of several pieces I wrote here: About the worked examples that I’m trying out in Oxford, the PixelSpreadsheet, and contrasting the study abroad I’m teaching on and MOOCs.  I […]

    Reply
  • […] I wouldn’t really call what Kahn Academy has “scaffolding,” at least, not the way that Cindy and I defined it, nor in a way that I find compatible with Wood, Bruner, and Ross’s original definition.  There’s not really a tutor or a teacher.  There are videos as I learned from this blog post, and later found for myself.  The intro video (currently available on the main Kahn Academy page) says that students should just “intuit” how the code works.  Really?  There’s a lot more of this belief that students should just teach themselves what code does.  The “scaffolding” in Kahn Academy has no kind of process modeling or guidance, nothing to explain to students what they’re doing or why, nothing to encourage them to explain it to themselves. […]

    Reply
  • […] decide to try some of the techniques Mark Guzdial calls “worked examples and self-explanations” in my introductory programming course.  Some of the benefits he mentions are that the […]

    Reply
  • […] I’m teaching Python MediaComp, first time in 8 years on campus.  We have just shy of 300 students, and I have 155 in my lecture.  While I’m a big fan of worked examples, the way I’ve used them in small classes of 30-40 won’t work with 155. […]

    Reply
  • 13. Python – Worked Examples | Kevin Wassong  |  November 6, 2014 at 6:35 am

    […] von Mark Guzdial habe ich einige Worked-Examples für dein Informatikunterricht in Python zusammengestellt. Ich […]

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed


Enter your email address to follow this blog and receive notifications of new posts by email.

Join 4,376 other followers

Feeds

Recent Posts

Blog Stats

  • 1,596,403 hits
June 2012
M T W T F S S
« May   Jul »
 123
45678910
11121314151617
18192021222324
252627282930  

CS Teaching Tips


%d bloggers like this: