Creating (and improving) options for CS practice: Practice-It! and beyond

June 25, 2010 at 11:29 am 5 comments

One of the (several!) pedagogical methods that I learned about at the DCCE meeting a couple weeks ago was Practice-It!, a new (to me) website from the University of Washington.  Practice-It! provides a variety of practice activities for students, from multiple choice questions, to predict-the-output problems, to exercises where students write a single method to solve a problem.  These help to fill the huge gap between reading the book and attending lecture on one side, and facing a full IDE (“a speeding compiler”) on the other side.  It joins pedagogical tools like CodingBat and Problets in an important, but surprisingly sparse area of tools for computing students.

I really like these tools and think that they fill an important role.  However, given that there is more than one tool in this space now, I have a criticism of all the existing tools, that I mean to be constructive.  Currently, the coding problems in these tools invoke a compiler or interpreter and return the error generated.  We can do better, and need to because the errors messages of virtually any interpreter or compiler presume a knowledgeable, professional programmer.  They are unclear, often useless, and always infuriating for a novice.

Here are a couple examples.  In CodingBat, I tried the Python problem where I have to write a function to determine if I can sleep in, depending on whether it’s a weekday or during vacation.  The inputs are booleans, but I tried (like many students) to write the function without reading the description of the inputs. I made assumptions about the inputs being objects and collections.  I compounded the error by writing the code “weekday is in vacation” as opposed to the correct “weekday in vacation.”  The error message isn’t useful.  My semantic error (of ignoring that the inputs are booleans) is hard (but doable!) to catch and address.  My syntax error is not helped by this message.

Practice-It! works similarly.  Here, I’m asked to write a Java method, and (again, as a student might) I decide to just get the basic method declaration in first — but get it wrong because I forget to deal with input parameters.

The error messages from the Java compiler are worse than useless.  They mention things like “enums” that I (as a student working on such a simple problem) have never seen.  To the credit of Practice-It!, they are collecting these awful error messages and trying to generate student “friendly” versions.

I wonder, though, if we can do even better than rewriting Java’s error messages. In each of these situations, we have a lot of knowledge about what code the student should be writing, what errors we might expect, and what the student knows already.  We should be able to tune the error messages to the problem.

Here’s the radical idea I’ve been exploring:  What about building our own parsers?  A parser for any of these problems does not have to be a parser for all of Java or Python. In fact, it shouldn’t be.  We know from lots of research (e.g., Lewis Johnson’s PROUST and Anderson’s Cognitive Tutors) that student answers to coding problems mostly fall in a small range of options, and those with radically different answers are far more likely to be radically wrong than brilliant-and-different-thinking — and letting beginning students flail with the all the flexibility of the full language is simply wasted time.  Only let the students type in a subset of the language, but provide understandable, informed error messages for that subset that are tuned to the problem.

As computer scientists, we might blanch at the complexity of writing parsers, remembering hours spent battling Lex and YACC.  Lex and YACC were written before 1975. What’s the possibility that we could do better in 35 years of development and Moore’s Law increases? I’ve been exploring OMeta lately for just this purpose — it makes it possible to build parsers for rich languages in surprisingly few lines of code.

The constructive recommendation that I have for computing educators building (or like me, considering to build) tools to fill the book-to-IDE gap: Be computer scientists, who build their own programming languages tools and can use these tools for improving education.  We don’t need to use a compiler as a monolithic piece of software.  We know the techniques used in building those compilers, and we can mix-and-match the techniques and components in our tools to help our students learn.

Entry filed under: Uncategorized. Tags: , , .

More teacher education vs. centralized control Dave Patterson on fixing high school CS education

5 Comments Add your own

  • 1. Barry Brown  |  June 25, 2010 at 3:14 pm

    Creating subsets of Java is a project I’ve been toying with for a few months. I’m convinced it could be a powerful tool for teaching beginners. There’s no reason at all that a novice should have access to the full language. It just leads to confusion, especially when they encounter error messages that can only be explained by a language feature they haven’t encountered yet.

    The How to Design Program folks have been using language subsets for years and it works well.

  • 2. Greg Wilson  |  July 16, 2010 at 5:01 pm

    I remember this being one of the weaknesses of SP/k (a series of ever-larger subsets of PL/1 created for teaching — see Each language subset only added one or two new features, and the error messages never used concepts that hadn’t been introduced, but they assumed more sophistication on the part of readers than they were likely to have. Of course, if programming systems were built as truly extensible toolkits, like GIMP and the Apache web server, rather than as a few monolithic “take ’em or leave ’em” blocks, this sort of thing would fall out naturally 🙂

  • […] bet that the Problets, Practice-It!, or CodingBat folks could get better results on either the Lister problem set or Allison’s new test.  My […]

  • 4. links for 2010-08-18 - Emerging Technologies Consulting  |  August 18, 2010 at 7:03 am

    […] Creating (and improving) options for CS practice: Practice-It! and beyond « Computing Education Blo… (tags: computing education) No comments for this entry yet… […]

  • […] method for an array of integers. These problems are similar to ones you might find in a system like CodingBat where students are given a method header and asked to implement code for the interior of a single […]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 10,184 other subscribers


Recent Posts

Blog Stats

  • 2,053,614 hits
June 2010

CS Teaching Tips

%d bloggers like this: