Excel programming for non-programmers: How mental models are developed

May 16, 2012 at 8:09 am 2 comments

This new system for end-user programming from MIT raises a question for me about users’ mental models, which I think is key for computing education (e.g., for figuring out how to do inquiry learning in computer science).

Imagine that you use the system described below:  You give the system some examples of text you want transformed, before-and-after.  You run the system on some new inputs.  It gets it wrong. What happens then? I do believe the authors that they could train the system in three examples, as described below.  How hard is it for non-programmers to figure out the right three examples?  More interesting: When the system gets it wrong, what do the non-programmers think that the computer is doing, and what examples do they add to clear up the bug?

Technically, the chief challenge in designing the system was handling the explosion of possible interpretations for any group of examples. Suppose that you had a list of times in military format that you wanted to convert to conventional hour-and-minute format. Your first example might be converting “1515” to “3:15.” But which 15 in the first string corresponds to the 15 in the second? It’s even possible that the string “3:15” takes its 1 from the first 1 in “1515” and its 5 from the second 5. Similarly, the first 15 may correspond to the 3, but it’s also possible that all the new strings are supposed to begin with 3’s.

“Typically, we have millions of expressions that actually conform to a single example,” Singh says. “Then we have multiple examples, and I’m going to intersect them to find common expressions that work for all of them.” The trick, Singh explains, was to find a way to represent features shared by many expressions only once each. In experiments, Singh and Gulwani found that they never needed more than three examples in order to train their system.

via Excel programming for nonprogrammers – MIT News Office.

Entry filed under: Uncategorized. Tags: , .

Fewer women in top U.S. tech jobs since 2010: Only half of survey respondents think it’s a problem EdTech Magazine’s Dean’s List: 50 Must-Read Higher Education Technology Blogs

2 Comments Add your own

  • 1. Neil Brown  |  May 16, 2012 at 10:15 am

    This doesn’t seem so much “programming for non-programmers” as “supervising machine learning for non-programmers”. With a white-box machine learning system (i.e. if it gave an easily-readable summation of the generated program) the mental model should be easier because the system can explain its thinking to you. But if it remains black-box, then forming the mental model must be very hard, especially for non-programmers. I wonder if it works best if the three examples are fairly straightforward, or if it the three examples are the difficult corner cases than a programmer would instinctively check?

    • 2. Rob St. Amant  |  May 16, 2012 at 2:40 pm

      This doesn’t seem so much “programming for non-programmers” as “supervising machine learning for non-programmers”.

      I think this is pretty much right. Singh and Gulwani’s work is an example of programming by example (PBE); they distinguish it from programming by demonstration (PBD) but not everyone does. In traditional PBE/PBD, the system observes the user walking through a sequence of steps to solve an example of a problem, and then generalizes those steps into a program or macro that can be repeat the steps on further examples. Singh and Gulwani look at the harder computational problem (which they call PBE) of mapping inputs directly to outputs, without information about the intermediate steps.

      But the goal of PBE/PBD usually isn’t to teach users how to reproduce what the system does on their behalf. Instead, it’s to accomplish some specific, repetitive task within the system, with less effort on the part of the user than carrying out the same task by hand.

      I think that this would be an interesting area to integrate with ideas about how to teach people to program. In Oblinger et al.’s augmentation-based learning approach, for example, users are shown a model of the procedure learned by the system and allowed to edit it. I think the assumption is that users know how to make sense of the model, though.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 9,052 other followers


Recent Posts

Blog Stats

  • 2,030,559 hits
May 2012

CS Teaching Tips

%d bloggers like this: