Learning to code is really learning to code something: One doesn’t just “learn programming” nor “learn tracing”

I asked a group of social studies educators what programming language(s) they might want to use in their classes. One of the interesting themes in the responses was “the same as what’s in math and science classes.” One teacher said that she didn’t want a “weird hierarchy” where there’s one programming language in STEM and another in “history and English” for fear they’d be seen as “dumbed down.” Another said that maybe teaching JavaScript in history class “would make history cool.”

There’s a belief in this theme that I think is wrong. Learning to program in science class probably won’t transfer without a bunch of work to programming in mathematics class, and programming STEM classes will probably be a very different thing than programming in the humanities classes. Even expert programmers learn to program in a domain, and have a hard time transferring that knowledge of programming between domains. Expertise is expertise in a domain.

My advisor, Elliot Soloway, was involved in some of the early studies that supported this claim. The first paper was “The role of domain experience in software design” by Beth Adelson and Elliot Soloway from 1985. I quote from the abstract:

A designer’s expertise rests on the knowledge and skills which develop with experience in a domain. As a result, when a designer is designing an object in an unfamiliar domain he will not have the same knowledge and skills available to him as when he is designing an object in a familiar domain.

In this study, they took expert software designers in various fields, and have them design systems in other fields. They also asked novice designers to do some of the same tasks. For example, maybe we have a software designer who has been building banking software, and another who has been designing real-time control systems. Now, let’s ask both designers to design an elevator control system.

What they found was that the designers in the new domain struggled. They stopped planning (e.g., making notes). When they were in the familiar domain, they would often visualize the working system (“simulation” in the paper). Novices didn’t. The experts didn’t when they were faced with a new domain. Experts in an unfamiliar domain looked much like novices. Now, experts in an unfamiliar domain were better than the novices at noticing constraints on the design, so something transferred.

The second paper is even more striking. “Empirical Studies of Programming Knowledge” (1984) by Elliot Soloway and Kate Ehrlich. From the abstract:

We suggest that expert programmers have and use two types of programming knowledge: (1) programming plans, which are generic program fragments that represent stereotypic action sequences in programming, and (2) rules of programming discourse, which capture the conventions in programming and govern the composition of the plans into programs.

When we teach programming, we tend to focus on the syntax and semantics of the language. We don’t explicitly teach plans — chunks of code that do something useful. But we expect students to figure them out. We rarely teach discourse rules. The domain-specific knowledge lies in both plans and discourse rules.

To test the claim about the importance of these discourse rules, they produce sets of two programs: Alpha and Beta. Alpha is a perfectly fine program. Beta breaks the rules. For example, if you see a variable initialized n := 0;, you would find it weird to later see read(n); (to input a new value for n). It’s not wrong. The code might work just fine — in fact, it does work just fine in the experimental construction of Beta. But the program breaks the rules of discourse. They write:

Notice that both Alpha version and the Beta version are runnable programs that in almost all cases compute the same values. Moreover, to an untrained eye their differences may even not be apparent; they always only differ by a very few textual elements.

Here’s an example of one Alpha and Beta — these both work. In this case, they do not do the same thing:

Beta isn’t wrong. It successfully computes minimum. However, it uses the variable max which is confusing. It breaks our discourse rule. The program does work.

Different domains use different standards and different styles of programming. Engineers using MATLAB rarely use FOR or WHILE loops, for example. Graphic designers writing JavaScript code use far more exception handling than we ever expected.

Soloway and Ehrlich showed these programs to “experts” (undergraduate juniors to graduate students) and novices (students in their first programming course). When asked questions about Alpha (e.g., “What goes in this missing line in the code?” Or “Do you remember that code that I showed you?”), experts do far better than novices. When asked questions about Beta, experts do essentially the same as novices (no statistically significant differences).

I find it particularly notable that the expert drop is steeper.  Experts rely heavily on cues like variable names, even more than novices. CS expertise is really expertise in the discourse rules.

If expert programmers “knew programming,” they should be able to just trace the code (“be a computer”) and answer the questions correctly. Instead, they struggle to understand what’s going on. They’re pretty much like a student in their first semester of programming. The experts know Alpha well because it’s just like all the other programs they’ve ever seen — they can pattern match, rather than reason about the code itself. The experts struggle with Beta. It’s kind of like the difference between humans and Econs. Econs can reason through code rationally. Humans rely on expectations.

These results also suggest that the question of “Does tracing come before writing?” is moot.  Tracing what?  The program matters.  Some programs are harder to trace than others — for everyone, and particularly, with expertise.  There is no generic “tracing skill.”

Conclusion: People don’t just learn “coding.” Programmers in general know plans and discourse rules. Break the rules and you just have the programming language — and even experts aren’t really good at just applying the syntax and semantics rules. No better than a novice. If you have enough expertise in different domains, then you can work in different domains. When you start programming in a new domain, you’re not that much different than a new programmer.

The social studies teachers I’m working with have a sense that students can just
“know JavaScript.” I don’t think that’s true. I think if I taught students to write JavaScript code to use Google’s Charts service for making data visualizations, it wouldn’t be much easier to teach them Web programming with React, to write scripts for Adobe Photoshop, nor to build simulations in Lively Web. It’s all JavaScript, and the syntax and semantics are the same in each — but in terms of what people really know and use (i.e., plans and discourse rules), it’s completely different.