Why programming in a non-majors, CS course is unlikely to lead to computational thinking (but is still a good idea): We must go beyond Intuition to Evidence

The March 2015 issues of Inroads (linked here) has a special section on “The role of programming in a non-major, CS course.” I was disappointed by several of the articles in the special section for making arguments without empirical evidence, and decided to write my February Blog@CACM article on the need for evidence-based practice in computing education (see post linked here).

I left out Henry Walker’s second article in the Blog@CACM post, and will discuss it here. In the first article, he argues against teaching programming because it would not leave enough time for other, more important topics. In the second one, he argues for teaching programming, if your learning objective is computational thinking.

If a non-majors course in computer science seeks to help students sharpen their skills in computational thinking, then students must be able to write solutions precisely. Further, students must be able to analyze the correctness of solutions and compare alternative solutions. Such work requires precision in writing. English or another natural language allows precision, but does not require precision.

Like in his first article, Henry offers no evidence for his claims. I do agree that programming requires greater precision than natural language. Henry argues for a value of the use of programming that is not supported by our research evidence.

If defined in sufficient detail, pseudo-code can enforce rigorous thinking, but pseudo-code cannot be run to check correctness or test efficiency. Ultimately, the use of a programming language is essential if computing courses are to help students sharpen their problem-solving skills.

In the decades of studies that have tried to find such transfer, the research evidence is that computing courses do not help students sharpen their problem-solving skills. I am not aware of studies that have rebutted David Palumbo’s 1990 review of the literature on programming and problem-solving (see paper reference here). It is possible to teach problem-solving skills using programming, but students do not gain general problem-solving skills from computing courses (with or without programming).

Henry’s evidence that this does happen is an anecdote:

An upper-level political science major who took my introductory computer science course indicated that her logical thinking in computer science had a clear payoff when she put arguments together for her papers in political science.

As a rationalization for a teaching decision, this is weak evidence. It’s self-report from a single student. The student probably did learn something interesting and useful from the class. Maybe the student did gain in logical thinking. Given the preponderance of evidence against general problem-solving skills coming from a programming class, I’m skeptical. Maybe she just saw her existing skills in a new light because of the computer science class — a useful learning outcome. In any case, is the positive experience of one student justification for designing a course for dozens or hundreds?

The conclusion of my Blog@CACM post still applies here. We don’t know what non-CS majors need or what they can learn. We shouldn’t just guess, because our intuition is very likely wrong — and it’s dangerous, since our experience (as mostly white, mostly male, CS faculty) is so different than those of most non-CS majors. We need the humility to admit that we don’t know. We must seek out evidence to inform our decision-making.