Why programming in a non-majors, CS course is unlikely to lead to computational thinking (but is still a good idea): We must go beyond Intuition to Evidence

February 23, 2015 at 7:22 am 11 comments

The March 2015 issues of Inroads (linked here) has a special section on “The role of programming in a non-major, CS course.” I was disappointed by several of the articles in the special section for making arguments without empirical evidence, and decided to write my February Blog@CACM article on the need for evidence-based practice in computing education (see post linked here).

I left out Henry Walker’s second article in the Blog@CACM post, and will discuss it here. In the first article, he argues against teaching programming because it would not leave enough time for other, more important topics. In the second one, he argues for teaching programming, if your learning objective is computational thinking.

If a non-majors course in computer science seeks to help students sharpen their skills in computational thinking, then students must be able to write solutions precisely. Further, students must be able to analyze the correctness of solutions and compare alternative solutions. Such work requires precision in writing. English or another natural language allows precision, but does not require precision.

Like in his first article, Henry offers no evidence for his claims. I do agree that programming requires greater precision than natural language. Henry argues for a value of the use of programming that is not supported by our research evidence.

If defined in sufficient detail, pseudo-code can enforce rigorous thinking, but pseudo-code cannot be run to check correctness or test efficiency. Ultimately, the use of a programming language is essential if computing courses are to help students sharpen their problem-solving skills.

In the decades of studies that have tried to find such transfer, the research evidence is that computing courses do not help students sharpen their problem-solving skills. I am not aware of studies that have rebutted David Palumbo’s 1990 review of the literature on programming and problem-solving (see paper reference here). It is possible to teach problem-solving skills using programming, but students do not gain general problem-solving skills from computing courses (with or without programming).

Henry’s evidence that this does happen is an anecdote:

An upper-level political science major who took my introductory computer science course indicated that her logical thinking in computer science had a clear payoff when she put arguments together for her papers in political science.

As a rationalization for a teaching decision, this is weak evidence. It’s self-report from a single student. The student probably did learn something interesting and useful from the class. Maybe the student did gain in logical thinking. Given the preponderance of evidence against general problem-solving skills coming from a programming class, I’m skeptical. Maybe she just saw her existing skills in a new light because of the computer science class — a useful learning outcome. In any case, is the positive experience of one student justification for designing a course for dozens or hundreds?

The conclusion of my Blog@CACM post still applies here. We don’t know what non-CS majors need or what they can learn. We shouldn’t just guess, because our intuition is very likely wrong — and it’s dangerous, since our experience (as mostly white, mostly male, CS faculty) is so different than those of most non-CS majors. We need the humility to admit that we don’t know. We must seek out evidence to inform our decision-making.

Entry filed under: Uncategorized. Tags: , , .

The Individual Teacher versus the Educational System: What if Finland’s great teachers taught in U.S. schools? More diversity and more progress with CS teachers vs just on-line: Code.org

11 Comments Add your own

  • 1. shriramkrishnamurthi  |  February 23, 2015 at 7:29 am

    The simplest, best reason to teach programming is so that students can _program_. That will let them actually run things on a computer that give them a sense that the machine can do things—with a good curriculum, even _useful_ things—at their command. Run a longitudinal study of people who have and not had that experience and see whether their experience, five years out, is affected by that (or alternatively, what kind of programming it takes to have that effect). That’s what really matters, not any number of zero-time-lapse evaluations.

    • 2. Mark Guzdial  |  February 23, 2015 at 7:38 am

      I agree that the assessment you’re describing would be valuable. I’d be fine with a wide variety of evidence-gathering approaches, as long as we gathered evidence.

  • 3. lenandlar  |  February 23, 2015 at 8:00 am

    This topic is close to me at the moment because we are at curriculum review stage of 2 non-majors course and one of the big question is whether we should include some programming and if so, what programming language and how much of it.

    We spoke briefly to the Departments(Education, Health Sciences, Engineering) who will be sending their students to these courses. Their needs vary. In their 2nd year Electrical Engineering students take a full programming course in C but the other Engineering programs do not require this course. The EE students are expected to do low level programming of control systems i suspect. In year 1 they all take this one non-cs course in the first year.

    Interestingly, one of the lecturers from the health sciences noted it’d be nice for their students to have some foundation programming skill because it will help when they take a subsequent course on medical image processing (using MATLAB) i think.
    So from a utilities perspective, some prior programming skill (computational thinking?) might be an advantage.

  • 4. Kathi Fisler  |  February 24, 2015 at 6:57 am

    I think we often mistakenly focus on the outcome of programming (i.e., producing a program) rather than the process of programming. As I understand the transfer literature, process is what transfers: a learner has to understand how a process learned in one domain can apply to another; this in turn requires enough structural similarity between the process as needed in the two domains to enable transfer (it also requires explicit instruction in the similarity to make the transfer occur).

    If we are going to make headway on having programming help with (some) general problem-solving skills, we need to develop more explicit processes of developing programs AND show that they
    correspond to problem solving methods that apply elsewhere. If we only discuss “programming skill” without this process, we’re unlikely to see results. Students can often get to programs that seem to work with copy/paste plus trial and error, but that isn’t a useful process.

    We suspect that deep process alignment such as this underlies our positive transfer results from programming to solving algebra word problems in Bootstrap. We’ll be presenting on this at SIGCSE next week (Sat 10:45).

  • 5. Peter Donaldson  |  February 24, 2015 at 1:58 pm

    Even before reading this, I had strong itch to write a paper forensically pulling apart prior research on the broader benefits of learning CS; particularly looking at studies that tried to link it to improving general problem solving or thinking skills.

    Leaving aside the way in which the CS learning experiences in these experiments were structured, there are a number of serious issues with attempts to use general problem solving as an impact measure. The most glaring of these is whether such as thing as general problem solving ability exists to any great degree or whether well structured domain knowledge accounts for the majority of differences in performance. Throw in issues within the broader research literature about how difficult it is to achieve far transfer and you create a situation where you’re unlikely to find any statistically significant correlations or relationships.

    Failed experiments can often tell us as much, or more, about a particular area of interest but I think it’s led to a number of unpleasant side effects within and outwith the CS education community. The most frustrating is how patchy research into more reasonable and appropriate claims are and how these studies allowed CS and programming in particular to become marginalised or entirely absent in primary and secondary curricula; “it’s difficult (we didn’t/possibly still don’t know how to effectively teach the concepts) and it doesn’t really improve their general problem solving skills so why bother”.

    @Kathi I agree that learners picking up the process of how to produce a solution is vital otherwise you end up with very inflexible knowledge that may not even transfer to other programming problems never mind anything else.

    @shriramkrishnamurthi definitely. A variety of different academics and public figures have been predicting the death of programming for decades. What seems to have happened is that the number of other non-IT professions that engage in writing scripts and other more substantial programs has increased. Fundamentally that’s because writing programs allows you to directly use computation as a flexible medium of expression. Any action that I can describe in programmatic terms can be partially or fully automated. This is the mental equivalent of replacing direct human physical labour with partial or fully mechanised solutions during the industrial revolution. Increasingly, being able to repeatedly carry out routine information process myself is losing it’s economic value and instead being able to think about, use or describe computational processes is key. Simple examples of this include computer controlled phone systems that answer or carry out routine requests. No one in a cinema these days will answer the phone just to let you know what films are on and when they’re scheduled.

  • 6. Lisa Kaczmarczyk  |  February 26, 2015 at 2:31 pm

    Hi Mark – Henry and Micheal have written a response to this post over on the Inroads blog http://tinyurl.com/mwqspgy FYI

  • […] Goldweber and Henry Walker responded to my blog posts (here in Blog@CACM and here in this blog) in the Inroad blog (see article here). My thanks to them for taking the time to respond to me. I […]

  • […] valued evidence and repeatability as much as we need to for rigor and scientific advancement — in education, too.  One of my favorite papers by Michael Caspersen is his Mental models and programming […]

  • […] I agree strongly with the idea of “learning engineers.” Having learning engineers doesn’t relieve faculty who teach from the responsibility to learn more about learning sciences (see my blog post about testing teachers about PCK). Just teaming up subject-matter experts with learning engineers does not inform a teacher’s day-to-day and in-class decision-making.  The general theme below is one I strongly agree with — we should rely more on evidence-based and research-based teaching. […]

  • […] is pervasive in our community.  (See a screenshot of my Google search below, and consider this blog post of a few weeks ago.)  But there is no support for that belief.  If this study finds no evidence that explicitly […]

  • […] Lucy Sanders is terrific as always in this NYTimes piece.  I particularly like that the article draws on evidence, which is too rarely used in making CS Ed decisions. […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed

Recent Posts

February 2015
« Jan   Mar »


Blog Stats

  • 1,268,963 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 4,568 other followers

CS Teaching Tips

%d bloggers like this: