Why programming in a non-majors, CS course is unlikely to lead to computational thinking (but is still a good idea): We must go beyond Intuition to Evidence
February 23, 2015 at 7:22 am 12 comments
The March 2015 issues of Inroads (linked here) has a special section on “The role of programming in a non-major, CS course.” I was disappointed by several of the articles in the special section for making arguments without empirical evidence, and decided to write my February Blog@CACM article on the need for evidence-based practice in computing education (see post linked here).
I left out Henry Walker’s second article in the Blog@CACM post, and will discuss it here. In the first article, he argues against teaching programming because it would not leave enough time for other, more important topics. In the second one, he argues for teaching programming, if your learning objective is computational thinking.
If a non-majors course in computer science seeks to help students sharpen their skills in computational thinking, then students must be able to write solutions precisely. Further, students must be able to analyze the correctness of solutions and compare alternative solutions. Such work requires precision in writing. English or another natural language allows precision, but does not require precision.
Like in his first article, Henry offers no evidence for his claims. I do agree that programming requires greater precision than natural language. Henry argues for a value of the use of programming that is not supported by our research evidence.
If defined in sufficient detail, pseudo-code can enforce rigorous thinking, but pseudo-code cannot be run to check correctness or test efficiency. Ultimately, the use of a programming language is essential if computing courses are to help students sharpen their problem-solving skills.
In the decades of studies that have tried to find such transfer, the research evidence is that computing courses do not help students sharpen their problem-solving skills. I am not aware of studies that have rebutted David Palumbo’s 1990 review of the literature on programming and problem-solving (see paper reference here). It is possible to teach problem-solving skills using programming, but students do not gain general problem-solving skills from computing courses (with or without programming).
Henry’s evidence that this does happen is an anecdote:
An upper-level political science major who took my introductory computer science course indicated that her logical thinking in computer science had a clear payoff when she put arguments together for her papers in political science.
As a rationalization for a teaching decision, this is weak evidence. It’s self-report from a single student. The student probably did learn something interesting and useful from the class. Maybe the student did gain in logical thinking. Given the preponderance of evidence against general problem-solving skills coming from a programming class, I’m skeptical. Maybe she just saw her existing skills in a new light because of the computer science class — a useful learning outcome. In any case, is the positive experience of one student justification for designing a course for dozens or hundreds?
The conclusion of my Blog@CACM post still applies here. We don’t know what non-CS majors need or what they can learn. We shouldn’t just guess, because our intuition is very likely wrong — and it’s dangerous, since our experience (as mostly white, mostly male, CS faculty) is so different than those of most non-CS majors. We need the humility to admit that we don’t know. We must seek out evidence to inform our decision-making.
Entry filed under: Uncategorized. Tags: computing for everyone, non-CS majors, non-majors.
12 Comments Add your own
Leave a comment
Trackback this post | Subscribe to the comments via RSS Feed
Computing Ed Research - Guzdial's Take 
1.
shriramkrishnamurthi | February 23, 2015 at 7:29 am
The simplest, best reason to teach programming is so that students can _program_. That will let them actually run things on a computer that give them a sense that the machine can do things—with a good curriculum, even _useful_ things—at their command. Run a longitudinal study of people who have and not had that experience and see whether their experience, five years out, is affected by that (or alternatively, what kind of programming it takes to have that effect). That’s what really matters, not any number of zero-time-lapse evaluations.
2.
Mark Guzdial | February 23, 2015 at 7:38 am
I agree that the assessment you’re describing would be valuable. I’d be fine with a wide variety of evidence-gathering approaches, as long as we gathered evidence.
3.
lenandlar | February 23, 2015 at 8:00 am
This topic is close to me at the moment because we are at curriculum review stage of 2 non-majors course and one of the big question is whether we should include some programming and if so, what programming language and how much of it.
We spoke briefly to the Departments(Education, Health Sciences, Engineering) who will be sending their students to these courses. Their needs vary. In their 2nd year Electrical Engineering students take a full programming course in C but the other Engineering programs do not require this course. The EE students are expected to do low level programming of control systems i suspect. In year 1 they all take this one non-cs course in the first year.
Interestingly, one of the lecturers from the health sciences noted it’d be nice for their students to have some foundation programming skill because it will help when they take a subsequent course on medical image processing (using MATLAB) i think.
So from a utilities perspective, some prior programming skill (computational thinking?) might be an advantage.
4.
Kathi Fisler | February 24, 2015 at 6:57 am
I think we often mistakenly focus on the outcome of programming (i.e., producing a program) rather than the process of programming. As I understand the transfer literature, process is what transfers: a learner has to understand how a process learned in one domain can apply to another; this in turn requires enough structural similarity between the process as needed in the two domains to enable transfer (it also requires explicit instruction in the similarity to make the transfer occur).
If we are going to make headway on having programming help with (some) general problem-solving skills, we need to develop more explicit processes of developing programs AND show that they
correspond to problem solving methods that apply elsewhere. If we only discuss “programming skill” without this process, we’re unlikely to see results. Students can often get to programs that seem to work with copy/paste plus trial and error, but that isn’t a useful process.
We suspect that deep process alignment such as this underlies our positive transfer results from programming to solving algebra word problems in Bootstrap. We’ll be presenting on this at SIGCSE next week (Sat 10:45).
5.
Peter Donaldson | February 24, 2015 at 1:58 pm
Even before reading this, I had strong itch to write a paper forensically pulling apart prior research on the broader benefits of learning CS; particularly looking at studies that tried to link it to improving general problem solving or thinking skills.
Leaving aside the way in which the CS learning experiences in these experiments were structured, there are a number of serious issues with attempts to use general problem solving as an impact measure. The most glaring of these is whether such as thing as general problem solving ability exists to any great degree or whether well structured domain knowledge accounts for the majority of differences in performance. Throw in issues within the broader research literature about how difficult it is to achieve far transfer and you create a situation where you’re unlikely to find any statistically significant correlations or relationships.
Failed experiments can often tell us as much, or more, about a particular area of interest but I think it’s led to a number of unpleasant side effects within and outwith the CS education community. The most frustrating is how patchy research into more reasonable and appropriate claims are and how these studies allowed CS and programming in particular to become marginalised or entirely absent in primary and secondary curricula; “it’s difficult (we didn’t/possibly still don’t know how to effectively teach the concepts) and it doesn’t really improve their general problem solving skills so why bother”.
@Kathi I agree that learners picking up the process of how to produce a solution is vital otherwise you end up with very inflexible knowledge that may not even transfer to other programming problems never mind anything else.
@shriramkrishnamurthi definitely. A variety of different academics and public figures have been predicting the death of programming for decades. What seems to have happened is that the number of other non-IT professions that engage in writing scripts and other more substantial programs has increased. Fundamentally that’s because writing programs allows you to directly use computation as a flexible medium of expression. Any action that I can describe in programmatic terms can be partially or fully automated. This is the mental equivalent of replacing direct human physical labour with partial or fully mechanised solutions during the industrial revolution. Increasingly, being able to repeatedly carry out routine information process myself is losing it’s economic value and instead being able to think about, use or describe computational processes is key. Simple examples of this include computer controlled phone systems that answer or carry out routine requests. No one in a cinema these days will answer the phone just to let you know what films are on and when they’re scheduled.
6.
Lisa Kaczmarczyk | February 26, 2015 at 2:31 pm
Hi Mark – Henry and Micheal have written a response to this post over on the Inroads blog http://tinyurl.com/mwqspgy FYI
7.
Starting from the students to build engaging computing courses for non-CS majors: Response to Goldweber and Walker | Computing Education Blog | March 5, 2015 at 8:38 am
[…] Goldweber and Henry Walker responded to my blog posts (here in Blog@CACM and here in this blog) in the Inroad blog (see article here). My thanks to them for taking the time to respond to me. I […]
8.
Repeatability as a Core Value in March CACM: For Software and Education | Computing Education Blog | April 6, 2015 at 8:24 am
[…] valued evidence and repeatability as much as we need to for rigor and scientific advancement — in education, too. One of my favorite papers by Michael Caspersen is his Mental models and programming […]
9.
Why We Need Learning Engineers and Faculty who Know Learning Sciences | Computing Education Blog | May 27, 2015 at 8:46 am
[…] I agree strongly with the idea of “learning engineers.” Having learning engineers doesn’t relieve faculty who teach from the responsibility to learn more about learning sciences (see my blog post about testing teachers about PCK). Just teaming up subject-matter experts with learning engineers does not inform a teacher’s day-to-day and in-class decision-making. The general theme below is one I strongly agree with — we should rely more on evidence-based and research-based teaching. […]
10.
Little Evidence That Executive Function Interventions Boost Student Achievement: So why should computing? | Computing Education Blog | June 1, 2015 at 7:45 am
[…] is pervasive in our community. (See a screenshot of my Google search below, and consider this blog post of a few weeks ago.) But there is no support for that belief. If this study finds no evidence that explicitly […]
11.
Making Computer Science More Inviting: A Look at What Works from Evidence | Computing Education Blog | June 12, 2015 at 8:32 am
[…] Lucy Sanders is terrific as always in this NYTimes piece. I particularly like that the article draws on evidence, which is too rarely used in making CS Ed decisions. […]
12.
Why should we teach programming (Hint: It’s not to learn problem-solving) | Computing Education Blog | October 18, 2017 at 12:30 pm
[…] problem-solving skills, and it is simply not there. I talk about these in my book, I reference the Palumbo meta-review in this blog post, and NYTimes wrote about it this last spring. Like “learning styles” and “Latin teaches […]