Is learning to program inherently hard?
April 14, 2010 at 10:41 am 15 comments
In our educational technology class yesterday, we read and discussed a classic paper by John Anderson, Albert Corbett, Ken Koedinger, and Ray Pelletier, Cognitive Tutors: Lessons Learned from The Journal of the Learning Sciences (1995, 4(2), 167-207). This paper presented 10 years worth of data on cognitive tutors, including the Lisp Tutor. When the Lisp Tutor was tested in 1984, tutor-using students completed exercises 30% faster and performed 43% better on a posttest. In 1991, they did a more careful evaluation under a structure that was more like a real course. In that one, tutor-using students completed exercises 64% faster and still did 30% better on a posttest. Wow!
Yet, the students weren’t really learning to program. Yes, they learned Lisp really well. They knew nothing about debugging. They didn’t know anything about going from a problem to a program. In class, we made the good argument that those limitations are good things. The Lisp Tutor succeeded because it made the task manageable. CS1 has too large a cognitive load.
But here’s the question we got to wondering about: Could you build a cognitive tutor for all of programming? Cognitive tutors teach process, like problem-solving. They guide students through a process using a technique they call “model-tracing.” Designing is not a fixed process — there is no single path, and it involves tradeoffs. Debugging is an immensely difficult task, requiring the programmer to internalize a dynamic mental model of the program. These aren’t traceable problem-solving processes.
CHI2010 is going on this week here in Atlanta. I’m actually not attending (too expensive for too little that’s close to what I do), but I am following the Twitter feed and reading some of the papers. One of those is a really interesting paper on Toque: designing a cooking-based programming language for and with children. A group of researchers at U. Maryland College Park (with the always-intriguing Allison Druin) worked with children to design a programming language for themselves and other children, which they programmed using a visual notation input through body motions (tracked via a Wiimote). As I scanned through the paper, the headline leaped out at me,“Confusing and Boring”: Loops. Even when you have gifted designers, non-textual languages, and no keyboards, loops are just hard.
So what makes programming so hard to learn? Here’s a possibility: It’s inherently hard. Maybe the task of programming is innately one of the most complex cognitive tasks that humans have ever created.
Is it really harder than most other tasks? In considering this premise, I keep coming back to debugging. Physics wants students to develop a mental model of the physical world, one where gravity tugs and friction resists and electromagnetism is understood even if never seen. We have evidence that not all students really develop this complete mental model. However, using equations that can be applied plug-and-chug, and a limited model, students can get by, and even take jobs using this more-limited model of Physics understanding.
How do you debug without really understanding how the code works? How do you debug at all without developing a mental model of the program? There are still cognitive scientists who disbelieve that humans actually develop executable/runnable mental models at all. I bet someone could prove that they exist using computer programming, because programmers have to have them to successfully understand and fix program behavior.
I don’t really believe that programming is the most cognitively complex activity humans have created. I am wondering about how hard it is, how to measure that complexity, and how the challenge of computing education may be greater than the challenge of other forms of STEM education.
Entry filed under: Uncategorized. Tags: cognitive science, cognitive tutors, computing education, end-user programming, programming, visual programming.
15 Comments Add your own
Leave a comment
Trackback this post | Subscribe to the comments via RSS Feed
Computing Ed Research - Guzdial's Take 
1.
Erik Engbrecht | April 14, 2010 at 1:11 pm
I don’t think programming, at least in the small, is very hard. But it requires a mode of thinking that’s hard to acquire later in life, much like learning a new natural language. Programming is very precise and people just aren’t just to it.
I also don’t think systems thinking is particularly hard. It takes some coaxing, but I think people who haven’t been had the tendency to think about the big picture and to question assumptions actively beaten out of them by authority figures can generally do it. In other words, I think people naturally do it, but it is often punished so it appears to go away.
Larger programs are harder to write because they require the same precision as small programs while requiring systems thinking at the same time. So “real programming” requires one mode of thinking that’s hard to acquire late in life, and other that’s actively punished, especially early in life.
So I don’t think programming is inherently hard. Not in the least. But by the time a student is sitting in CS1 there’s a good chance ~18 years of neglect and abuse have conspired to make it hard for the student.
2.
Alan Kay | April 15, 2010 at 12:37 pm
Hi Mark,
I think this is an important blog post, but it’s hard to find a short worthwhile reply.
Rementioning several favorite themes:
1. “hard or easy” for *whom*? Even if Americans don’t like the idea of ability* being unevenly distributed, in education we have to heed this as a main principle of developing approaches to teaching.
2. “Hard” can be good if the surmounting of it builds a better brain/mind. E.g. most scientists who have studied reading and compared literate and illiterate societies have concluded that the literate societies not only have more knowledge, and more access to knowledge, but also think about ideas qualitatively differently than oral societies. (I think this is true for science learning also — and I think there is a form of computing that could bestow similar special benefits as well.)
3. “Gratuitously Hard” is a bad idea and bad design. This quartz lamp can and should be aimed at programming languages, curricula, pedagogy, and IDEs.
4. Quite a big of programming is *coping* with side-conditions — someone else has chosen the machine, language, problem to be worked on, operating system, and DE, and this conglomeration can give rise to bugs all by itself. *Coping* is *not* a good mind-set for getting the perspective to be critical enough to improve things via invention (in fact, I think it masks even the idea of real change).
In the old days of machine code (in my case early 60s), one was taught one’s first machine in a 40 hour week, and first programs were often done to the plan of someone else’s flowchart. (The people who did the flowcharts were called “programmers” and the people who made and debugged the programs were called “coders”.)
In the Air Force, they allowed beginning coders to try to design programs as well, and the transition to good design (and the skills to do some systems programming) progressed over several years. Part of design back then was learning how to use macroassemblers to make functional chunks that could be used as building blocks — this was a kind of language extension that was thought to be part of designing.
After 5 years of this in the AF and at NCAR (while majoring in other subjects), I went to grad school and was shocked to find that there were really great routes to getting out from under many of the built-in pitfalls of *coping* with what I could now see were really bad ideas.
For example, the combination of Val Shorre’s Meta II paper, and Wirth’s Euler paper, showed how to make something more powerful than Algol much more easily than I had dreamed of. Sketchpad and Simula showed how to deal with complexity in much better ways than procedures and data structures. LISP showed how to think deeply yet simply about programming semantics. PLANNER showed how to think and do strategic programming as well as tactics. The Burroughs B5000 showed how to run these higher level systems on lower level hardware. And so forth.
The bottom line was that you didn’t have to put up with anything you didn’t like — once you made one of the top priorities to gain the knowledge and skills to make new systems and languages as the situations demanded.
Reflecting later, it was pretty clear that the progression in the Air Force and workaday programming could consume one’s life, and there was plenty to learn in this pursuit, but oddly and importantly, there wasn’t any obvious path from the architectures that seemed natural in this world to the ones that computer science had been inventing. They were really two different worlds of thought and discourse.
This is partly because the problems in CS were much more interesting and demanding than those in the workaday world. And partly because the CSers were lazier (in the best sense of the term) than we who were willing (or forced) to cope.
I hope that some of what is terribly wrong in CS education today is made more visible by the above ……
———-
* a combination of talent, learned skill and will
3.
lixkid | April 15, 2010 at 3:34 pm
Hi Alan,
In your Turing lecture (and in other occasions) you used the phrase “different kind of simplicity” to describe special techniques ARPA/PARC community used in handling complexity. In that context you also mentioned Maxwell’s equations and The Constitution of the USA which are scientific theories. How do you determine whether some work done in Computer Science is scientific or not and how much is your criteria influenced by particular view of ARPA/PARC community that has been realised in systems some of which you mentioned in your reply?
Recently I found on the Web notes from one of your talks in 1984 whose abstract was (cannot find a link right now):
“This talk is about the battle between Form and Content in Design and why “being smart” usually causes content to lose. “Insightful laziness” is better because (1) it takes maximum advantage of others work and (2) it encourages “rotating” the problem into its simplest essence — often by changing it completely. In other words: Point of view is worth 80 IQ points!”.
Do you still maintain similar view and how that relates to your comment that “CSers were lazier” in the reply?
4.
Michael Kölling | April 15, 2010 at 1:01 pm
I don’t have any real data to add, just my own meandering thoughts. So I’ll try to keep it short.
I can believe that programming is harder than many other tasks.
Typically, in any discipline, it takes a certain level of skill and knowledge to understand or do something. However, it takes a meta-skill, at a higher (and harder) cognitive level, to explain or teach that same thing. It is harder, since it requires the same understanding, plus an ability to rationalise, structure, formalise and serialise the concepts.
I might argue that programming is, by its very nature, on that higher cognitive level. Because you have to express anything you want to do to a machine, you essentially have to explain (rationalise, structure, serialise) it. It is not enough to understand a solution yourself, you have to go meta.
And even worse: it’s not even an intelligent listener on the other end, but a dumb, picky idiot (which we call compiler), that isn’t helping.
So, yes, in computing we are faced on the first day with a challenge that in many other disciplines only experts face after a long time of practice.
Best,
Michael
5.
Leigh Ann | April 15, 2010 at 4:48 pm
Marcia Linn when talking about the simulation work that she does with students talks about “desirable difficulties”. I believe that our goal as educators is to control the desirable difficulties our students are encountering and scaffold them to keep them in Vygotsky’s zone of proximal development.
Perhaps the difficulty with programming is we tend to miss out on that range. There is so many knowledge components that go into writing a single method, especially in modern languages, that we have often seen our students get overloaded from this “small” task.
I am currently in the process of building a tutor for looping as a part of a study I’m conducting. I think, given LOTS of time, we could build tutors for all of the topics in CS1 and work with appropriate cognitive models to make them cognitive tutors. Perhaps I’ll even do some of that work in the future. But even with the Algebra tutor which has been around for a long time, they are still exploring things like peer tutoring, self explanation, and other cognitive strategies that can help students problem solve.
I think that the question of whether we can build a tutor is easily answered. Not so easily answered is how do we start?
6.
Leandro | October 4, 2010 at 8:53 am
Can someone post a free link to Val Shorre Meta II paper? Thank you.
7.
Steven Shaw | February 18, 2011 at 2:14 am
http://www.ibm-1401.info/AlanKay-META-II.html
8.
JS | January 8, 2011 at 4:00 pm
I would take it a step further and say that programming isn’t just inherently hard; rather, programmatic thinking is either inherent or it isn’t. A kid who built a videogame in BASIC in 1980 would have no problem picking up Flash in 2010. Coding is 90% about having an existing mental framework for how to think about a process. There are people who go into a Logic 101 class and simply know the answers to every question in the textbook, as if they’re the most obvious thing in the world, while other people struggle with the first chapter.
I would also argue that an ability for process thinking doesn’t necessarily predispose one to coding. For my money, there’s nothing we do as coders, mentally, that isn’t also done by most skilled airplane mechanics, electricians, watchmakers and novelists. What these professions have in common is the ability to look through the noise of a complex system — “look through the code” — and see what hose hooks up to what paragraph if you will, where the problem is, and how it needs to be rewired.
9.
Revolves | January 8, 2011 at 11:58 pm
Programming, or anything else for that matter, is NEVER hard. If you understand everything, there is no “difficulty.”
From my own experience watching other students at my university struggle with programming, I understood that the primary problem was that people don’t know how to teach. As simple as that.
Many teachers don’t understand the difference between the language specification, the “documentation”, a “tutorial”, or a generic programming concept of a language while they teach students.
You don’t need to do extensive research to understand that. The only thing such complex “research” proves is that we have a hard time understanding why others don’t understand.
Normally, if you watch people who understand programming very well, you may find out that they started very young. They learned from Internet sources, maybe. There, they’re free to do whatever they want, however they want. Since they’re learning because they “want” to learn, and not because they “have” to learn just because it’s in their syllabus, they explore a lot.
Just take a look at how you began programming. There might be days when you sat on a single “hello world” type problem for ages. Trying to understanding how it works. Changing stuff here and there until it breaks or creates something new. And all of a sudden, you start “getting” it and can learn any new language probably in a day from thereon.
I thought I understood the importance of OOP, while I was trying to code a structured PHP application. However, I only realized the true significance of OOP when I was constantly trying to keep things straight and was thinking, “If I’d used OOP, I could have done this and that, and I wouldn’t have had a problem.”
You can help people memorize a thousand programming concepts. However, they’ll only learn if you truly understand the meaning behind them by stumbling upon those concepts themselves.
10.
Mark Guzdial | January 9, 2011 at 7:26 am
When working from one’s own experience, I can see why you might come to that assumption — that programming isn’t hard, and some people just teach it badly. There have now been three multi-national, multi-institutional attempts to measure student learning about computer science, as an attempt to control for individual teacher performance or individual institution’s curriculum. In none of the three was the average score “passing.” That suggests that something deeper is going on. I’m not sure why this blog post from April suddenly became popular (over 5K reads in the last 48 hours), but we returned to this topic, with actual data, often during 2010. See https://computinged.wordpress.com/2010/08/16/a-challenge-to-computing-education-research-make-measurable-progress/ and https://computinged.wordpress.com/2010/08/19/the-first-language-independent-valid-measure-of-cs1-knowledge-allison-tew-defends/ for two examples, with research data.
11.
Steven Osborn | January 9, 2011 at 12:56 am
It’s a lot like Chess or Poker, you can teach anyone the basic rules in a few minutes and they will be able to have some level of success, but it can take you a lifetime to master.
12.
Ingbert Floyd | January 9, 2011 at 8:18 pm
A friend of mine brought this post to my attention in the context of the question, “is programming a teachable/learnable skill, or is it a mystical ability?” This comment is a superficial response to that query.
From my background in cognitive science, I can tell you that people’s brains just work differently sometimes. Part of the endeavor in cog sci and experimental psychology is to figure out who/what/how/when/why, and how regularly. Many of the comments seem to derive from a basic assumption that all people’s brains work alike, and the only difference is in what their experience has been. This is unlikely to be a sound assumption. A simple counter-example is the fact that some people are naturally right-handed, and others are naturally left-handed, and when you train a person who is naturally left-handed to write right-handed, the difficulty they face in that writing is reflected in how they grip their writing implement.
Reflecting on my own experience both learning and teaching, I think it is not as simple as being “purely” teachable or not. Extrapolating from work done by Elizabeth Spelke on how children learn mathematics, I think there is probably a large opportunity to figure out how certain concepts build upon other concepts to help people understand the principles behind programming. However, I think that the cognitive styles mentioned in some of the comments also play a role. Some people are naturally oriented to paying exhaustive attention to detail. The people who suffer from an extreme version of this condition are typically called “obsessive-compulsive”. Not all people share this orientation. Add to that the pragmatic experience that is gained through learning-by-doing, and you start to form a complex picture of how knowledge of “how to program” may be built out of many different components/aspects of cognition.
So, quick answer, I don’t think it is as simple as being teachable vs. mystical ability. And that if we want to understand how to teach it better, we need to understand how each of these components (and likely what I have mentioned are not all the relevant components) fit together, and which are teachable, and which are not. Essentially, I think it’s an empirical question that we could investigate via a collage of detailed ethnographic studies coupled with some efforts at developing different teaching strategies and carefully monitoring how different students find each more or less useful.
13.
Mark Guzdial | January 9, 2011 at 8:32 pm
Ingbert, I think you’d find the “Koli Calling” conference interesting. It focuses on ethnographic approaches to studying issues in computing education. The Scandinavian countries, in particular, have several experts in phenomenographic methods who study differences in how people experience learning to program.
14. The physical world | How the world works | April 7, 2011 at 9:38 pm
[…] physical world Posted on April 7, 2011 by zeynel Someone in the interwebs writes: Physics wants students to develop a mental model of the physical world, one where gravity tugs and […]
15.
No, Really – Programming is Hard and CS Flipped Classrooms are Complicated: ITICSE 2016 Award-Winning Papers | Computing Education Blog | October 3, 2016 at 9:26 am
[…] title. He does a good job of exploring the evidence that programming is hard (and even uses this blog as a foil, since I’ve claimed several times that programming is hard), and overall, is a terrific […]