Accounting for the Two Humps
September 22, 2009 at 10:11 am 18 comments
One of the features of CS1 outcomes that has had teachers perplexed for years is the “Two Humps” phenomenon: Some students really get it, and some don’t. Grade curves for CS1′s typically have two humps, one for each group of students. What causes that? The mythical “Geek Gene” is one explanation.
Anthony Robins has an alternative explanation. He thinks that it’s because everything we do in intro programming builds on the earlier stuff. If you ever get behind, you’re toast — you can’t do the later homework without getting the earlier homework right.
Michael Kölling has built on Anthony’s explanation to develop a new way of teaching his CS1, which he calls Quality-oriented teaching of programming. The idea is to think about succeeding in the class in terms of number of programs completed, rather than achieving a passing grade on all or most homework assignments.
I haven’t read Anthony’s paper yet, but am looking forward to it. I’ve tended to believe that the two humps are caused by a priori knowledge, but missing out on the early stuff would lead to a similar effect.
Entry filed under: Uncategorized. Tags: computing education, Geek Gene, Two Humps.
1.
Erik Engbrecht | September 22, 2009 at 11:14 am
Do the humps persist if the class is taught with a language that students are unlikely to have learned on their own or in high school, such as Common Lisp, Smalltalk, or Scheme?
2.
Mark Guzdial | September 22, 2009 at 1:14 pm
I don’t know of anyone who’s studied this formally. My experience suggests that the language doesn’t matter. We’ve seen double-hump curves no matter how esoteric the language, and I’ve seen definitely seen students with high school experience in programming showing up in both humps.
3.
Alfred Thompson | September 22, 2009 at 11:32 am
I have noticed this in my own teaching. In high schools I have seen it with students who came in with no knowledge of programming at all. In fact I was in the second hump myself as a student. Things just do not click the same way for everyone. Sooner or later, with good teaching and finding different ways to explain things, it usually does click.
What I observed in a very unscientific way was that once things click for someone they catch up very quickly. Their achievment spikes so by the end of the semester most people are at the same level. Well at least the bell looks like you’d expect.
4.
Greg Wilson | September 22, 2009 at 12:28 pm
What other subjects exhibit double-hump curves? Can we work backward from that to find similarities in content/presentation that might be general explanations?
5.
Alan Kay | September 22, 2009 at 2:29 pm
Are we again in a phase where the idea that some small percentage of any group will be “more favorably predisposed by nature” towards certain kinds of actions and thinking is not “politically correct”?
I hope not.
To me “pedagogy” is about how to deal with the other sortings of predispositions. I think there are three that are worth finding solutions for.
(a) those who with some additional scaffolding help and skill building can now deal fluently with the subject matter
(b) those who need a lot of scaffolding and skill building to get fluent with the subject matter
(c) those who need enormous amounts of aid (or perhaps no extra efforts will help).
Most of us can relate strongly to this. Some things we just “got” and some things we are very interested in nonetheless required a lot more effort to get fluent. Somethings even after lots of effort seem intractable (at least for our initial aspirations).
There is a lot of actual scientific evidence for this in cognitive science.
So we really have a kind of a matrix between “initial predispositions” and “intrinsic motivations” that imply the need for a number of special approaches to any subject we are trying to teach widely. In some cases we have to scaffold predispositions, in others: motivations, in some: both, and for those happy few: neither.
For subjects such as reading & writing, mathematics, science, etc., we just have to find ways to get all but the tiniest fraction to fluency.
I see these as much more of an amalgam than separate subjects, where Education with the big “E” is concerned. For me, computing fits into these, and as another facet of the amalgam, not as a separate subject.
Best wishes,
Alan
6.
Jeff Graham | September 25, 2009 at 3:02 pm
I’m not sure there is really a lot of dispute about your three groups. I think many of us might just see too many in the last group, particularly in intro programming classes. What I think we argue about is how many are really in group c versus how many we could help if we knew how.
7.
Alan Kay | September 25, 2009 at 3:26 pm
Hi Jeff,
My experience has been — in the case of 9-13 year olds (which are the children we have the most experience with) — that the outer groups (very well set up and very poorly set up) are no larger than 10%, and that the majority in the middle can really be worked with. And, this correlates very well with the even more studied learning abilities for reading and writing, non-verbal reasoning, etc.
One of the simplest things that is often not addressed is that a person’s ability to remember different kinds of things has a huge effect on rate of learning whether cognitive or physical.
The students who have trouble here (and visualization is somewhat related) can be greatly aided by just making the display show the difficult to remember stuff (this is PARC GUI 101 but is not often carried through to programming).
So we can (and should) show all the state, bindings, histories, etc. of what is being worked on.
Most programming languages have assignments to variables with the variable and the expression close, but not passing of parameters. OTOH, e.g. Python can show an exception here, but mars it by using the = sign. We can readily imagine much better ways to make these things much more uniform and visible.
To me these are all parts of a reasonable GUI for programming (and not just for beginners).
Cheers,
Alan
8.
Michael Kölling | September 29, 2009 at 3:38 pm
I don’t think it has much to do with being (or not being) politically correct. While I think that some are just more able than others to learn programming, this does not seem to explain the unusual observations in programming courses.
As you say, people are more or less talented for many things, yet in most subjects we do not see such a strong bipolar distribution. In intro programming classes, however, this seems common.
So there clearly is something different about either the talent for programming (as opposed to other predipositions) or programming itself.
Since people have been looking for special aptitude and predictors in personal histories for decades, with very limited success, I am quite excited by the alternative hypothesis that it’s not the people who are special (to a larger degree than in any other subject), but it’s a discipline.
It should be possible to verify this by looking for other disciplines with similar grade distributions and comparing the nature of the material within them.
Robbins has some short mention of this in his paper: He mentions computational mathematics and molecular chemistry.
I think it’s just interpretation of an actual observed effect, not political correctness.
Regards,
Michael
9.
Alan Kay | September 30, 2009 at 8:59 am
Hi Michael,
I certainly don’t know the answer here.
Before the GUI, we would see something like the “double hump” in observations about those “who would use a computer or not”. The GUI bridged the gap for those who for whatever reason didn’t want to learn and be able to remember what was required before.
And we get a double hump from a normal curve distribution if we look at “exceptional behavior” (this curve would be gotten by turning the curve upside down to emphasize the “non-normals”). The two standard deviations outside of “normal” each account for about 17% of the total (which is near enough to Mark’s “20%”).
The National Literacy Foundation some years ago did a survey that indicated only “20%” of Americans could read a well written fairly short discourse on a vital subject (in this case it was Tom Paine’s “Common Sense”). If you took this as one “hump”, then I think you would find another one at the absolute bottom of “functional illiteracy” (which I think used to be about 23% in the US).
From the various kinds of surveys of Americans wrt mathematics and science, we get something like a double hump (with the “fluent hump” much smaller).
I’m guessing that if you forced “real math” and or “real science” on all college entrants, then you would get a double hump that might be a bit more balanced. This was sometimes done “long ago” in State colleges (which had to admit all) to trim down the students.
In the late 50s and early 60s, IBM had a “programming aptitude test” that was a great predictor of success. It was used both in industry and in the armed services. As far as I know, there were no studies to see what percentage of those who did not pass the test could learn to program anyway, and with what kind of aid.
Our own experience over about 40 years of teaching children to program can be roughly summarized by positing the need for “different and more” scaffolding for each of the standard deviations, and that you can get 80-90 fluency if the various kinds of scaffolding is carefully done. This is very similar to the much more studied areas of reading and writing, and other required subjects.
So to me, the real question is not so much whether there are normal distributions of predispositions (and even “luck”) for different subject areas (it would really be surprising not to see these), but whether what is needed can be supplied by scaffolding and more practice (and the time and energy and space to allow these to be done).
I do think there is enough evidence to support existence of non-uniform distribution of abilities, and to also be justifiably optimistic that inventions such as reading, writing, mathematics, and programming, which are difficult for some, can still be taught successfully to a healthy majority of students.
I also “believe” (am pretty sure) that part of the pedagogy of these subjects for those who are likely to have difficulties has to involve cleaning up the many “gratuitous difficulties” which can be so distracting, infuriating, and discouraging.
For example, I’m very sure that trying to teach “poorly designed commercial and hacker languages” along with their terrible UIs to the general public is a complete disaster, and just shouldn’t be attempted.
Best wishes,
Alan
10.
Leigh Ann | September 22, 2009 at 4:43 pm
This is actually aligned with a question I am starting to see as a more integral underlying component of my research. In the theories of conceptual change we look at how people move from the novice level to expert level in a particular domain. Generally this migration is accompanied by the ability to abstract within the domain at increasingly more complex levels.
As computer science educators we are often asking students to abstract right away. There is no slow path up Bloom’s hierarchy – our most basic problems start several levels up and require that novices integrate several different categories of information and abstraction all at once.
I dont think there is a ‘geek gene’ that our exceptional students posses – instead I think that they have more experience with and are more comfortable working at the level of abstraction that we ask them to. Perhaps as we look at scaffolding the curriculum to remove the valley between the two humps, we need to pay closer attention to the amount of practice we give students at the lower levels of Bloom in order to foster the grounded understanding that will allow the abstraction to happen. I’ve always believed that the assignment trail for a particular concept should be a series of program assignments, starting with very small (in terms of what the student needs to add or change) concrete examples that slowly guide the student to the larger more abstract examples. Yet I often see many of my colleagues across the country giving a single large assignment that jumps right into the abstraction.
I’m not saying that we are not trying to scaffold with regards to ‘difficulty’ – I’m just claiming that’s different from abstraction levels. Intuitively if we redesigned our assessments to be more balanced with abstraction I believe the two hump phenomenon would diminish significantly.
11.
david klappholz | September 23, 2009 at 7:38 pm
I assume that when you say ” Intuitively if we redesigned our assessments to be more balanced with abstraction I believe the two hump phenomenon would diminish significantly.” what you mean is that if we redesigned our exercises to draw students up the levels of cognitive difficulty in smaller steps, then more students would end up in the “got it” group. (I don’t quite see what redesigning assessments has to do with this.)
As I posted on Mark’s wall, in response to someone’s surprise at the fact that there’s a nice bell curve in math courses: Sure, if the math course is Intro Calc, where a trivial algorithm for finding derivatives of polynomials is taught and exercises ask students to find derivatives of polynomials. I’ll bet that the first math course that requires proofs gets two-hump results.
12.
Chris Champion | September 24, 2009 at 1:06 pm
I certainly see Alan Kay’s three humps more than just two – and it’s that “c” hump that I struggle with the most – these are the students that have several similar problems:
Inconsistent spelling and/or poor spelling
txtCountry.visable is not the same as txtCountry.visible, nor is Sytem.out.println the same as System.out.println
/* this becomes a problem when the student doesn’t KNOW that they are spelling a word wrong – it’s not just a typo, it’s a true spelling issue */
Insufficient grasp of programming conventions
when do I use IF…THEN or DO…WHILE
/* it’s not just “when do I use “while” versus “until”, some don’t process the “why” of each convention */
Poor grasp of real-world problem solving
How do you calculate tax? How do you average a series of numbers?
/* one of my favorite programs to have students write in VB is the “McDonald’s Cash Register”. But each year I have several that would be confounded by math problems if handed the most simple ones */
////////////////
I sometimes wonder if there could be a litmus test to show when a student is ready to grasp these concepts. Is there a minimum reading level? Math score? Our school is prohibited from accepting students into classes based on grades in other classes.
At what point do you say to a guidance counselor, “I don’t think this is a good fit for Johnny”?
13.
Alan Kay | September 24, 2009 at 4:16 pm
Hi Chris,
I’m assuming (hoping) that you are at least referring to HS students and not college?
Cheers,
Alan
14.
Chris Champion | September 24, 2009 at 4:48 pm
Oh definitely high school – one would hope that college entrance exams would weed out the students that we HS teachers must deal with on a regular basis
15.
Alan Kay | September 24, 2009 at 5:17 pm
Hi Chris,
I figured wrt the spelling problems.
But it is interesting that some of the problems reported in first courses of computing in engineering universities — for example with variables — kind of imply that the math prep of some of the admitees is below any reasonable standard …
Cheers,
Alan
16.
Chris Champion | September 24, 2009 at 5:43 pm
Certainly – I frequently have to teach or refresh basic algebraic concepts where numbers are represented as variables – although my students sometimes report that if only their math teacher had replaced “x” with a real-world example they might have understood it better the first time. I’m sure we all try to use real-world examples when we are programming, even if it is to show a simple programming convention like a FOR…NEXT statement. Replacing “i” with “apples” for instance makes the lightbulb go on for many students.
The thing I struggle with the most is students who, despite practice and step-by-step guidance, cannot look at a problem and describe how to solve it using the compiler. For instance, after numerous examples at moving an element (a picture file in VB) to the top, bottom, left, and right of a form, each year I have students that just don’t grasp this (what I would consider) simple task (object.left = 0 will move it to the left side of the form).
I’m afraid to think about performing mathematical operators, or comparison statements with these students.
17.
David Klappholz | September 29, 2009 at 8:54 pm
Writing code to solve any but the most trivial problem is way high on any scale of cognitive demand/achievement than finding the derivative of a polynomial (Calc I).
Dave
18.
Alan Kay | September 30, 2009 at 10:46 am
A simpler way to look at it (maybe too simple).
(Apologies in advance for the length of this.)
After 43 years of not playing jazz guitar, I got motivated to take it up again last year. (I had been playing classical keyboards since about 1980, so I was not approaching music itself again from scratch).
Jazz guitar is one of those “10,000 hour skills” and has some really steep slopes, especially in the beginning.
I should mention that when I played professionally, I was one of those players who had to do a lot of practicing every day. This is because some of the “machinery” that is nicely developed for some players is not as well set up for others. In my case, certain kinds of memories (especially muscular) needed tons of rehearsal compared to some of my friends. I also needed to work on being able to play complex rhythmic schemes every day. On the other hand, “theory” (harmony counterpoint, etc.) was really easy for me and more difficult for some others.
This is completely normal in music. There are many different fluencies involved in being a musician, and it is a very rare person who is well set up for all of them (those few are wonderfully amazing when you encounter them). This is also very similar in sports. Most players practice daily, but what they work on are the different difficulties not covered by predispositions.
A lot of the ups and downs (still going on) of my experience over the last 18 months could be explained very well by the collision of “7+-2″ and tempo (with a few little complexity factors thrown in also.)
Taking improvisation as the biggest difference between most classical music and all of jazz, the start up was very much from scratch — I could essentially not remember anything except a few blues rifs — and the results depended more on *tempo* than almost anything.
(I also had a chance to compare this with improvising on the same songs on a piano. It was still very difficult, but also much easier, in no small part because the piano keyboard is really great for thinking in the combinations of harmony and melody that improv requires. By contrast — if the piano is kind of like writing with an alphabet — the guitar is much more like writing with literally thousands of ideograms: and having to deal with several forms and positions *for the very same notes and chords*.)
This is the deal with this instrument. (However, I did spend 2 months deciding whether or not to make the *several hours a day for several years* effort required to start getting really fluent again. I figured that regardless of the past it would be 2000-3000 hours to rebuild, and to also build from scratch that which had really gotten lost. I thought it would be difficult to do more than 700 hours in a year, given my day job.)
Having done this before (and actually having taught jazz guitar almost 50 years ago) I was not at all dismayed by this, but expected it. So I knew ahead of time that I could be successful if I put in the work, because I’d done it in the past. The bugaboo of “maybe I can’t” wasn’t present.
When I taught guitar, I would get the students to learn to play the keyboards a bit because they allow certain kinds of thinking to be done easier and earlier. (Why play guitar at all? Because, if you *can* play it, then it just sounds so much better and more musically expressive than keyboard! It’s “more organic, more intimate” and is in a different “tradeoff space”.)
The improvisation experience is rather like learning to drive a very complicated car with tons of controls in a complex landscape.
If the pace is really slow and there is a road and it is well lit, then things “work a little”. For example, improvising over a slow ballad is usually possible because the planning that used to be automatic can still be done consciously in time — not just to be aware of what is going on, but to look out into the “landscape” to think about where to go and how. This planning is easier on the piano than the guitar and this is reflected in what kinds of stuff gets to happen.
Adding in “you are driving at night” is a good analogy. One of the things that happens is the headlights go from 1 watt bulbs to much brighter illuminators of “the possibilities for routes”. “Getting used to the gear box” is another analogy. At some point more automatic muscle learning offloads many of the physical thinking that makes other kinds of thinking almost impossible.
Note that this very simple model will have sharp cutoffs and will put the practitioner in one hump or the other, sometimes from week to week. If the tempo is too fast, you may not be able to do anything. Slow it down by a factor of 2 and you might be able to do something musical.
“Slow practicing” of all kinds is the mantra you hear in all music learning whether classical or jazz.
And so on. I “believe” that these are real analogies to virtually all “fluency learning” for most learners in most subjects. And that the trade-offs between different predispositions and guided practicing and scaffolding are real and have to be heeded.
As a music teacher, to me the key was the extent to which the students came in liking music and being able to respond to it, perhaps via dancing or singing or beating time, etc. (This includes most human beings.) They could (a) all be helped to get to real fluencies on an instrument, and (b) where they could get depended greatly on the individual. This is also true for sports learning (c.f. the accomplishments of Tim Gallwey in different sports teaching).
We can see that the general school setup is not likely to be good for many students on “stuff that can’t be faked” unless they are highly predisposed. The tempos and individual needs don’t match the “factory assembly line”.
For general education, let me claim that these “first real fluencies” are what we are trying to achieve.
We must help most learners to achieve real fluencies in important subjects. So there are important thresholds that we have to help them get across. But these thresholds are extremely different in level and kind from what a much smaller percentage of the population — who will be experts and pros — can get to (regardless of the amount of work put in).
This seems OK to me.
Best wishes,
Alan