What’s worse: Boredom or failure?
October 14, 2009 at 10:43 am 13 comments
I’m continuing to dig into the literature on design of instruction, informed by cognitive load theory and worked examples research. One of the analyses that these researchers often perform is to split the subject pool by performance on a pre-test, then look at how the high-scoring (sometimes called “high-ability” and sometimes just called “more knowledge” group) students compared to the low-scoring students. A good example is Mimi Recker and Pete Pirolli’s paper teaching recursion using a worked example approach.
The results are fairly predictable. High-ability students tend to prefer more bells-and-whistles, more flexibility, more learner control. Low-ability students don’t do as well with all the bells-and-whistles, and sometimes just downright fail. We just read a paper in educational technology this last week about how highly-flexible simulations simply frustrate low-ability students, and another paper about how interesting dynamic visualizations actually impeded learning in the low-ability students. On the other hand, in the less sophisticated, more directed instruction, high-ability students are bored — but importantly, everyone learns. High-ability students do learn when bored.
As an instructional designer, how do you choose? Lots of people say, “Don’t choose! Give students a choice!” I tried that in Emile — low-ability students often choose the environment that they are worse at. Metacognition is hard. And it’s really hard for the designer to build in such flexibility. Thus, designers typically have to choose: Do you bore your best students? Or do you fail more students? What’s particularly hard is that this burden is falling on different students — you’re comparing boredom of one group with failure of another.
Of course, squelching student creativity is an awful thing. It’s also an awful thing to live out life as a high-school or college drop-out. I think it’s particularly hard because we as teachers (at high-school or college) tend to empathize more with the best students. We were good students, or we wouldn’t be here now. We remember what it was like to be bored, to feel like your imagination was being leached away by mindless activity. We may not have had that experience of flunking out of activities that the smart kids did well at, while we were frustrated and deciding to give up.
What’s better for society, if we were to choose all-discovery (for high-ability) or all-structured (for low-ability)? Choose highly structured instruction, and you bore some students (but still teach them), and decrease your number of drop-outs. Choose open-ended, exploratory instruction, and you increase the number of drop-outs, but perhaps (it’s a gamble) dramatically inspire your high-ability students and they become great scientists and inventors. Both are tough choices, somewhat like Pascal’s argument for the belief in God. What helps society more, to be pretty sure that you’ll have fewer people on the dole, or to have a chance at more great discoveries and inventions?
Entry filed under: Uncategorized. Tags: cognitive science, computing education research, teachers.
Computing Ed Research - Guzdial's Take 
1.
Mark Miller | October 14, 2009 at 8:56 pm
Just an idea, but I think one way that schools have handled this in the past is they offered a few basic classes for the “low-ability” students to prepare them for more advanced classes. The “high-ability” students could test out of those classes and move on to more advanced stuff that would be interesting to them. Except here it would be about preparing them to exercise their mind differently. The thing is, does anyone on the faculty even know how to test preparedness for the more advanced approach?
The unfortunate thing about making it boring for the advanced students is I imagine they lose interest in the subject, and I don’t think it serves even the “low-ability” students.
Mathematicians understand that the arithmetic/math education schools have delivered for decades is a dumbed down version of what they practice. Kids generally understand it as frustrating, meaningless, and boring. This is true of lower level math courses in college as well. It’s not that the content is dumbed down, but rather the pedagogy that goes into it. One reason for this is that mathematicians believe “you either have it, or you don’t,” as if you’re born with the ability. Another reason is that the educational establishment has never really understood why mathematics is important. They just know that it is, so they do the best they can to teach it with their own limited understanding of it.
The educational establishment is satisfied with this, because they think people who are going to become scientists and engineers don’t need to truly understand mathematics. They just need to be able to use it competently. Mathematicians often come into their profession despite the way they were taught math. They came to understand what it really is on their own. This has not served the discipline well. It leaves the vast majority of people (low-ability, high-ability, and everywhere in between) not understanding it, and avoiding it whenever they can.
Your post talks about having to make a choice between making it boring and making it interesting, and that saddens me. Maybe I’m being (unrealistically) idealistic, but I side with keeping it interesting, with remedial education for those who are not mentally prepared to deal with it.
I understand what it’s like to see a beautiful, advanced concept in computing and reject it, because I didn’t understand it. I didn’t believe I could understand it. It felt like it was beyond my comprehension. What really changed my mind was people showing me why it’s beautiful (this was years after college). The first motivation was competitiveness and efficiency, and I was still motivated by the idea of “Maybe this can be useful for building applications”. You’ve already heard me talk about “Beating the Averages,” by Paul Graham. As I started learning about the ideas I came to understand the beauty a little bit, and that really made me want to understand more. I was told directly that I needed to change my perspective on computing, and that I needed to learn some concepts if I wanted to go further with this. After that I was motivated to change my mind, to try to see the interesting stuff for what it is. As you know, it’s much easier for a willing student to learn.
Have you explored why “low-ability” students tend to choose the wrong environment for themselves? I wonder if it’s because they want to challenge themselves, but they get in over their head. Maybe they recognized something interesting in it, or wanted to be thought of as smart (choosing what the “high-ability” students picked).
I haven’t met your students, but my guess is that the “low-ability” students, from how you describe your different efforts, came from an environment where they were given most of the knowledge they needed to accomplish a set of tasks, and then were given goals to accomplish using that knowledge, exercising their problem-solving ability. Their ability to “learn how to learn”, explore, and experiment was probably not exercised much. One way that a university education could really serve them is to introduce them to this other way of looking at a situation.
2.
Mark Miller | October 14, 2009 at 10:08 pm
What I saw in some of my college CS classes, and this could be an alternative to the remedial classes I talked about, is the course started out structured, but moved to unstructured as the course progressed. I found this helpful since there were subjects where I didn’t have much of a clue about what I was dealing with. The structure helped me a get a handle on that. I loved the unstructured part later, because it allowed me to experiment, explore the area, and learn some things on my own. So that might be a balance you could consider, as opposed to making a black or white choice.
I did have one CS course that I think is more along the lines of what you’re talking about, in terms of students struggling with an unfamiliar approach. Where the professor threw us into “the deep end of the pool” was requiring us to do our own research to complete assignments (as in, looking up our own materials to learn about the subject matter), and he wasn’t helpful about coaxing us along in that process. What we needed to know for our research was not in our textbooks. There were a few really smart kids in the class who seemed to thrive in this. Most of us were not used to this approach. What usually happened is a few students (including myself) would actually do the research, while everyone else either winged it, or mooched off of us (this wasn’t how it was supposed to go, but it was either this or mass “fail” carnage). I overheard one of the students saying that he snuck a peek at the score chart for the class, and half the students were failing. The professor ended up applying what was considered to be a large curve to our grades.
I’ve still thought about that course years later. An important part of the reason why most of the students did not do well is most of us were not prepared for the educational approach. I don’t believe it was that we were incapable of doing it. It’s just that when we were “handed the baton” most of us were left bewildered and wondering what was up. So I think some coaching/preparation would’ve gone a long way towards helping students “get with the program”.
3.
Erik Engbrecht | October 15, 2009 at 4:34 pm
Depends on your objective.
I think the idea that all CS students should start with the same intro class is rather preposterous. The two humps should be easy to sort out with a test.
But when do the two humps merge? How many classes?
4.
Mark Guzdial | October 16, 2009 at 8:16 pm
Such tests are really hard to write. Michael Caspersen’s dissertation has a nice review of the literature on predicting performance in CS1. A couple dozen papers and no consensus on a single factor that predicts success.
Allison Tew did a paper at ICER 2005 which showed all CS1 differences disappearing by the end of CS1. She hasn’t been able to replicate that result, though, which is what has led her to her current work to develop reliable tests of CS1 knowledge.
5.
Katherine Mancuso | October 16, 2009 at 11:41 pm
Do all high-ability students learn when bored? Are we sure about that? Is that a proposition that works? Obviously I have a pretty serious learning disability issue as well, but I personally don’t. I frequently act out, tune out, or disengage, which causes me suffering. And obviously people then assume that I wasn’t bored or wasn’t high ability in the first place, but often I was. Or sometimes I just plain don’t do the work because I never engage with the problem space of the class (sometimes against my struggles to make myself engage – but there are just so many more INTERESTING things to think about in the world than classes which aren’t engaging, particularly if most of the rest of your time is spent doing or thinking about things that can make an actual difference in the universe). Which can put me in a weird situation because I can intellectually understand a class’s material but still be flunking or failing to do the work. Grades for me are sometimes basically a punitive measure to force me to engage with problem spaces of required classes which are uninteresting. Sometimes I’m more likely to engage with a class that I don’t understand or am low-ability in than a class that I’m high-ability but bored in.
6.
Katherine Mancuso | October 17, 2009 at 12:28 am
I want to return to Mark Miller’s issue of “how are we teaching our students how to learn,” because I actually think that’s really important when we’re thinking about how students learn at the graduate level and at advanced undergrad schools. This is of course the source of curriculum movements like universal design for learning and university 101 but those don’t get everyone. I really think that many students don’t know what to do with research and self-directed learning tools that we as people focused on education assume they should know how to use – and that inability to use reference tools may be a confounding factor in some of this conversation. Basically, if you didn’t show the students how to use the library (or in the case of a computing language or a very focused problem domain perhaps show them how to use important textbooks and online references/docs for the language), I think that it may be a failure to assume that they know how to do it. People need to learn how to learn and how to access knowledge – metacognition – and I think teaching that process and measuring how it works is very very hard. I’m not sure I have any idea how to crack that nut at all. I can’t generalize from my own experience here – looking back on my years, I just sort of figured it out over time, and there are certain things and modalities that I still have difficulty learning – what’s irritating is going into a new class and having to figure out the modality that works for me because frequently my modalities don’t seem to be the same modalities other students use. SciTrain is definitely not working at this level at this point. Ideas, anyone?
7.
Katrin Becker | October 17, 2009 at 10:39 am
The school where I taught offered a single intro course for majors (like most schools) which I taught between 1998-2005. The course assumed no previous computing experience, and there were some huge classes during that time.
One approach that seemed to meet the needs of both the exceptional and the average (and struggling) students was to specify detailed ‘A’, ‘B’, and ‘C’ level requirements for each assignment. Many of the students treated it as a choice (which often encourages a feeling of ownership) – if they were dealing with heavy demands in other courses that week, they set their sights on the ‘C’ requirements. The ‘C’ requirements always covered the core concepts connected with that assignment so they still got what they needed to go on, but they felt less pressure, and were rarely surprised by their marks.
The ‘A’ requirements were always set so that they were within reach of the average students provided they worked at it. They sometimes came to me for extra help and would say something like, “I’m going for the ‘A’ solution and there’s a bit here I’m having trouble with.” The top students almost always went for the ‘A’ solution, of course.
For the really high-ability students I added an additional set of challenges. This was over and above the ‘A’ requirements and they were classified as ‘bonus’. I wanted to make it clear this was ‘extra’ and not part of the standard course requirements because I felt it was important to keep the top mark (an ‘A’) within reach of the average student. Most of the high-ability students end up with ‘A’s anyways, so they don’t actually NEED the extra work for marks. What they need is something to challenge them. For each assignment I would list a number of things they could do to embellish it and assigned each thing a point range. Most of these bonuses were in the 2-4 point range, with an occasional 10 point offer. The 10-point offers were often things that would challenge a 3rd or 4th year student. A few were insanely difficult.
The bonus points got tallied over the term and were used at the end to deal with border-line grades. For example, someone with almost but not quite a ‘B’ would get the B if they had bonus points, and the B- if they didn’t. Regardless of how many bonus points they accumulated, the most that it would ever affect their grade was by 1/2 a letter, so they could go from a B+ to an A-, but not to an ‘A’.
Most of the average students simply ignored the bonus challenges most of the time, but many would do some bonus on at least one of the assignments. The top students treated the bonus points like a game score and would sometimes do things just to increase their ‘score’. The all-time record holder in the years I was doing this came away with 156 points in one term. He did (of course) end up going on to do a PhD.
This approach seemed to meet the needs of both groups of students – the average ones had requirements they could achieve (possible though not easy) and the high-end students had plenty to keep them interested and engaged. And, as an added benefit for me, complaints about grades and grading almost disappeared.
8.
Great Recession caused by sub-standard public education? « Computing Education Blog | October 23, 2009 at 12:58 pm
[…] this theme, that we’re ignoring all-but-the-best students, in my recent blog posts, about how we choose exciting the top students instead of educating all, and about how the movement to on-line education tends to wipe out the bottom half of the student […]
9.
Boredom vs. Failure Part 2: The New Demographic « Computing Education Blog | January 7, 2010 at 11:06 am
[…] challenged, can be bored. This figure highlights the challenging trade-off that I mentioned in a previous post. Do we risk failing the low-knowledge students by catering to the high-knowledge students, or do […]
10.
Equality in school leads to system excellence: Boredom and failure, part 3 « Computing Education Blog | January 5, 2012 at 6:53 am
[…] in the US education system. The trade-off is one that I talked about in my posts on boredom and failure. The Finns don’t pull out the gifted students to keep them from getting bored, and they […]
11.
Big-D “Design” for education and online courses: Let’s build more and different « Computing Education Blog | May 14, 2012 at 8:35 am
[…] at the low to middle performing students? Maybe we need on-line courses or books that seek to bore and drive away the upper […]
12.
More evidence for Aptitude-Treatment Interactions | Computing Education Blog | August 6, 2013 at 1:37 am
[…] The same kind of educational opportunity does not work for all students. In particular, constructionism may not provide enough structure for low achieving students. (See previous discussion about boredom vs. failure.) […]
13.
Proposal #2 to Change CS Education to Reduce Inequity: Make the highest grades achievable by all students | Computing Education Research Blog | July 27, 2020 at 7:00 am
[…] I give out a lot of A’s. In years past, I did a series of blog posts on “Boredom vs Failure” (here’s the first post in the series, and here’s the last one). The question is: which is worse, to bore and give easy A’s to the […]