The 10K Hour Rule: Deliberate Practice leads to Expertise, and Teaching can trump Genetics

October 13, 2014 at 8:21 am 18 comments

A recent article in Slate (see here) suggests that practice may not lead to expertise, that the “10,000 hour rule” is wrong. The “10,000 hour rule” was popularized by Malcolm Gladwell in his book Outliers (see excerpt here), but really comes from an important paper by K. Anders Ericsson and colleagues, “The Role of Deliberate Practice in the Acquisition of Expert Performance.” Ericsson claimed that 10,000 hours of deliberate practice results in expert-level performance.

The Slate article is based mostly on a new meta-analysis (see here) by Macnamara, Hambrick (also a co-author on the Slate article), and Oswald which reviewed and combined studies on expertise. They found that practice always was positively correlated with better performance, but did not explain all of (or even most of) the difference in expertise between study participants. The Slate article authors suggest, then, that deliberate practice is not as important as genetics or innate talent.

Deliberate practice left more of the variation in skill unexplained than it explained…There is now compelling evidence that genes matter for success, too…What all of this evidence indicates is that we are not created equal where our abilities are concerned.

The paper and article make two big mistakes that leave the “10,000 hour rule” as valid and valuable. The first is that practice is not the same as deliberate practice, and the second is that the fallback position can’t be genetics/innate talent. In general, their argument hinges on practice hours all being of equal value, which shows a lack of appreciation for the role of teaching.

Practice is not the same as deliberate practice

Ericsson was pretty clear in his paper that all practice is not created equal. Deliberate practice is challenging, focused on the skills that most need to be developed, with rapid feedback. (Here’s a nice blog post explaining deliberate practice.) Simply putting in 10,000 hours of practice in an activity does not guarantee expertise. Ericsson and the Slate authors would be in agreement on this point.

I’m sure that we’ve all seen musicians or athletes (and if we’re honest, we’ve probably all been like those musicians or athletes) who sometimes just “phone it in” during practice, or even during a game. I used to coach my daughters’ soccer teams, and I can absolutely assure you that there were hours in games and rehearsals where some of my players really didn’t make any progress. They found ways of getting through practice or games without really trying.

In the Macnamara paper, whether practice was “deliberate” or not was determined by asking people. They collected practice logs, surveys, and interviews. The participants in the studies self-reported whether the practice was deliberate. Imagine someone telling the interviewer or writing in their log, “Yeah, well, about 5,000 of those 10,000 hours, I was really lazy and not trying very hard.”  It’s impossible to really distinguish practice from deliberate practice in this data set.

The bottom-line is that the Macnamara study did not test Ericsson’s question. They tested a weak form of the “10,000 hour rule” (that it’s just “practice,” not “deliberate practice”) and found it wanting. But their explanation, that it’s genetics, is not supported by their evidence.

Genetics/Innate starts at birth, no later

The Slate authors argue that, if practice doesn’t explain expertise, then it must be genetics. They cite two studies that show that identical twins seem to have similar music and drawing talent compared to fraternal twins. But that’s correlation and doesn’t prove causation — there may be any number of things on which the identical twins aren’t similar. (See this great Radiolab podcast exploring these kinds of miraculous misconceptions.)

If you’re going to make the genetics/innate argument, you have to start tracking participants at birth. Otherwise, there’s an awful lot that might add to expertise that’s not going to get counted in any practice logs.

I took classes on how to coach soccer. One of the lessons in those classes was, “It’s a poor coach who makes all practices into scrimmage.” Rather, we were taught to have students do particular drills to develop particular skills. (Sound like deliberate practice?) For example, if my players were having trouble dribbling, I might have them dribble a ball in a line around cones, across distances, through obstacles.

Can you imagine a child who one day might play in a soccer team with official practices — but before those practices and perhaps even before joining a team might dribble a ball around the neighborhood? Wouldn’t that be developing expertise? And yet, it wouldn’t be counted in player logs or practice hours. A kid who did lots of dribbling might come into a team and seem like a superstar with all kinds of innate talent. One might think that the kid had the “Soccer gene.”

To start counting hours-towards-expertise anything later than birth is discounting the impact of learning in the pre-school years on up. We know that pre-school years make a difference (see this website that Diana Franklin sent me, and the argument for pre-school in this recent Freakonomics podcast). A wide variety of activities can develop skills that can be influence expertise. If you don’t start tracking students from birth, then it’s hard to claim that you’ve counted in the practice log everything that’s relevant for expertise.

The claim that expertise is determined at birth is a common claim among CS educators. Most CS teachers to whom I’ve asked the question are convinced some people “can’t” learn to code, that it’s genetic or innate to learn programming. That’s where the myth of the “Geek Gene” came from (Raymond Lister has written several times on that). Couldn’t it be that there are dribbling-around-the-neighborhood activities that lead toward CS expertise? Consider the famous pre-programming activity of writing the instructions out for making a peanut-butter-and-jelly sandwich (like here). If we believe that that kind of practice helps to develop CS expertise, then other “writing instructions out” activities might lead towards CS expertise. Maybe people who seem to have genetic/innate ability in CS just did a lot of those kinds of activities before they got to our classes.

The clock on developing expertise doesn’t start when students walk through our door.

Bigger than P=NP: Is teaching > genetics?

In the end, it’s very difficult to prove or disprove that genetics accounts for expertise in cognitive skill. I don’t think Macnamara et al. settled the score. But my point about deliberate practice actually points to a much bigger issue.

Teachers Matter is the two word title of a 2012 OECD report (available here). There is a difference between great teachers and poor teachers, and the difference can be seen in terms of student performance. If you believe that (and there’s gobs of evidence that says you should), then it seems obvious that all practice is not created equal. Hours spent in practice with a good teacher are going to contribute more to expertise than hours spent without a teacher. Look back at that definition of “deliberate practice” — who’s going to pick the activities that most address your needs or provide the immediate feedback? The definition of deliberate practice almost assumes that there’s going to be teacher in the loop.

An open question is just how far we can get with excellent teaching. How much can we use teaching to get beyond genetic disparities? Is teaching more powerful than genetics? That’s an important question, and far more important than the classic CS question whether P=NP. I believe that there are limits. There are genetic problems that teaching alone can’t address. But we don’t know what those limits are.

We certainly have evidence that we can use teaching to get past some differences that have been chalked up to genetics or being innate. Consider the fact that men have better spatial skills than women. Is it innate, or is it learned? It’s not clear (see discussion on that here). But the important point is: it doesn’t matter. Terlecki, Newcombe, and Little have found that they can teach women to perform as well as men on visual skills and that the improvements in spatial ability both transfers and persists (see the journal article version here). The point is that spatial skills are malleable, they can be developed. Why should we think that other cognitive skills aren’t? The claims of the Slate authors and Macnamara et al ignore the power of a great teacher to go beyond simple rote practice to create deliberate opportunities to learn. The words teach, teacher, and teaching don’t appear in either article.

Here’s my argument summarized.  The Slate authors and Macnamara et al. dismiss the 10K hour rule too lightly, and their explanation of genetic/innate basis for expertise is too simple.  Practice is not the same as deliberate practice, or practice with a teacher. Expertise is learned, and we start learning at birth with expertise developing sometimes in ways not directly connected to the later activity. The important part is that we are able to learn to overcome some genetic/innate disparities with good teaching. We shouldn’t be giving up on developing expertise because we don’t have the genes. We should be thinking about how we can teach in order to develop expertise.

Entry filed under: Uncategorized. Tags: , , , .

What Computing Education Research does that Engineering Ed and Physics Ed Research doesn’t Teaching Computer Science Better to get Better Results

18 Comments Add your own

  • 1. shriramkrishnamurthi  |  October 13, 2014 at 8:46 am

    Thanks for taking on that Slate article; as an uninformed outsider it’s good to see a debate on this topic. As an uninformed outsider, I do have two questions. (1) You criticize the method of self-reporting; but that is what the original study on this topic itself did (in part): e.g., “The subjects were then asked to estimate how many hours per week they had practiced alone with the violin for each year since they had started to practice”. (2) You haven’t actually showed that genetics begins “at birth, no later”, simply that there are more factors than pure genetics at play [*]. But genetics could be a catalyst, one that enriches each of these bits of training and practice a tiny bit, and we know how that compound interest formula works.

    [*] This being the public Internet, I must litter this message with a disclaimer: I’m not taking any position on genetics, race, or any number of issues that could be imputed to me that I didn’t actually say. I’m simply using logic to pick apart statements, no more. Indeed, as an educator I have to hope that genetics is not sufficiently deterministic, otherwise what I do is somewhat pointless.

    Reply
    • 2. Michael S. Kirkpatrick  |  October 13, 2014 at 10:17 am

      “You haven’t actually showed that genetics begins ‘at birth, no later,’ simply that there are more factors than pure genetics at play.”

      That’s exactly the point. Is it genetics? Or is it parental influence (i.e., children imitating the things their parents are good at)? In the case of the twins with artistic ability, is it sibling rivalry? I’ve seen accounts of twins separated at birth that end up having similar characteristics, but I’ve never seen a controlled study that suggests ALL twins separated at birth are so closely matched. I don’t think that Mark was trying to suggest that genetics doesn’t play a role, just that there is no data to use as a basis for a claim that genetics plays the most important role.

      Reply
    • 3. Mark Guzdial  |  October 13, 2014 at 10:36 am

      I agree with both of your critiques, Shriram, but I believe that my argument against the Slate article is still valid. The Slate authors are saying (in shortest form), “Practice hours alone do not account for all the variance in expertise. Therefore, genetics or innate abilities explain the rest of the variance.” They’re arguing that practice+genetics accounts for the development of expertise (without any explicit evidence that genetics explains the rest). I’m aiming to dispute that “genetics” is the only explanatory factor that’s left. I point out:

      1. Not all practice is created equal. If you had counted deliberate practice separate from (let’s call it) easy practice, then you might have found that deliberate practice explains more of variance. It’s okay that Ericsson didn’t do that in his study. The value of deliberate practice over practice does follow from the theory, and does weaken the claim of genetics.
      2. Yes, prior-to-practice experiences might be influenced by genetics. But all I have to show is that it’s reasonable that there are differentiators between the participants other than genetics and prior to recording practice to dispute the genetics claim.

      Finally, Ericsson is just as guilty as the Slate authors in forgetting about teaching. The findings that teaching trumps genetics are dramatic and powerful. (I’ve got another great example of that in my Blog@CACM post going up tomorrow, also on this theme.) To me, that’s the most important point. Rather than arguing about how our students are lacking in genetics or innate abilities, we should be figuring out ways to teach them.

      Reply
      • 4. shriramkrishnamurthi  |  October 13, 2014 at 10:49 am

        Fair enough, Mark. That Slate article has lots of problems; I guess I didn’t take it that seriously to begin with (I mean, it’s Slate; the author is clearly trying to play the market of public opinion, and finding the widest audience of low review quality to make it)—other than the value in having a debate on this topic. On re-reading it now, I see now how you are directly responding to it. My criticism still makes sense when your article is taken as a self-contained entity, but not as much when it’s viewed as a response to that piece (which, to be fair, is how you labeled and intended it). And we can all agree, I think, that leaving the teaching side of the equation out is rather silly.

        Reply
  • 5. richde  |  October 13, 2014 at 9:10 am

    Reblogged this on Innovate.EDU and commented:
    Mark Guzdial comments on a recent Slate article reporting results of a metastudy that (1) claims to show that deliberate practice does not account for acquisition of expertise, and (2) that the only other source must be innate factors like genetics. It would be an important finding if true, but Guzdial points out some critical flaws:
    “The bottom-line is that the [Slate] study did not test [the 10,000 Rule] question. They tested a weak form of the “10,000 hour rule” (that it’s just “practice,” not “deliberate practice”) and found it wanting. But their explanation, that it’s genetics, is not supported by their evidence.”

    Reply
  • 6. Eric I.  |  October 13, 2014 at 10:56 am

    What I find interesting in this post is how the standards change depending on whether the author likes a conclusion or dislikes it.

    So the Ericsson article apparently shows that 10,000 hours of “deliberate practice” causes one to become an expert. But wait, the data is correlational, and yet causation is simply accepted.

    The Slate article claims to show that there’s a genetic component in expertise in some fields. No, that’s correlational and doesn’t imply causation according to the author.

    These twin studies are well-established and I don’t think the author can sweep them away so easily given he finds their implications undesirable.

    The basic idea is that we’re dealing with many sets of twins, some fraternal, some identical, all separated early in life to remove environment as a factor. If the correlation along some data is significantly higher among the identical twins than the fraternal twins, it makes a strong argument that there’s a genetic factor

    So let’s say that there’s a much higher correlation between identical twins of diagnosis of cancer of the X (whatever the X might be) than between the fraternal twins. That’s a strong argument that there’s a genetic component to susceptibility to cancer of the X. Yes, it’s correlational.

    If you thought the conclusion is wrong, you’d need an even stronger argument against it. And citing Radio Lab simply would not qualify. And you most certainly don’t need to log all action starting at birth to make a strong genetic argument.

    Reply
    • 7. Mark Guzdial  |  October 13, 2014 at 11:50 am

      I am convinced that there’s a genetic component in expertise in some fields. I don’t see convincing evidence that there’s a genetic component in cognitive skills expertise.

      Absolutely, you can use identical and fraternal twins to do “natural experiments” where you can go beyond correlations to causation. I don’t doubt that. But the Slate authors are suggesting that genetics explains the rest of variance in expertise the practice alone doesn’t explain. The authors don’t offer any evidence to support that. The twins studies aren’t used in the regression analysis to explain the rest of the variance in expertise.

      If you want to explain expertise in terms of practice+genetics, I’m arguing that you have to consider kind_of_practice (deliberate, teacher-guided, coasting) + expertise_developed_before_practice + genetics. If you just look at practice, you can’t claim that the rest is genetics without considering these other factors.

      Reply
      • 8. nickfalkner  |  October 14, 2014 at 8:54 pm

        There’s no real argument about genetic advantage in the physical space – to be an excellent Australian Rules Footballer, your handspan has to be big enough to grasp the ball well, for example, although this does not appear to guarantee recognised success on the field (http://www.ncbi.nlm.nih.gov/pubmed/16854624). Thus, deliberate practice, the development of fitness and skills on the field have to be built on top of even this existing advantage because AFL is more than just throwing and catching a ball.

        I totally agree that there is no convincing evidence regarding cognitive skills. In the absence of enough evidence to choose between two hypotheses, I have a natural tendency to pick the one that is more generous to the most people, which would reject the genetic argument as being exclusive for no demonstrated reason.

        My apologies if this is a little confusing, I’m a bit tired.

        Reply
  • […] Guzdial, in The 10K Hour Rule: Deliberate Practice leads to Expertise, and Teaching can trump Genetics | Computi…, responds to a Slate article claiming that genetics is more important than […]

    Reply
  • 11. Tomas  |  October 13, 2014 at 5:00 pm

    You are absolutely right to criticize the Hambrick and Macnamara study as failing to show what the authors claim – that deliberate practice isn’t important. The Hambrick and Macnamara study claims that deliberate practice only explains 4% of outcomes in education, which seems suspiciously low. Taking a close look at one of the papers in the meta-analysis shows why their analysis is so flawed.

    One relevant underlying papers included in the meta-analysis is Chinn, Sheard, Carbone, & Laasko “Study Habits of CS 1 Students: What do they do outside the classroom?”, which looked at study habits of students in an introductory computer programming course. The authors of the meta-analysis find that “practice” was unimportant for success on this course. Let’s take a look at what that means.

    The activities that the researchers (Hambrick and MacNamara) count as “practice” are activities such as:
    – Looking up material on the internet
    – Discussing the course on the online discussion forum
    – Working through tutorial problems
    These “practice” activities were found to be unimportant for success on the course. What were the “non-practice” activities? These included:
    – Turning up at lectures and programming workshops which were run for the course
    – Previous programming experience before the course
    These two activities were found by the original researchers to be the most important factors for success. This should be no great surprise. It’s not particularly radical to find that the people who do best on a computer programming course are those who were most experienced to start with (so had a head start), and turn up to the lectures and workshops.

    But Hambrick and MacNamara don’t count turning up at lectures, or having prior experience, as “practice”. But they do count inefficient study activities like looking up information on the internet as practice, even though presumably information on the internet is much less likely to be directly relevant to the course. Given that, it’s hardly surprising that they find that “practice” is unimportant. It’s because their definition of “practice” is flawed.

    Reply
  • 12. lizaloop  |  October 13, 2014 at 10:55 pm

    Bravo, Mark. Nice article and cogent replies. You advocate juggling 4 variables:
    1. Previous experience (including parental influence)
    2. Deliberate practice
    3. High quality teaching (including parental influence)
    4. Genetically determined talent (provided by parents)
    At first blush it would seem that who your parents are might be pretty important. However, my own studies suggest a different explanation when it comes to computer science.

    I’ve watched thousands of little kids (age 2 to 12) fool around with computer-controlled toys, computers running games and applications, and kids learning beginning programming. Actually this observation applies to adults as well. I can separate these people into three categories: Those who ‘get’ computing easily; those who get it somewhat with teaching; those who just don’t get it. The difference wasn’t ‘practice’ either deliberate or easy. They were all beginners (this was in the ’70s and ’80s when computers were rather scarce). So what’s the difference?

    My hypothesis is that some people spontaneously build mental models of the black boxes they are interacting with (group 1), others don’t. Of the ones that don’t, some can be taught to build models (group 2), others can’t be taught or won’t learn (group 3). It doesn’t matter whether the black box is a computer, a clothes washing machine or a baseball game. Having a mental model helps you figure out what the system will do next as well as what to do to the system to get a desired result. And what is a computer program? It’s a formal model.

    My youngest anecdotal study subject was 8 months old. The kid who got me thinking along these lines was 2 and came from a family of bright folks that did not engage much in systems thinking. Although I suspect there is a genetic component at work here, I chose not to engage that issue but rather to see if I could tease out any measurable variables. I didn’t get that far but I’m still working on it 35 years later.

    That 2-year-old, Scotty, pushed every key and button on the electronic piano I put in front of him – systematically. And he went back and retested the effect of each key. His actions were deliberate, not random. Others under the same circumstances just banged on the keyboard and never even looked at the button. What was he doing if not building a mental model? I think I saw the same behavior in the kids who were successful at programming: explore, hypothesize, test, evaluate the outcome. Many really bright kids come to school already knowing how to apply this process. Others can get it if they are taught. But if you can’t remember what you saw or can’t hold the observations in your mind in the same order that they happened, well perhaps that can’t be taught. In the beginning it’s not a matter of age or practice. It’s rudimentary mental model building.

    So now we have young kids, one group of whom has serendipitously discovered how to construct and use mental models to manipulate their worlds. Two groups who have not. Those in group 1 practice this skill a great deal. Groups 2 and 3 don’t have it to practice. By high school we’re seeing huge differences in school success,vocational activities, sometimes both.

    So how does all this relate to the four variables we started with? If the expertise of interest is model building – a skill which may underlie success in many other activities including coding and carpentry – the kids who started doing it as toddlers (group 1) have a huge leg up. Their ‘previous experience’ is extensive and largely unaccounted for in research designs. Much of it is preverbal. They probably already have 10,000 hours of ‘deliberate’, even if not conscious, ‘practice’ in model building. Talk to folks who do this. They can’t imagine not thinking in models. For these folks,’high quality teachers’ are part of their learning tool kits. If home, community or school don’t place the teachers they need in their paths they will seek them out. ‘Genetic talent’? Well maybe. Excellent memory and pattern recognition certainly don’t hurt. But I suspect there is a lot of good luck that happens to produce early success with building cognitive models.

    The students in group 2 have a lot of catching up to do. Many of them will have some previous experience learned via math word problems, graphing sentence structure, choreographing a dance or making plans to build a birdhouse. But because they haven’t automated the model building process they probably haven’t applied it throughout their lives and they will have to practice consciously to make it habitual. High quality teaching is especially important with this group to ensure that the learner has so much success using models that he or she becomes intrinsically motivated to practice it, thus gaining experience. Also, the teacher may have to provide opportunities for the learner to discover the power of transferring the model building process among disparate domains. It would be easy to attribute the failure to build models spontaneously to lower IQ or ‘lack of talent’ but I saw no evidence for choosing that over limited experience and/or practice.

    The third group presents a real problem and puzzle for educators. Although these kids may not be failing in school, I suspect their success comes from the bottom two skills of Bloom’s taxonomy, knowledge and understanding. The next level up, application, requires being able to recognize one situation as a model of another. It will be interesting to explore the capabilities of this group within the framework I have described but I haven’t spent much effort on it.

    In conclusion, I predict that none of the parameters mentioned in Mark’s blog — previous experience, deliberate practice, high quality teaching, or genetically determined talent — will provide us with a key to the development of expertise although each may be a necessary component. Overshadowing all of these we will find the propensity to build models and skill at constructing these mental representations of the domain of expertise and/or models of specific problems or activities within a given domain. The model may be as ephemeral as an echo of the melody the pianist is attempting to play but if it isn’t heard in the mind it won’t be played with expertise.

    By now the expert scholars among you may be completely frustrated with the speculative nature of this piece. Me too. I enrolled in Stanford School of Education in 1984 with the objective of using this theme as my PhD thesis. I never made it through. I’m still searching for a theoretical framework within which to ground this work. If you know of one or are interested in expanding on this theme, please contact me.

    Reply
    • 13. Mark Guzdial  |  October 14, 2014 at 6:57 pm

      Thanks for the thoughts, Liza. One additional variable (and I’m sure that there are many more) that I didn’t mention, that I do think relates to your point, is whether students believe that the computer is rule-based, i.e., there is a know-able model. Dehnadi and Bornat found in their pre-test to predict success in CS that the most successful students were those who applied their conception of how the computer worked (whether or not it was right) consistently. Someone who would apply a misconception consistently does, at least, think that there are rules that explain computational behavior. An often-cited SIGCSE 2001 by Cantwell Wilson and Shrock looked at the factors that led to success in CS1, and one of the most strongly correlated negative factors was attributing success to luck. If you think it’s all luck, not replicable, you’re unlikely to look for a model.

      Reply
  • […] the first post (see here), I argued that the practice+genetics is too simple to explain expertise. First, practice can be […]

    Reply
  • 15. Three Posts from Mark Guzdial | Teaching Software Carpentry  |  October 16, 2014 at 7:37 am

    […] used to require participants in this course to follow Mark Guzdial’s blog. These three articles (which form a series) have got me thinking I should do so again – or at least […]

    Reply
  • […] is a post about CS education. It is prompted by a seriesofposts by Mark Guzdial in which he criticizes the pervasive belief among CS educators that when it […]

    Reply
  • […] argued before that there is no reason to believe in the Geek Gene (see post here), and every reason to believe that good teaching can overcome “innate” differences (see […]

    Reply
  • […] I talked to Helenrose about what her work means for University CS teachers.  In general, the work she describes is about school teachers, not faculty.  She agreed that it’s possible for University CS teachers to have high self-efficacy even if they are not successful teachers, because University teachers define self-efficacy differently than school teachers.  School teachers are responsible for student learning.  They know individual students.  They actually know if they are successful in their teaching or not (in terms of student learning and engagement).  University teachers tend to have larger classes, and they tend to teach via lecture.  They usually have little knowledge of individual student learning and engagement.  Their sense of self-efficacy may arise from their ability to succeed at their task, “I can give great lectures. (Almost nobody falls asleep.)  I can manage huge classes.”  Where they do have knowledge of learning and evidence of ineffective teaching, they may simply decide that it’s the student’s fault. Perhaps this is where the Geek Gene is born. […]

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Recent Posts

October 2014
M T W T F S S
« Sep   Nov »
 12345
6789101112
13141516171819
20212223242526
2728293031  

Feeds

Blog Stats

  • 1,294,189 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 4,598 other followers

CS Teaching Tips


%d bloggers like this: