Posts tagged ‘teachers’

Teaching Computer Science Better to get Better Results

This is my third blog post in a series inspired by a thread in the SIGCSE-Members list and by the Slate article which argued that “Practice doesn’t make perfect.” Macnamara et al did a meta-analysis of studies of expertise, and found that a relatively small percentage of variance in expertise can be explained through hours of practice. The Slate authors argue that this implies that genetics explains the rest of the variance.

  • In the first post (see here), I argued that the practice+genetics is too simple to explain expertise. First, practice can be deliberate, lazy, or teacher-led. Second, there is experience that leads to expertise which is between genetics and practice. The most significant flaw of both Macnamara et al. and Ericsson et al. is ignoring teaching.
  • In the second post (appearing yesterday in Blog@CACM), I addressed a claim in the SIGCSE-Members list that programmers are “wired” differently than others. Most CS teachers agree with the Slate authors, that students can NOT be more successful with more work. The evidence that better teaching leads to better learning is overwhelming. In fact, there is significant evidence that teaching can even overcome genetic/innate-ability differences.

Lots of CS teachers believe in the Geek Gene Hypothesis, and for good reason. It’s frustrating to have seemingly no impact on some, especially the lower-end, students. Even the award-winning Porter, Zingaro, and Lister paper points out that the earliest assessments in the class they studied correlate very highly with the final grade. Gas Station without Pumps voiced a similar sentiment in his blog post in response to the Slate article:

But the outcomes for individual students seem to depend more on the students coming in than on what I do.  Those students who come in better prepared or “innately” smarter progress faster than those who come in behind, so the end result of the teaching is that differences among the students are amplified, not reduced. Whether the differences in the students coming in are due to prior practice, prior teaching, or genetics is not really knowable, but also not really relevant.

I agree. It’s not really knowable where the difference comes from and it’s not really relevant. The point of my Blog@CACM post is: we can do better. If we can teach spatial ability and subitizing, two skills that have a much stronger claim to being innate than programming, then we can certainly teach people to program better.

If we follow common practice and it’s unsuccessful, it’s not surprising that we think, “I tried. I explained carefully. I gave interesting assignments. I gave good feedback. It’s got to be an innate trait. Some students are just born wired to program.

I watch my children taking CS classes, along with English, Chemistry, Physics, and Biology classes. In the CS classes, they code. In the other classes, they do on-line interactive exercises, they write papers, they use simulations, they solve problems by-hand. Back in CS, the only activity is coding with feedback. If we only have one technique for teaching, we shouldn’t be surprised if it doesn’t always work

Here’s a reasonable hypothesis: We get poor results because we use ineffective teaching methods. If we want to teach CS more effectively, we need to learn and develop better methods. If we don’t strive for better methods, we’re not going to get better results.

A first step is to be more methodical with how we choose methods. In a 2011 paper by Davide Fossati and me (see here), we found that CS teachers generally don’t use empirical evidence when making changes in how we teach. We act from our intuition, but our students aren’t like us, and our intuition is not a good indicator of what our students need.

Next, we need to experiment with more methods. We want to get to a place where we identify known problems in our students’ understanding, and then used well-supported methods that help students develop more robust understandings. We probably don’t have a wide range of different techniques for teaching assignment, iteration, recursion, and similar concepts? We should try well-supported techniques like pair programming, peer instruction, or Media Computation (see CACM article on these). We should try to expand our techniques repertoire beyond simply grinding at code. We could try techniques like worked examples, Problets, CodingBat, games with learning outcomes like Wu’s Castle, multiple choice questions like in Gidget, the Parson’s Problems in the Runestone Interactive ebooks, or even computing without computers as in CS Unplugged.

We do not make it easy for CS teachers to pick up new, better, more proven methods. Sure, there are the SIGCSE Symposium proceedings, but that’s not a systematic presentation of what to use when. This is on the CS education research community to do better. But it’s also on the CS teaching community to demand better, to seek out better methods and studies of techniques.

If we taught better, there are a lot of problems in CS that we might impact. We might bring in a more diverse group of students. We might make our current students more successful. We might change attitudes about computing. Perhaps most importantly, maybe we as teachers will come to believe that we can teach anyone to program.

October 15, 2014 at 8:32 am 27 comments

The 10K Hour Rule: Deliberate Practice leads to Expertise, and Teaching can trump Genetics

A recent article in Slate (see here) suggests that practice may not lead to expertise, that the “10,000 hour rule” is wrong. The “10,000 hour rule” was popularized by Malcolm Gladwell in his book Outliers (see excerpt here), but really comes from an important paper by K. Anders Ericsson and colleagues, “The Role of Deliberate Practice in the Acquisition of Expert Performance.” Ericsson claimed that 10,000 hours of deliberate practice results in expert-level performance.

The Slate article is based mostly on a new meta-analysis (see here) by Macnamara, Hambrick (also a co-author on the Slate article), and Oswald which reviewed and combined studies on expertise. They found that practice always was positively correlated with better performance, but did not explain all of (or even most of) the difference in expertise between study participants. The Slate article authors suggest, then, that deliberate practice is not as important as genetics or innate talent.

Deliberate practice left more of the variation in skill unexplained than it explained…There is now compelling evidence that genes matter for success, too…What all of this evidence indicates is that we are not created equal where our abilities are concerned.

The paper and article make two big mistakes that leave the “10,000 hour rule” as valid and valuable. The first is that practice is not the same as deliberate practice, and the second is that the fallback position can’t be genetics/innate talent. In general, their argument hinges on practice hours all being of equal value, which shows a lack of appreciation for the role of teaching.

Practice is not the same as deliberate practice

Ericsson was pretty clear in his paper that all practice is not created equal. Deliberate practice is challenging, focused on the skills that most need to be developed, with rapid feedback. (Here’s a nice blog post explaining deliberate practice.) Simply putting in 10,000 hours of practice in an activity does not guarantee expertise. Ericsson and the Slate authors would be in agreement on this point.

I’m sure that we’ve all seen musicians or athletes (and if we’re honest, we’ve probably all been like those musicians or athletes) who sometimes just “phone it in” during practice, or even during a game. I used to coach my daughters’ soccer teams, and I can absolutely assure you that there were hours in games and rehearsals where some of my players really didn’t make any progress. They found ways of getting through practice or games without really trying.

In the Macnamara paper, whether practice was “deliberate” or not was determined by asking people. They collected practice logs, surveys, and interviews. The participants in the studies self-reported whether the practice was deliberate. Imagine someone telling the interviewer or writing in their log, “Yeah, well, about 5,000 of those 10,000 hours, I was really lazy and not trying very hard.”  It’s impossible to really distinguish practice from deliberate practice in this data set.

The bottom-line is that the Macnamara study did not test Ericsson’s question. They tested a weak form of the “10,000 hour rule” (that it’s just “practice,” not “deliberate practice”) and found it wanting. But their explanation, that it’s genetics, is not supported by their evidence.

Genetics/Innate starts at birth, no later

The Slate authors argue that, if practice doesn’t explain expertise, then it must be genetics. They cite two studies that show that identical twins seem to have similar music and drawing talent compared to fraternal twins. But that’s correlation and doesn’t prove causation — there may be any number of things on which the identical twins aren’t similar. (See this great Radiolab podcast exploring these kinds of miraculous misconceptions.)

If you’re going to make the genetics/innate argument, you have to start tracking participants at birth. Otherwise, there’s an awful lot that might add to expertise that’s not going to get counted in any practice logs.

I took classes on how to coach soccer. One of the lessons in those classes was, “It’s a poor coach who makes all practices into scrimmage.” Rather, we were taught to have students do particular drills to develop particular skills. (Sound like deliberate practice?) For example, if my players were having trouble dribbling, I might have them dribble a ball in a line around cones, across distances, through obstacles.

Can you imagine a child who one day might play in a soccer team with official practices — but before those practices and perhaps even before joining a team might dribble a ball around the neighborhood? Wouldn’t that be developing expertise? And yet, it wouldn’t be counted in player logs or practice hours. A kid who did lots of dribbling might come into a team and seem like a superstar with all kinds of innate talent. One might think that the kid had the “Soccer gene.”

To start counting hours-towards-expertise anything later than birth is discounting the impact of learning in the pre-school years on up. We know that pre-school years make a difference (see this website that Diana Franklin sent me, and the argument for pre-school in this recent Freakonomics podcast). A wide variety of activities can develop skills that can be influence expertise. If you don’t start tracking students from birth, then it’s hard to claim that you’ve counted in the practice log everything that’s relevant for expertise.

The claim that expertise is determined at birth is a common claim among CS educators. Most CS teachers to whom I’ve asked the question are convinced some people “can’t” learn to code, that it’s genetic or innate to learn programming. That’s where the myth of the “Geek Gene” came from (Raymond Lister has written several times on that). Couldn’t it be that there are dribbling-around-the-neighborhood activities that lead toward CS expertise? Consider the famous pre-programming activity of writing the instructions out for making a peanut-butter-and-jelly sandwich (like here). If we believe that that kind of practice helps to develop CS expertise, then other “writing instructions out” activities might lead towards CS expertise. Maybe people who seem to have genetic/innate ability in CS just did a lot of those kinds of activities before they got to our classes.

The clock on developing expertise doesn’t start when students walk through our door.

Bigger than P=NP: Is teaching > genetics?

In the end, it’s very difficult to prove or disprove that genetics accounts for expertise in cognitive skill. I don’t think Macnamara et al. settled the score. But my point about deliberate practice actually points to a much bigger issue.

Teachers Matter is the two word title of a 2012 OECD report (available here). There is a difference between great teachers and poor teachers, and the difference can be seen in terms of student performance. If you believe that (and there’s gobs of evidence that says you should), then it seems obvious that all practice is not created equal. Hours spent in practice with a good teacher are going to contribute more to expertise than hours spent without a teacher. Look back at that definition of “deliberate practice” — who’s going to pick the activities that most address your needs or provide the immediate feedback? The definition of deliberate practice almost assumes that there’s going to be teacher in the loop.

An open question is just how far we can get with excellent teaching. How much can we use teaching to get beyond genetic disparities? Is teaching more powerful than genetics? That’s an important question, and far more important than the classic CS question whether P=NP. I believe that there are limits. There are genetic problems that teaching alone can’t address. But we don’t know what those limits are.

We certainly have evidence that we can use teaching to get past some differences that have been chalked up to genetics or being innate. Consider the fact that men have better spatial skills than women. Is it innate, or is it learned? It’s not clear (see discussion on that here). But the important point is: it doesn’t matter. Terlecki, Newcombe, and Little have found that they can teach women to perform as well as men on visual skills and that the improvements in spatial ability both transfers and persists (see the journal article version here). The point is that spatial skills are malleable, they can be developed. Why should we think that other cognitive skills aren’t? The claims of the Slate authors and Macnamara et al ignore the power of a great teacher to go beyond simple rote practice to create deliberate opportunities to learn. The words teach, teacher, and teaching don’t appear in either article.

Here’s my argument summarized.  The Slate authors and Macnamara et al. dismiss the 10K hour rule too lightly, and their explanation of genetic/innate basis for expertise is too simple.  Practice is not the same as deliberate practice, or practice with a teacher. Expertise is learned, and we start learning at birth with expertise developing sometimes in ways not directly connected to the later activity. The important part is that we are able to learn to overcome some genetic/innate disparities with good teaching. We shouldn’t be giving up on developing expertise because we don’t have the genes. We should be thinking about how we can teach in order to develop expertise.

October 13, 2014 at 8:21 am 16 comments

Study finds increased STEM enrollment: Taking from education and business

First the good news: STEM enrollment is up.  Then the surprising news: Humanities are not losing students to STEM.  Rather, it’s the professional fields like education that are losing enrollment.  That makes CS Ed (and other STEM discipline-based education research (DBER) fields) the odd winner-losers.  Yay, there are more students, but there will be fewer STEM teachers in the future to teach them.

Policy makers regularly talk about the need to encourage more undergraduates to pursue science and technology fields. New data suggest that undergraduates at four-year institutions in fact have become much more likely to study those fields, especially engineering and biology.

And while much of the public discussion of STEM enrollments has suggested a STEM vs. liberal arts dichotomy (even though some STEM fields are in fact liberal arts disciplines), the new study suggests that this is not the dynamic truly at play. Rather, STEM enrollments are growing while professional field enrollments (especially business and education) are shrinking.

via Study finds increased STEM enrollment since the recession | Inside Higher Ed.

The ComputerWorldK agrees. They claim that the smart students were going into business, then Wall Street collapsed, and now they’re going into CS and that’s why we’re having sky-rocketing enrollments.

The number of computer science graduates will continue to increase. Computer science enrollments rose by nearly 30% in the 2011-12 academic year, and they increased 23% the year before that.

The trend of enrollment increases since 2010 bodes well for a “future increase in undergraduate computing production,” according to the report.

The recession that hit in 2008 sent IT unemployment soaring, but it may have done more damage to the finance sector, especially in terms of reputation. That prompted some educators at the time to predict that the recession might send math-inclined students from the world of hedge funds to computer science.

via Wall Street’s collapse was computer science’s gain – Computerworld.

August 10, 2014 at 9:34 am 3 comments

New ExploringCS Working Paper: How do we avoid CS10K going to CS5K only five years later?

An important new working paper from the ExploringCS group asks the question: If we achieve CS10K, how do we avoid only having CS5K left after only five years?  This is exactly the question that Lijun Ni was exploring in her dissertation on CS teacher identity.

Of the 81 teachers who have participated in the ECS program over the last
five years, 40 are currently teaching ECS in LAUSD. These numbers reveal that we
have “lost” more teachers than we have “retained.” Of the 40 teachers who are
currently teaching the ECS course, 5 of them had a 1-2 year interval in which they
did not teach the course. This means that fully 45 of the 81 teachers who have
participated in the ECS program have experienced a teaching “disruption” which has
ended their participation in the ECS teacher community for a year or longer.

In particular, they ask us to consider the dangers of short-term fixes to long-term problems, which is a point I was trying to make when arguing that we may be 100 years behind other STEM subjects in terms of making our discipline-based education available to all.

In response to scaling up challenges, we can expect a rise of “quick-fix”
solutions that have a potential to undercut progress. One quick-fix “solution” to
address CS teacher shortage or the need for deepened teacher content knowledge
are programs that bring industry professionals to assist teachers in CS classrooms.
While we are interested in learning more about the outcomes of these programs,
because there can be value in students hearing from experts in the field, there are
also risks to having industry professionals take on a teaching role in the classroom
without professional development in effective and relevant pedagogy and belief
systems and equitable practices. Will industry professionals deliver content
knowledge the way they were taught, not having had experience working with the
novice learner? Will they focus on working with the students who think more like
they do, to the neglect of the other students? In short quick fixes like these may
inadvertently perpetuate the persistent divides in the field.

I add to their list of questions: Does bringing in IT professionals reduce the administrative pressure that pushes teachers out of CS?  Does it help to create the context and environment that supports CS teachers?

I used this working paper in my post this month for Blog@CACM.  Vint Cerf recently gave testimony in the Senate recommending a requirement for CS in all primary and secondary schools.  The ECS experience (and Lijun Ni’s work) point toward the need to create a supportive environment for CS teaching if we want to achieve Vint’s recommendation.

Highly recommended read.

July 27, 2014 at 9:35 am 1 comment

We need computing in schools, in whatever category will work

At the NCWIT Summit this year, I heard an interesting concern.  If CS counts as a mathematics or science course towards high school graduation requirements, will that make CS even less diverse?  Should we keep CS as a business topic (elective) where the women and under-represented minorities are?

I took up that question for my Blog@CACM post for this month: Why Counting CS as Science or Math is Not Considered Harmful. I argue that our goal is universal computational literacy, with everyone using computing in every class and everyone taking CS.  I don’t really care how it gets a foothold in schools.  It was fun to write about Alan Kay, Adele Goldberg, and Andy diSessa, pointing out that they were talking about these ideas long time before computational thinking.

 

June 24, 2014 at 8:48 am 8 comments

Teaching programming could be made easier

Gas station without pump’s post on Garth’s complaint “Teaching programming is not getting easier” intrigued me.  Garth does a good job of pulling together a lot of the themes of what makes teaching CS hard today.  I think that we can improve the situation.  I’m particularly interested in learning how to scaffold the development of programming knowledge, and we have to find ways to create professional communities of CS teachers.  There are techniques to share (worked examples, peer instruction, pair programming, Parson’s problems, audio tours), and we’re clearly not doing a good job of it yet.

In programming there are 4 homework problems over the period of a week, none of which are “easy”, and all require some problem solving and thinking.  There is somewhat of an incremental progression to the problems but that step from written problem to code is always a big one.  It is somewhat similar to solving word problems in math, every student’s favorite task.  For programming there are no colleagues available that have as much or more experience to pull teaching ideas from, if there are any other programming teachers at all.  There are no pedagogical resources anywhere online for teaching strategies.  After watching a number (3) of programming teachers teach it seems the teaching strategy is pretty consistent; show and tell and hope.

via Teaching programming is not getting easier. | Garth’s CS Education Blog.

June 17, 2014 at 8:55 am 12 comments

Older Posts


Recent Posts

October 2014
M T W T F S S
« Sep    
 12345
6789101112
13141516171819
20212223242526
2728293031  

Feeds

Blog Stats

  • 968,954 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 3,149 other followers

CS Teaching Tips


Follow

Get every new post delivered to your Inbox.

Join 3,149 other followers