Posts tagged ‘educational psychology’
I’m currently reading Nobel laureate Daniel Kahneman’s book, “Thinking Fast, Thinking Slow” (see here for the NYTimes book review). It’s certainly one of the best books I’ve ever read on behavioral economics, and maybe just the best book I’ve ever read about psychology in general.
One of the central ideas of the book is our tendency to believe “WYSIATI”—What You See Is All There Is. Kahneman’s research suggests that we have two mental systems: System 1 does immediate, intuitive responses to the world around us. System 2 does thoughtful, analytical responses. System 1 aims to generate confidence. It constructs a story about the world given what information that exists. And that confidence leads us astray. It keeps System 2 from asking, “What am I missing?” As Kahneman says in the interview linked below, “Well, the main point that I make is that confidence is a feeling, it is not a judgment.”
It’s easy to believe that University CS education in the United States is in terrific shape. Our students get jobs — multiple job offers each. Our graduates and their employers seem to be happy. What’s so wrong with what’s going on? I see computation as a literacy. I wonder, “Why is our illiteracy rate so high? Why do so few people learn about computing? Why do so many flunk out, drop out, or find it so traumatic that they never want to have anything to do with computing again? Why are the computing literate primarily white or Asian, male, and financially well-off compared to most?”
Many teachers (like the comment thread after this post) argue for the state of computing education based on what they see in their classes. We introduce tools or practices and determine whether they “work” or are “easy” based on little evidence, often just discussion with the top students (as Davide Fossati and I found). If we’re going to make computing education work for everyone, we have to ask, “What aren’t we seeing?” We’re going to feel confident about what we do see — that’s what System 1 does for us. How do we see the people who aren’t succeeding with our methods? How do we see the students who won’t even walk in the door because of how or what we teach? That’s why it’s important to use empirical evidence when making educational choices. What we see is not all there is.
But, System 1 can sometimes lead us astray when it’s unchecked by System 2. For example, you write about a concept called “WYSIATI”—What You See Is All There Is. What does that mean, and how does it relate to System 1 and System 2?
System 1 is a storyteller. It tells the best stories that it can from the information available, even when the information is sparse or unreliable. And that makes stories that are based on very different qualities of evidence equally compelling. Our measure of how “good” a story is—how confident we are in its accuracy—is not an evaluation of the reliability of the evidence and its quality, it’s a measure of the coherence of the story.
People are designed to tell the best story possible. So WYSIATI means that we use the information we have as if it is the only information. We don’t spend much time saying, “Well, there is much we don’t know.” We make do with what we do know. And that concept is very central to the functioning of our mind.
I wrote that blog post because we really have had a long debate in our faculty email list about many of those topics. I recently saw our Dean at an event, and he told me that he hadn’t read the thread yet (but he planned to) because “it must be 100 messages long.” Most of the references in that blog post came from messages that I wrote in response to that thread. It was a long post because people generally didn’t agree with me. Several senior, well-established (much more famous than me) faculty strongly disagreed with the evidence-based argument I was making. The thread finally ended when one of the most senior, most respected faculty in the College wrote a note saying (paraphrased), “There are probably better teaching evaluation methods than the ones we now use. I’m sure that Mark knows teaching methods that would help the rest of us teach better.” And that was it. Thread ended. The research-based evidence that I offered was worth fighting about. The word of authority was not.
I’ll bet that faculty across disciplines similarly respond to authority more than evidence. We certainly see the role of authority in Physics Education Research (PER). Pioneering PER researchers were not given much respect and many were ostracized from their departments. Until Eric Mazur at Harvard had his students fail the Force Concept Inventory (FCI), and he changed how he taught because of it. Until Nobel laureate Carl Wieman decided to back PER (all the way to the Office of Science Technology and Policy in the White House). Today, the vast majority of physics teachers know research-based teaching methods (even if they don’t always use them). FCI existed before Mazur started using it, but it really started getting used after Mazur’s support. The evidence of FCI didn’t change physics teaching. The voice of authority did.
While we might wish that CS faculty would respond more to evidence than authority (see previous post on this theme), this insight suggests a path forward. If we want CS faculty to improve their teaching and adopt evidence-based practices, top-down encouragement can have large impact. Well-known faculty at top institutions publicly adopting these practices, and Deans and Chairs promoting these practices can help to convince faculty to change.
I’ve written a couple times now about the workshop I attended at the University of Oldenburg the first week of June. (See the post where I talked about my two weeks in Germany.) For Blog@CACM, I wrote a post about teaching as collective practice and the workshop I took with Barbara Hofer (see post here). I wrote about learning about teacher beliefs and self-efficacy from Helenrose Fives here (see post).
Before we left for the workshop, I got to spend time with Ira Diethelm at the University of Oldenburg and one of her students. Ira is one of at least 16 (that Ira could count) CS Education professors in Germany. Ira works with pre-service teachers, in-service teachers, and graduate students. Her graduate students build outreach efforts and curricula as part of their research, then roll them out and provide resources to teachers. It’s remarkable what Ira is doing, and I understand that the other German CS Ed professors do similar things. I came away with a new insight: If we want to bootstrap and sustain CS Education in the United States, we should fund several endowed chairs of CS Education at top Schools of Education. Eventually, we have to have pre-service computing education programs if we want to make CS education sustainable (see that post here). Creating these endowed chairs gives us the opportunity to create positions like Ira’s in the United States.
Overall, the workshop was a terrific experience. The PhD student work was fascinating, and I enjoyed discussing their research with them. It was great to hear about German research perspectives that I hadn’t previously, like the Model of Educational Reconstruction that informs science education (see paper here). Barbara and Helenrose were only two of a several outstanding international education researchers who attended. As I mentioned to Pat Alexander (who has a length Wikipedia page of her accomplishments), I enjoyed being able to wallow in educational psychology for a week, because I so rarely get to do that. I gave a talk on three of our projects related to the theme of developing teachers: on Lijun Ni’s work on teacher identity, on the Disciplinary Commons for Computing Education, and on our ebook for preparing CS teachers. (See Slideshare here.)
The response to my talk was fascinating. Some of the German mathematics education researchers are deeply opposed to computing education in schools. (I suspect that more than one of them completely skipped my talk because they are so opposed.) “Computing education keeps stealing from mathematics teachers, and learning mathematics is more important.” At my talk, Pat Alexander asked me the same question that Peter Elias asked Alan Perlis in 1961, “Won’t the computer eventually just understand us? Doesn’t the computer just become invisible and not need to be programmed?” I told the story about Alan Perlis’s talk and about Michael Mateas’s argument, “There will always be friction.” From the computing educators, I heard a lot of anger. The German computing education researchers feel that other fields squeeze CS out because the they are not willing to allow computing education to take up any time or budget in the curriculum.
Probably the most interesting pushback was against computational thinking. The educational psychologists thought it was unbelievable that learning computing would in any way impact the way that people think or problem-solve in everyday life. “Didn’t we believe that once about Latin? and Geometry?” asked Gavin Brown. The psychologists at the workshop I attended saw a clear argument that we need to introduce computing in high school so that students can see if it’s for them, but not to teach general problem-solving skills. If we really want algorithmic thinking, they can design easier ways to achieve that goal than teaching programming.
We can probably help students to learn about computing in such a way that it might influence problem-solving on the computer. That’s part of Jeanette Wing’s model of Computational Thinking (see her 2010 paper). It’s the “Computational Thinking in Daily Life” part that the psychologists weren’t buying. That learning about computation helps with computational X is quite reasonable. If you understand what IP addresses are, we can help you to understand DNS problems and to realize that it’s not really that big of a deal if Wikipedia stores your IP address (see story about Erika Poole’s research). There is evidence that learning one programming language will likely transfer to another one (see Michal Armoni’s paper on transfer from Scratch to a text-based language). Learning to program is unlikely to influence any problem-solving in everyday life.
Source of the “Geek Gene”? Teacher beliefs: Reading on Lijun Ni, Learning from Helenrose Fives on teacher self-efficacy
I discovered the below quoted post when I was looking up a paper by my former student, Lijun Ni. It’s nice to see her work getting recognized and reviewed! I talked a lot about her work when I was talking to PhD students at the University of Oldenburg program — Lijun has studied the beliefs of CS teachers, and that’s super important.
One of the other international guests at the Oldenburg program I attended last month (see post here) was Helenrose Fives who has literally written the book on teacher beliefs (see Amazon reference). Several of the PhD students who presented their research talked about student teachers having lower self-efficacy after actually being in the classroom, less commitment to ideals like inquiry learning, and less belief that students can learn. Helenrose said that that’s really quite common. Teachers have a high level of self-efficacy (“I can teach using novel approaches that will really help students learn!”) before they enter the classroom, and that sense of self-efficacy falls off a cliff once they face the reality of the classroom. The self-efficacy rises over time (up and down, but mostly up) but never reaches the optimism of before teachers enter the classroom.
I talked to Helenrose about what her work means for University CS teachers. In general, the work she describes is about school teachers, not faculty. She agreed that it’s possible for University CS teachers to have high self-efficacy even if they are not successful teachers, because University teachers define self-efficacy differently than school teachers. School teachers are responsible for student learning. They know individual students. They actually know if they are successful in their teaching or not (in terms of student learning and engagement). University teachers tend to have larger classes, and they tend to teach via lecture. They usually have little knowledge of individual student learning and engagement. Their sense of self-efficacy may arise from their ability to succeed at their task, “I can give great lectures. (Almost nobody falls asleep.) I can manage huge classes.” Where they do have knowledge of learning and evidence of ineffective teaching, they may simply decide that it’s the student’s fault. Perhaps this is where the Geek Gene is born.
Here’s a hypothesis: If a University teacher has high self-efficacy (great confidence in his or her teaching ability) and sees evidence of students not learning, it’s rational for that teacher to believe that the problem lies with the students and that the problem is innate — beyond the ability of the teacher to improve it.
In the first study, Ni interviewed teachers about their identity in order to establish what strengths and weaknesses are common in high school computer science teachers. She found that the teaching identity of computer science teachers is largely underdeveloped compared to teachers in other fields, and that often computer science teachers prefer to identify as a math teacher or a business teacher, rather than a computer science teacher.
Further, she found that high school computer science teachers generally do not have any sort of teaching support community to turn to, because they are often the only computer science teacher at their school.
All of these problems combine to keep computer science teachers from developing a strong teaching identity centered in the computer science field. Instead, we have teachers with low commitment levels to the field training our next generation of programmers in basic computing skills that are generally unrelated to the field of computer science itself.
Annie Murphy Paul has a nice article about autodidacts — yes, there are some, but most of us aren’t. MOOCs are mostly for autodidacts. The paper from Educational Psychologist is excellent, and I reading the original as well as Paul’s review.
In a paper published in Educational Psychologist last year, Jeroen J.G. van Merriënboer of Maastricht University and Paul A. Kirschner of the Open University of the Netherlands challenge the popular assumption “that it is the learner who knows best and that she or he should be the controlling force in her or his learning.”
There are three problems with this premise, van Merriënboer and Kirschner write. The first is that novices, by definition, don’t yet know much about the subject they’re learning, and so are ill equipped to make effective choices about what and how to learn next. The second problem is that learners “often choose what they prefer, but what they prefer is not always what is best for them;” that is, they practice tasks that they enjoy or are already proficient at, instead of tackling the more difficult tasks that would actually enhance their expertise. And third, although learners like having some options, unlimited choices quickly become frustrating—as well as mentally taxing, constraining the very learning such freedom was supposed to liberate.
A recent article in Slate (see here) suggests that practice may not lead to expertise, that the “10,000 hour rule” is wrong. The “10,000 hour rule” was popularized by Malcolm Gladwell in his book Outliers (see excerpt here), but really comes from an important paper by K. Anders Ericsson and colleagues, “The Role of Deliberate Practice in the Acquisition of Expert Performance.” Ericsson claimed that 10,000 hours of deliberate practice results in expert-level performance.
The Slate article is based mostly on a new meta-analysis (see here) by Macnamara, Hambrick (also a co-author on the Slate article), and Oswald which reviewed and combined studies on expertise. They found that practice always was positively correlated with better performance, but did not explain all of (or even most of) the difference in expertise between study participants. The Slate article authors suggest, then, that deliberate practice is not as important as genetics or innate talent.
Deliberate practice left more of the variation in skill unexplained than it explained…There is now compelling evidence that genes matter for success, too…What all of this evidence indicates is that we are not created equal where our abilities are concerned.
The paper and article make two big mistakes that leave the “10,000 hour rule” as valid and valuable. The first is that practice is not the same as deliberate practice, and the second is that the fallback position can’t be genetics/innate talent. In general, their argument hinges on practice hours all being of equal value, which shows a lack of appreciation for the role of teaching.
Practice is not the same as deliberate practice
Ericsson was pretty clear in his paper that all practice is not created equal. Deliberate practice is challenging, focused on the skills that most need to be developed, with rapid feedback. (Here’s a nice blog post explaining deliberate practice.) Simply putting in 10,000 hours of practice in an activity does not guarantee expertise. Ericsson and the Slate authors would be in agreement on this point.
I’m sure that we’ve all seen musicians or athletes (and if we’re honest, we’ve probably all been like those musicians or athletes) who sometimes just “phone it in” during practice, or even during a game. I used to coach my daughters’ soccer teams, and I can absolutely assure you that there were hours in games and rehearsals where some of my players really didn’t make any progress. They found ways of getting through practice or games without really trying.
In the Macnamara paper, whether practice was “deliberate” or not was determined by asking people. They collected practice logs, surveys, and interviews. The participants in the studies self-reported whether the practice was deliberate. Imagine someone telling the interviewer or writing in their log, “Yeah, well, about 5,000 of those 10,000 hours, I was really lazy and not trying very hard.” It’s impossible to really distinguish practice from deliberate practice in this data set.
The bottom-line is that the Macnamara study did not test Ericsson’s question. They tested a weak form of the “10,000 hour rule” (that it’s just “practice,” not “deliberate practice”) and found it wanting. But their explanation, that it’s genetics, is not supported by their evidence.
Genetics/Innate starts at birth, no later
The Slate authors argue that, if practice doesn’t explain expertise, then it must be genetics. They cite two studies that show that identical twins seem to have similar music and drawing talent compared to fraternal twins. But that’s correlation and doesn’t prove causation — there may be any number of things on which the identical twins aren’t similar. (See this great Radiolab podcast exploring these kinds of miraculous misconceptions.)
If you’re going to make the genetics/innate argument, you have to start tracking participants at birth. Otherwise, there’s an awful lot that might add to expertise that’s not going to get counted in any practice logs.
I took classes on how to coach soccer. One of the lessons in those classes was, “It’s a poor coach who makes all practices into scrimmage.” Rather, we were taught to have students do particular drills to develop particular skills. (Sound like deliberate practice?) For example, if my players were having trouble dribbling, I might have them dribble a ball in a line around cones, across distances, through obstacles.
Can you imagine a child who one day might play in a soccer team with official practices — but before those practices and perhaps even before joining a team might dribble a ball around the neighborhood? Wouldn’t that be developing expertise? And yet, it wouldn’t be counted in player logs or practice hours. A kid who did lots of dribbling might come into a team and seem like a superstar with all kinds of innate talent. One might think that the kid had the “Soccer gene.”
To start counting hours-towards-expertise anything later than birth is discounting the impact of learning in the pre-school years on up. We know that pre-school years make a difference (see this website that Diana Franklin sent me, and the argument for pre-school in this recent Freakonomics podcast). A wide variety of activities can develop skills that can be influence expertise. If you don’t start tracking students from birth, then it’s hard to claim that you’ve counted in the practice log everything that’s relevant for expertise.
The claim that expertise is determined at birth is a common claim among CS educators. Most CS teachers to whom I’ve asked the question are convinced some people “can’t” learn to code, that it’s genetic or innate to learn programming. That’s where the myth of the “Geek Gene” came from (Raymond Lister has written several times on that). Couldn’t it be that there are dribbling-around-the-neighborhood activities that lead toward CS expertise? Consider the famous pre-programming activity of writing the instructions out for making a peanut-butter-and-jelly sandwich (like here). If we believe that that kind of practice helps to develop CS expertise, then other “writing instructions out” activities might lead towards CS expertise. Maybe people who seem to have genetic/innate ability in CS just did a lot of those kinds of activities before they got to our classes.
The clock on developing expertise doesn’t start when students walk through our door.
Bigger than P=NP: Is teaching > genetics?
In the end, it’s very difficult to prove or disprove that genetics accounts for expertise in cognitive skill. I don’t think Macnamara et al. settled the score. But my point about deliberate practice actually points to a much bigger issue.
Teachers Matter is the two word title of a 2012 OECD report (available here). There is a difference between great teachers and poor teachers, and the difference can be seen in terms of student performance. If you believe that (and there’s gobs of evidence that says you should), then it seems obvious that all practice is not created equal. Hours spent in practice with a good teacher are going to contribute more to expertise than hours spent without a teacher. Look back at that definition of “deliberate practice” — who’s going to pick the activities that most address your needs or provide the immediate feedback? The definition of deliberate practice almost assumes that there’s going to be teacher in the loop.
An open question is just how far we can get with excellent teaching. How much can we use teaching to get beyond genetic disparities? Is teaching more powerful than genetics? That’s an important question, and far more important than the classic CS question whether P=NP. I believe that there are limits. There are genetic problems that teaching alone can’t address. But we don’t know what those limits are.
We certainly have evidence that we can use teaching to get past some differences that have been chalked up to genetics or being innate. Consider the fact that men have better spatial skills than women. Is it innate, or is it learned? It’s not clear (see discussion on that here). But the important point is: it doesn’t matter. Terlecki, Newcombe, and Little have found that they can teach women to perform as well as men on visual skills and that the improvements in spatial ability both transfers and persists (see the journal article version here). The point is that spatial skills are malleable, they can be developed. Why should we think that other cognitive skills aren’t? The claims of the Slate authors and Macnamara et al ignore the power of a great teacher to go beyond simple rote practice to create deliberate opportunities to learn. The words teach, teacher, and teaching don’t appear in either article.
Here’s my argument summarized. The Slate authors and Macnamara et al. dismiss the 10K hour rule too lightly, and their explanation of genetic/innate basis for expertise is too simple. Practice is not the same as deliberate practice, or practice with a teacher. Expertise is learned, and we start learning at birth with expertise developing sometimes in ways not directly connected to the later activity. The important part is that we are able to learn to overcome some genetic/innate disparities with good teaching. We shouldn’t be giving up on developing expertise because we don’t have the genes. We should be thinking about how we can teach in order to develop expertise.
This is part of Briana Morrison’s dissertation work. She’s asking the question about the role of explaining programs in different modalities (e.g., visual vs. oral text) have on understanding. If you know potential applicants (e.g., maybe advertise it to your whole class?), please forward this to them. We’d appreciate it!
Do you like to watch videos on the internet?
Want to help with a research study?
We need volunteers, age 18 and older, with no computer programming experience to help us determine the best way to explain code using videos.
No more than 2 hours of your time!
Completing a portion of the study allows you to enter a raffle for one of four
$50 Amazon Gift Cards
Completion of entire study allows you to enter a raffle for one
$100 Amazon Gift Card
Interested? Go to the following website: