Posts tagged ‘learning sciences’
Briana Morrison is presenting the next stage of our work on subgoal labeled worked examples, with Lauren Margulieux. Their paper is “Subgoals, Context, and Worked Examples in Learning Computing Problem Solving.” As you may recall, Lauren did a terrific set of studies (presented at ICER 2012) showing how adding subgoal labels to videos of App Inventor worked examples had a huge effect on learning, retention, and transfer (see my blog post on this work here).
Briana and Lauren are now teaming up to explore new directions in educational psychology space and new directions in computing education research.
- In the educational psychology space, they’re asking, “What if you make the students generate the subgoal labels?” Past research has found that generating the subgoal labels, rather than just having them given to the students, is harder on the students but leads to more learning.
- They’re also exploring what if the example and the practice come from the same or different contexts (where the “context” here is the cover story or word problem story). For example, we might show people how to average test grades, but then ask them to average golf scores — that’s a shift in context.
- In the computing education research space, Briana created subgoal labeled examples for a C-like pseudocode.
One of the important findings is that they replicated the earlier study, but now in a text-based language rather than a blocks-based language. On average, subgoal labels on worked examples improve performance over getting the same worked examples without subgoal labels. That’s the easy message.
The rest of the results are much more puzzling. Being in the same context (e.g., seeing averaging test scores in the worked examples, then being asked to average test scores in the practice) did statiscally worse than having to shift contexts (e.g., from test scores to golf scores). Why might that be?
Generating labels did seem to help performance. The Generate group had the highest attrition. That make sense, because increased complexity and cognitive load would predict that more participants would give up. But that drop-our rate makes it hard make strong claims. Now we’re comparing everyone in the other groups to only “those who gut it out” in the Generate group. The results are more suspect.
There is more nuance and deeper explanations in Briana’s paper than I’m providing here. I find this paper exciting. We have an example here of well-established educational psychology principles not quite working as you might expect in computer science. I don’t think it puts the principles in question. It suggests to me that there may be some unique learning challenges in computer science, e.g., if the complexity of computer science is greater than in other studies, then it’s easier for us to reach cognitive overload. Briana’s line of research may help us to understand how learning computing is different from learning statistics or physics.
Important new paper in Nature that makes the argument for active learning in all science classes, which is one of the arguments I was making in my Top Ten Myths blog post. The image and section I’m quoting below are about a different issue than learning — turns out that active learning methods are important for retention, too.
Active learning is winning support from university administrators, who are facing demands for accountability: students and parents want to know why they should pay soaring tuition rates when so many lectures are now freely available online. It has also earned the attention of foundations, funding agencies and scientific societies, which see it as a way to patch the leaky pipeline for science students. In the United States, which keeps the most detailed statistics on this phenomenon, about 60% of students who enrol in a STEM field switch to a non-STEM field or drop out2 (see ‘A persistence problem’). That figure is roughly 80% for those from minority groups and for women.
I’m currently reading Nobel laureate Daniel Kahneman’s book, “Thinking Fast, Thinking Slow” (see here for the NYTimes book review). It’s certainly one of the best books I’ve ever read on behavioral economics, and maybe just the best book I’ve ever read about psychology in general.
One of the central ideas of the book is our tendency to believe “WYSIATI”—What You See Is All There Is. Kahneman’s research suggests that we have two mental systems: System 1 does immediate, intuitive responses to the world around us. System 2 does thoughtful, analytical responses. System 1 aims to generate confidence. It constructs a story about the world given what information that exists. And that confidence leads us astray. It keeps System 2 from asking, “What am I missing?” As Kahneman says in the interview linked below, “Well, the main point that I make is that confidence is a feeling, it is not a judgment.”
It’s easy to believe that University CS education in the United States is in terrific shape. Our students get jobs — multiple job offers each. Our graduates and their employers seem to be happy. What’s so wrong with what’s going on? I see computation as a literacy. I wonder, “Why is our illiteracy rate so high? Why do so few people learn about computing? Why do so many flunk out, drop out, or find it so traumatic that they never want to have anything to do with computing again? Why are the computing literate primarily white or Asian, male, and financially well-off compared to most?”
Many teachers (like the comment thread after this post) argue for the state of computing education based on what they see in their classes. We introduce tools or practices and determine whether they “work” or are “easy” based on little evidence, often just discussion with the top students (as Davide Fossati and I found). If we’re going to make computing education work for everyone, we have to ask, “What aren’t we seeing?” We’re going to feel confident about what we do see — that’s what System 1 does for us. How do we see the people who aren’t succeeding with our methods? How do we see the students who won’t even walk in the door because of how or what we teach? That’s why it’s important to use empirical evidence when making educational choices. What we see is not all there is.
But, System 1 can sometimes lead us astray when it’s unchecked by System 2. For example, you write about a concept called “WYSIATI”—What You See Is All There Is. What does that mean, and how does it relate to System 1 and System 2?
System 1 is a storyteller. It tells the best stories that it can from the information available, even when the information is sparse or unreliable. And that makes stories that are based on very different qualities of evidence equally compelling. Our measure of how “good” a story is—how confident we are in its accuracy—is not an evaluation of the reliability of the evidence and its quality, it’s a measure of the coherence of the story.
People are designed to tell the best story possible. So WYSIATI means that we use the information we have as if it is the only information. We don’t spend much time saying, “Well, there is much we don’t know.” We make do with what we do know. And that concept is very central to the functioning of our mind.
I wrote that blog post because we really have had a long debate in our faculty email list about many of those topics. I recently saw our Dean at an event, and he told me that he hadn’t read the thread yet (but he planned to) because “it must be 100 messages long.” Most of the references in that blog post came from messages that I wrote in response to that thread. It was a long post because people generally didn’t agree with me. Several senior, well-established (much more famous than me) faculty strongly disagreed with the evidence-based argument I was making. The thread finally ended when one of the most senior, most respected faculty in the College wrote a note saying (paraphrased), “There are probably better teaching evaluation methods than the ones we now use. I’m sure that Mark knows teaching methods that would help the rest of us teach better.” And that was it. Thread ended. The research-based evidence that I offered was worth fighting about. The word of authority was not.
I’ll bet that faculty across disciplines similarly respond to authority more than evidence. We certainly see the role of authority in Physics Education Research (PER). Pioneering PER researchers were not given much respect and many were ostracized from their departments. Until Eric Mazur at Harvard had his students fail the Force Concept Inventory (FCI), and he changed how he taught because of it. Until Nobel laureate Carl Wieman decided to back PER (all the way to the Office of Science Technology and Policy in the White House). Today, the vast majority of physics teachers know research-based teaching methods (even if they don’t always use them). FCI existed before Mazur started using it, but it really started getting used after Mazur’s support. The evidence of FCI didn’t change physics teaching. The voice of authority did.
While we might wish that CS faculty would respond more to evidence than authority (see previous post on this theme), this insight suggests a path forward. If we want CS faculty to improve their teaching and adopt evidence-based practices, top-down encouragement can have large impact. Well-known faculty at top institutions publicly adopting these practices, and Deans and Chairs promoting these practices can help to convince faculty to change.
I’ve written a couple times now about the workshop I attended at the University of Oldenburg the first week of June. (See the post where I talked about my two weeks in Germany.) For Blog@CACM, I wrote a post about teaching as collective practice and the workshop I took with Barbara Hofer (see post here). I wrote about learning about teacher beliefs and self-efficacy from Helenrose Fives here (see post).
Before we left for the workshop, I got to spend time with Ira Diethelm at the University of Oldenburg and one of her students. Ira is one of at least 16 (that Ira could count) CS Education professors in Germany. Ira works with pre-service teachers, in-service teachers, and graduate students. Her graduate students build outreach efforts and curricula as part of their research, then roll them out and provide resources to teachers. It’s remarkable what Ira is doing, and I understand that the other German CS Ed professors do similar things. I came away with a new insight: If we want to bootstrap and sustain CS Education in the United States, we should fund several endowed chairs of CS Education at top Schools of Education. Eventually, we have to have pre-service computing education programs if we want to make CS education sustainable (see that post here). Creating these endowed chairs gives us the opportunity to create positions like Ira’s in the United States.
Overall, the workshop was a terrific experience. The PhD student work was fascinating, and I enjoyed discussing their research with them. It was great to hear about German research perspectives that I hadn’t previously, like the Model of Educational Reconstruction that informs science education (see paper here). Barbara and Helenrose were only two of a several outstanding international education researchers who attended. As I mentioned to Pat Alexander (who has a lengthy Wikipedia page of her accomplishments), I enjoyed being able to wallow in educational psychology for a week, because I so rarely get to do that. I gave a talk on three of our projects related to the theme of developing teachers: on Lijun Ni’s work on teacher identity, on the Disciplinary Commons for Computing Education, and on our ebook for preparing CS teachers. (See Slideshare here.)
The response to my talk was fascinating. Some of the German mathematics education researchers are deeply opposed to computing education in schools. (I suspect that more than one of them completely skipped my talk because they are so opposed.) “Computing education keeps stealing from mathematics teachers, and learning mathematics is more important.” At my talk, Pat Alexander asked me the same question that Peter Elias asked Alan Perlis in 1961, “Won’t the computer eventually just understand us? Doesn’t the computer just become invisible and not need to be programmed?” I told the story about Alan Perlis’s talk and about Michael Mateas’s argument, “There will always be friction.” From the computing educators, I heard a lot of anger. The German computing education researchers feel that other fields squeeze CS out because the they are not willing to allow computing education to take up any time or budget in the curriculum.
Probably the most interesting pushback was against computational thinking. The educational psychologists thought it was unbelievable that learning computing would in any way impact the way that people think or problem-solve in everyday life. “Didn’t we believe that once about Latin? and Geometry?” asked Gavin Brown. The psychologists at the workshop I attended saw a clear argument that we need to introduce computing in high school so that students can see if it’s for them, but not to teach general problem-solving skills. If we really want algorithmic thinking, they can design easier ways to achieve that goal than teaching programming.
We can probably help students to learn about computing in such a way that it might influence problem-solving on the computer. That’s part of Jeanette Wing’s model of Computational Thinking (see her 2010 paper). It’s the “Computational Thinking in Daily Life” part that the psychologists weren’t buying. That learning about computation helps with computational X is quite reasonable. If you understand what IP addresses are, we can help you to understand DNS problems and to realize that it’s not really that big of a deal if Wikipedia stores your IP address (see story about Erika Poole’s research). There is evidence that learning one programming language will likely transfer to another one (see Michal Armoni’s paper on transfer from Scratch to a text-based language). Learning to program is unlikely to influence any problem-solving in everyday life.
We’re years into the MOOC phenomenon, and I’d hoped that we’d get past MOOC hype. But we’re not. The article below shows the same misunderstandings of learning and teaching that we heard at the start — misunderstandings that even MOOC supporters (like here and here) have stopped espousing.
The value of being in the front row of a class is that you talk with the teacher. Getting physically closer to the lecturer doesn’t improve learning. Engagement improves learning. A MOOC puts everyone at the back of the class, listening only and doing the homework.
In many ways, we have a romanticized view of college. Popular portrayals of a typical classroom show a handful of engaged students sitting attentively around a small seminar table while their Harrison Ford-like professor shares their wisdom about the world. We all know the real classroom is very different. Especially in big introductory classes — American history, U.S. government, human psychology, etc. — hundreds of disinterested, and often distracted, students cram into large impersonal lecture halls, passively taking notes, occasionally glancing up at the clock waiting for the class to end. And it’s no more engaging for the professor. Usually we can’t tell whether students are taking notes or updating their Facebook page. For me, everything past the ninth row was distance learning. A good online platform puts every student in the front row.
I agree strongly with the idea of “learning engineers.” Having learning engineers doesn’t relieve faculty who teach from the responsibility to learn more about learning sciences (see my blog post about testing teachers about PCK). Just teaming up subject-matter experts with learning engineers does not inform a teacher’s day-to-day and in-class decision-making. The general theme below is one I strongly agree with — we should rely more on evidence-based and research-based teaching.
We are missing a job category: Where are our talented, creative, user-centric “learning engineers” — professionals who understand the research about learning, test it, and apply it to help more students learn more effectively?
Jobs are becoming more and more cognitively complex, while simpler work is disappearing. (Even that old standby, cab driving, may one day be at risk from driverless cars from Google!) Our learning environments need to do a better job of helping more people of all ages master the complex skills now needed in many occupations.
I am not suggesting that all subject-matter experts (meaning faculty members) need to become learning engineers, although some might. However, students and faculty members alike would benefit from increased collaboration between faculty members and learning experts — specialists who would respect each other’s expertise — rather than relying on a single craftsman in the classroom, which is often the case in higher education today.