Posts tagged ‘educational psychology’
I’m looking forward to these results! That interaction is better than video lectures is really not surprising. That it leads to better learning than even reading is quite a surprise. My guess is that this is mediated by student ability as a reader, but as a description of where students are today (like the prior posts on active learning), it’s a useful result.
Koedinger and his team further tested whether their theory that “learning by doing” is better than lectures and reading in other subjects. Unfortunately, the data on video watching were incomplete. But they were able to determine across four different courses in computer science, biology, statistics and psychology that active exercises were six times more effective than reading. In one class, the active exercises were 16 times more effective than reading. (Koedinger is currently drafting a paper on these results to present at a conference in 2016.)
Blog Post #2000: Barbara Ericson Proposes: Effectiveness and Efficiency of Adaptive Parsons Problems #CSEdWeek
My 1000th blog post looked backward and forward. This 2000th blog post is completely forward looking, from a personal perspective. Today, my wife and research partner, Barbara Ericson, proposes her dissertation.
Interesting side note: One of our most famous theory professors just blogged on the theory implications of the Parsons Problems that Barb is studying. See post here.
Barb’s proposal is the beginning of the end of this stage in our lives. Our youngest child is a senior in high school. When Barbara finishes her Human-Centered Computing PhD (expected mid-2017), we will be empty-nesters and ready to head out on a new adventure.
Title: EVALUATING THE EFFECTIVINESS AND EFFICIENCY OF PARSONS PROBLEMS AND DYNAMICALLY ADAPTIVE PARSONS PROBLEMS AS A TYPE OF LOW COGNITIVE LOAD PRACTICE PROBLEM
Barbara J. Ericson
Human Centered Computing
College of Computing
Georgia Institute of Technology
Date: Wednesday, December 9, 2015
Time: 12pm to 2pm EDT
Location: TSRB 223
Dr. James Foley, School of Interactive Computing (advisor)
Dr. Amy Bruckman, School of Interactive Computing
Dr. Ashok Goel, School of Interactive Computing
Dr. Richard Catrambone, School of Psychology
Dr. Mitchel Resnick, Media Laboratory, Massachusetts Institute of Technology
Learning to program can be difficult and can result in hours of frustration looking
for syntactic or semantic errors. This can make it especially difficult to prepare inservice
(working) high school teachers who don’t have any prior programming
experience to teach programming, since it requires an unpredictable amount of time for
practice in order to learn programming. The United States is trying to prepare 10,000
high school teachers to teach introductory programming courses by fall 2016. Most
introductory programming courses and textbooks rely on having learners gain experience
by writing lots of programs. However, writing programs is a complex cognitive task,
which can easily overload working memory, which impedes learning.
One way to potentially decrease the cognitive load of learning to program is to
use Parsons problems to give teachers practice with syntactic and semantic errors as well
as exposure to common algorithms. Parsons problems are a type of low cognitive load
code completion problem in which the correct code is provided, but is mixed up and has
to be placed in the correct order. Some variants of Parsons problems also require the
code to be indented to show the block structure. Distractor code can also be provided
that contains syntactic and semantic errors.
In my research I will compare solving Parsons problems that contain syntactic and
semantic errors, to fixing code with the same syntactic and semantic errors, and to writing
the equivalent code. I will examine learning from pre- to post-test as well as student
reported cognitive load. In addition, I will create dynamically adaptive Parsons problems
where the difficulty level of the problem is based on the learners’ prior and current
progress. If the learner solves one Parsons problem in one attempt the next problem will
be made more difficult. If the learner is having trouble solving a Parsons problem the
current problem will be made easier. This should enhance learning by keeping the
problem in the learner’s zone of proximal development as described by Vygotsky. I will
compare non-adaptive Parsons problems to dynamically adaptive Parsons problems in
terms of enjoyment, completion, learning, and cognitive load.
The major contributions of this work are a better understanding of how variants of
Parsons problems can be used to improve the efficiency and effectiveness of learning to
program and how they relate to code fixing and code writing. Parsons problems can help
teachers practice programming in order to prepare them to teach introductory computer
science at the high school level and potentially help reduce the frustration and difficulty
all beginning programmers face in learning to program.
Cognitive Load as a Significant Problem in Learning Programming: Briana Morrison’s Dissertation Proposal
Briana Morrison is defending her proposal today. One chapter of her work is based on her ICER 2015 paper that won the Chairs Award for best paper (see post here). Good luck, Briana!
Title: Replicating Experiments from Educational Psychology to Develop Insights into Computing Education: Cognitive Load as a Significant Problem in Learning Programming
Human Centered Computing
College of Computing
Georgia Institute of Technology
Date: Wednesday, November 11, 2015
Time: 2 PM to 4 PM EDT
Location: TSRB 223
Dr. Mark Guzdial, School of Interactive Computing (advisor)
Dr. Betsy DiSalvo, School of Interactive Computing
Dr. Wendy Newstetter, School of Interactive Computing
Dr. Richard Catrambone, School of Psychology
Dr. Beth Simon, Jacobs School of Engineering at University of California San Diego and Principal Teaching and Learning Specialist, Coursera
Students often find learning to program difficult. This may be because the concepts are inherently difficult due to the fact that the elements of learning to program are highly interconnected. Instructors may be able to lower the complexity of learning to program by designing instructional materials that use educational psychology principles.
The overarching goal of this research is to gain more understanding and insight into the optimal conditions under which learning programming can be successful which is defined as students being able to apply their acquired knowledge and skills in new or familiar problem-solving situations. Cognitive load theory (CLT), and its associated effects, describe the role of the learner’s memory during the learning process. By minimizing undesirable loads within the instructional materials the learner’s memory can hold more relevant information, thereby improving the effectiveness of the learning process.
This proposal uses cognitive load theory to improve learning in programming. First an instrument for measuring cognitive load components within introductory programming was developed and initially validated. We have explored reducing the cognitive load by changing the modality in which students receive the learning material. This had no effect on novices’ retention of knowledge or their ability to transfer knowledge. We then attempted to reduce the cognitive load by adding subgoal labels to the instructional material. This had some effect on the learning gains under some conditions. Students who learned using subgoal labels demonstrated higher learning gains than the other conditions on the programming assessment task. We also explored using a low cognitive load assessment task, a Parsons problem, to measure learning gains. This low cognitive load assessment task proved more sensitive than the open ended programming assessment tasks in capturing student learning. Students who were given subgoal labels regardless of context transfer condition out performed those in the other conditions.
In my final, proposed study I change how we teach a programming construct through its format and content in order to reduce cognitive load. The changed construct is presumed to be a more natural cognitive fit for students based on previous research.
Human students need active learning and Econs learn from lecture: NYTimes Op-Ed in defense of lecture
I’m sympathetic to the author’s argument (linked below), that being able to understand an argument delivered as a lecture is difficult and worthwhile. Her characterization of active learning is wrong — it’s not “student-led discussion.” Actually, what she describes as good lecture is close to good active learning. Having students answering questions in discussion is good — but some students might disengage and not answer questions. Small group activities, peer led team learning, or peer instruction would be better to make sure that all students engage. But that’s not the critical flaw in her argument.
Being able to listen to a complicated lecture is an important skill — but students (at least in STEM, at least in the US) don’t have that skill. We can complain about that. We can reform primary and secondary schooling so that students develop that skill. But if we want these students to learn, the ones who are in our classes today, we should use active learning strategies.
Richard Thaler introduced the term “Econs” to describe the rational beings that inhabit traditional economic theory. (See a review of his book Misbehaving for more discussion on Econs.) Econs are completely rational. They develop the skills to learn from lecture because it is the most efficient way to learn. Unfortunately, we are not econs, and our classes are filled with humans. Humans are predictably irrational, as Daniel Ariely puts it. And there’s not much we can do about it. In his book Thinking, Fast and Slow, Daniel Kahneman complains that he knows how he is influenced by biases and too much System 1 thinking — and yet, he still makes the same mistakes. The evidence is clear that the students in our undergraduate classes today need help to engage with and learn STEM skills and concepts.
The empirical evidence for the value of active learning over lecture is strong (see previous post). It works for humans. Lecture probably works for Econs. If we could find enough of them, we could run an experiment.
In many quarters, the active learning craze is only the latest development in a long tradition of complaining about boring professors, flavored with a dash of that other great American pastime, populist resentment of experts. But there is an ominous note in the most recent chorus of calls to replace the “sage on the stage” with student-led discussion. These criticisms intersect with a broader crisis of confidence in the humanities. They are an attempt to further assimilate history, philosophy, literature and their sister disciplines to the goals and methods of the hard sciences — fields whose stars are rising in the eyes of administrators, politicians and higher-education entrepreneurs.
A similar argument to mine is below. This author doesn’t use the Humans/Econs distinction that I’m using. Instead, the author points out that lecturers too often teach only to younger versions of themselves.
I will grant that nothing about the lecture format as Worthen describes it is inherently bad. But Worthen’s elegy to a format that bores so many students reminds me of a bad habit that too many professors have: building their teaching philosophies around younger versions of themselves, who were often more conscientious, more interested in learning, and more patient than the student staring at his phone in the back of their classrooms.
Lecia Barker had a terrific paper in SIGCSE 2015 that I just recently had the chance to dig into. (See paper in ACM DL here.) Here’s the abstract:
Despite widespread development, research, and dissemination of teaching and curricular practices that improve student retention and learning, faculty often do not adopt them. This paper describes the first findings of a two-part study to improve understanding of adoption of teaching practices and curriculum by computer science faculty. The paper closes with recommendations for designers and developers of teaching innovations hoping to increase their chance of adoption.
I’ve published in this area before. Davide Fossati and I wrote a paper about the practices of CS teachers (based on interviews with about a dozen CS university teachers): how they made change, what convinced them to change, and how they decided if the change worked. (See blog post about this here.) The general theme was that these decisions rarely had an empirical basis.
Lecia and her co-authors went far beyond our study. She interviewed and observed 66 CS faculty from 36 institutions, explicitly chosen to represent a diverse set of schools. The result is the best picture I’ve yet seen of how CS faculty make decisions.
Lecia found more evidence of teachers using empirical evidence than we did, which was great to see. But whether students “liked” it or not was still the most critical variable:
On the other hand, if students don’t “like it,” faculty are unlikely to continue using a new practice. At a public research university, a professor said, “You can do something that you think, ‘Wow! If the learning experience was way better this term, the experiment really worked.’ And then you read your teaching reviews, and it’s like the students are pissed off because you did not do what they expected.”
Lecia discovered a reason not to adopt that I’d not heard before. She found that CS teachers filter out innovations that didn’t come from a context like their own. Those of us at research universities are filtered out by some teachers at teaching-oriented institutions:
Faculty trust colleagues who have similar teaching and research contexts, share attitudes toward students and teaching, or teach similar subjects. In describing what conference speakers he finds credible at SIGCSE, a professor at a private liberal arts university acknowledged, “I do have the anti- ‘Research One’ bias. Like if the speaker is somebody who teaches at <prestigious public research university>, the mental clout that I give them as a teacher—unless they’re a lecturer—I drop them a notch. When someone stands up to speak and they’re from a really successful teaching college <names several> or universities that have a real reputation of being great undergraduate teaching institutions, I give them a lot of merit.”
The part that I found most depressing (even if not surprising) is that research evidence did not matter at all in adopting new ways to teach:
Despite being researchers themselves, the CS faculty we spoke to for the most part did not believe that results from educational studies were credible reasons to try out teaching practices.
Lecia’s study is well done, and the paper is fascinating, but the overall picture is rather dismal. She points out many other issues that I’m not going into here, like the trade-off between cost and benefit of adopting a new practice, and about the need for specialized equipment in classrooms for some new practices. Overall, she finds that it’s really hard to get higher education CS faculty to adopt better practices. We reported on that in “Georgia Computes!” (see post here) but it’s even more disappointing when you see it in a large, broad study like this.
The New York Times weighs in on the argument about active learning versus passive lecture. The article linked below supports the proposition that college lectures unfairly advantage those students who are already privileged. (See the post about Miranda Parker’s work for a definition of what is meant by privilege.)
The argument that we should promote active learning over passive lecture has been a regular theme for me for a few weeks now:
- I argued in Blog@CACM that hiring ads and RPT requirements should be changed explicitly to say that teaching statements that emphasize active learning would be more heavily weighted (see post here).
- The pushback against this idea was much greater than I anticipated. I asked on Facebook if we could do this at Georgia Tech. The Dean of the College of Engineering was supportive. Other colleagues were strongly against it. I wrote a blog post about that pushback here.
- I wrote a Blog@CACM post over the summer about the top ten myths of computing education, which was the top-visited page at CACM during the month of July (see post here). I wrote that post in response to a long email thread on a College of Computing faculty mailing list, where I experienced that authority was able to sway CS faculty more than research results (blog post about that story here).
The NYTimes piece pushes on the point that this is not just an argument about quality of education. The argument is about what is ethical and just. If we value broadening participation in computing, we should use active learning methods and avoid lecture. If we lecture, we bias the class in favor of those who have already had significant advantages.
Thanks to both Jeff Gray and Briana Morrison who brought this article to my attention.
Yet a growing body of evidence suggests that the lecture is not generic or neutral, but a specific cultural form that favors some people while discriminating against others, including women, minorities and low-income and first-generation college students. This is not a matter of instructor bias; it is the lecture format itself — when used on its own without other instructional supports — that offers unfair advantages to an already privileged population.
The partiality of the lecture format has been made visible by studies that compare it with a different style of instruction, called active learning. This approach provides increased structure, feedback and interaction, prompting students to become participants in constructing their own knowledge rather than passive recipients.
Research comparing the two methods has consistently found that students over all perform better in active-learning courses than in traditional lecture courses. However, women, minorities, and low-income and first-generation students benefit more, on average, than white males from more affluent, educated families.
My Blog@CACM post this month makes a concrete proposal (quoted and linked below). We (all academic computing programs) should incentivize faculty to use active learning methods by evaluating teaching statements for hiring, tenure, and promotion more highly that reference active learning and avoid lecture.
On my Facebook page, I linked to the article and tagged our Dean of Engineering, the Vice-Provost for Undergraduate Education, and the RPT Chair for our College, and asked, “Can we do this at Georgia Tech?” The pushback on my Facebook page was the longest thread I’ve ever been part of on Facebook.
The issues raised were interesting and worth discussing:
- Would implementing this put at a disadvantage new PhD’s who have no teaching experience and don’t learn about active teaching? Yes, but that incentivizes those PhD programs to change.
- My blog post title is “Be It Resolved: Teaching Statements must embrace Active Learning and eschew Lecture.” I chose the word “eschew” deliberately. It doesn’t mean “ban.” It means “deliberately avoid using” which is what I meant. Lecture has its place — I wrote a blog post defending lecture which still gets viewed pretty regularly. The empirical evidence suggests that we should use active learning more than lecture for undergraduate STEM education.
- Should such a requirement for teaching statements emerge from faculty talking about it, or should it be done by administrative fiat? I lean toward the latter. As I’ve pointed out, CS faculty tend to respond to authority more than evidence. The administration should do the right thing, and deal with educating teachers (e.g., what are active learning methods first? how do we use them? even in large classes?) later. Faculty will learn the active learning methods in order to create those teaching statements. The incentive comes first.
- Lots of respondents thought I was saying that we should require all teaching to be active learning. I wasn’t, and I don’t know how to enforce that anyway. By evaluating teaching statements more heavily that emphasize active learning, we create an incentive, not a requirement.
- Some faculty pushed back, “How about students that like lecture? Tough luck for them?” Since we know that active learning is better, even for students who like lecture — yes.
- Several respondents suggested that active learning is just too hard, that faculty are over-stressed as it is. Faculty are over-stressed, but active learning isn’t that hard. In fact, it’s hard for faculty because they have to be quiet and listen in class more. It is hard to make change, but that’s the point of incentives. We start somewhere.
- The biggest theme in the thread is that we should first aim to get faculty to care about teaching and to take active steps to improve their teaching. I don’t think that’s enough. Libertarian paternalism (see Wikipedia page) suggests that we set the incentive at the minimal acceptable level (use of active learning) then encourage choice above that (choosing among the wide variety of active learning methods). We don’t want people to choose options that won’t be in the best interests of the largest number of people.
The discussion went on for four days (and hasn’t quite petered out yet). I do wonder if active learning methods will be forced upon faculty if we don’t willingly pick them up. The research evidence is overwhelming, with articles in Nature and hundreds of studies reviewed in the Proceedings of the National Academy of Sciences. How long before we get sued for teaching but not using the best teaching methods? One of the quotes in the blog post says, “At this point it is unethical to teach any other way.” We should take concrete steps towards doing the right thing, because it’s the right thing to do.
Here is something concrete that we in academia can do. We can change the way we select teachers for computer science and how we reward faculty.
All teaching statements for faculty hiring, promotion, and tenure should include a description of how the candidate uses active learning methods and explicitly reduces lecture.
We create the incentive to teach better. We might simply add a phrase to our job ads and promotion and tenure policies like, “Teaching statements will be more valued that describe how the candidate uses active learning methods and seeks to reduce lecture.”