What Do We Actually Learn From TED Talk Videos?

I usually really like Annie Murphy Paul’s articles, but this one didn’t work for me.  Below are her reasons why TED talk videos work well in learning, with my comments interspersed.

• They gratify our preference for visual learning. Effective presentations treat our visual sense as being integral to learning. This elevation of the image—and the eschewal of text-heavy Power Point presentations—comports well with cognitive scientists’ findings that we understand and remember pictures much better than mere words.

Cognitive scientists like Richard Mayer have found that diagrams and pictures can enhance learning — absolutely.  But his work combined diagrams with words (e.g., best combination with diagrams: audio narration, not visual text).  This quote seems to suggest that pictures are better than words. For most of STEM, that’s not true.  We may have an affinity for visual, but that doesn’t mean that it works better for learning complex material.

• They engage the power of social learning. The robust conversation that videos can inspire, both online and off, recognizes a central principle of adult education: We learn best from other people. In the discussions, debates, and occasional arguments about the content of the talks they see, video-watchers are deepening their own knowledge and understanding.

Wait a minute — isn’t she just saying that TED talks give us something to talk about? TED talks are not themselves inherently social.  Isn’t a book discussed in a book club just as effective for “engaging the power of social learning”?  What makes TED talks so “social”?

• They enable self-directed, “just-in-time” learning. Because video viewers choose which talks to watch and when to watch them, they’re able to tailor their education to their own needs. Knowledge is easiest to absorb at the moment when we’re ready to apply it.

This was the quote that inspired this blog post.  It’s an open question, but here’s my hypothesis.  Nobody watches a TED talk for “just-in-time” learning.  People watch TED talk for entertainment.  ”I am about to go to my school board meeting — I think I’ll watch Sir Ken Robinson to figure out what to say!”  ”I need to be able to guess birthdays — isn’t there a TED talk on that?”  There are videos that really work for “just-in-time” learning.  TED talks aren’t like that.

• They encourage viewers to build on what they already know. Adults are not blank slates: They bring to learning a lifetime of previously acquired information and experience. Effective video instruction build on top of this knowledge, adding and elaborating without dumbing down.

via What Do We Actually Learn From Videos? | MindShift.

It’s absolutely true that effective instruction builds on top of existing knowledge, which is something that the best teachers know how to do — to figure out what students know and care about, and relate knowledge to that.  How does a fixed video build on what viewers (all hundreds of thousands of them) actually know?  No, I don’t see how TED talks do that.

AAAS Forum: Can MOOCs work for all students?

Way to go, Wendy!  My Georgia Tech colleague did really well at a recent AAAS forum on MOOCs.  The tone between the three speakers is striking.  Anant Agarwal says “Hype is a good thing!”  Kevin Wehrbach says that a MOOC is “an extraordinary teaching and learning experience.”  Then Wendy Newstetter lets loose with concerns supported with citations and hard research questions.

In any learning environment, students should gain “transferable knowledge” that can be applied in many contexts, said Newstetter, citing a 2012 National Academies’ report on Education for Life and Work. Specifically, she said, researcher James Pellegrino has identified an array of cognitive, interpersonal and intrapersonal skills that all students need in order to succeed. How can the array of new online learning models help students achieve those goals?

Newstetter proposed a series of questions that should be answered by research. Educators need to know, for example, under what conditions technology-mediated experiences can result in enhanced learning competencies, she said. Do MOOCs effectively encourage students to develop perseverance, self-regulation and other such skills? Is knowledge gained in a MOOC “transferable,” so that what students learn can help them solve problems in other contexts? How can MOOCs be enhanced to promote interpersonal skills, and what intrapersonal attributes are needed for optimal learning in MOOCs?

Some observers have suggested that MOOCs tend to work best for more affluent students, Newstetter noted. She mentioned the 2013 William D. Carey lecture, presented at the AAAS Forum by Freeman Hrabowski III, president of the University of Maryland, Baltimore County, who focused on strategies for helping underrepresented minorities succeed in science fields. “What he described was high-contact, intensive mentoring,” she pointed out.

via AAAS – Massive Open Online Courses Help Make STEM Education More Accessible, But Do They Work for All Students?.

Making Education a Team Sport

This is a compelling vision.  Set aside MOOCs or not — how could we use a team-based approach in building postsecondary education, so that we have the best of texts, tools, in-class experiences, videos, and individualized tutoring and advising?  If we want higher-quality, we can’t expect one teacher to perform all roles for increasing numbers of students.

The real threat to traditional higher education embraces a more radical vision that removes faculty from the organizational center and uses cognitive science to organize the learning around the learner. Such models exist now.

Consider, for example the implications of Carnegie Mellon’s Open Learning Initiative. More than 10 years ago, Herb Simon, the Carnegie Mellon University professor and Nobel laureate, declared, “Improvement in postsecondary education will require converting teaching from a solo sport to a community-based research activity.” The Open Learning Initiative (OLI) is an outgrowth of that vision and has been striving to realize it for more than a decade.

via Essay on how technology and new ways of teaching could upend colleges’ traditional models | Inside Higher Ed.

Free Early-Career Learning Sciences Workshop at CMU LearnLab

Call for Participation

2nd Annual Learning Science Workshop
Research and Innovation for Enhancing Achievement and Equity
http://www.learnlab.org/opportunities/summerworkshop.php
June 22-23
Carnegie Mellon University
Pittsburgh PA
Applications Due May 5, 2013

*No Cost To Attend*

Overview

LearnLab, an NSF Science of Learning Center (SLC) at Carnegie Mellon and the University of Pittsburgh, has an exciting summer research opportunity available to early career researchers in the fields of psychology, education, computer science, human-computer interfaces and language technologies.

The workshop is targeted to senior graduate students, post-docs and early career faculty. The workshop seeks broad participation, including members of underrepresented groups as defined by NSF (African American, Hispanic, Native American) who may be considering a research or faculty position in the learning sciences.

This two-day workshop immediately precedes the LearnLab Summer School (www.learnlab.org/opportunities/summer/). Our research theme is theresearch and innovation for enhancing achievement and equity, including these five areas:

* Enhancing Achievement through Educational Technology and Data Mining. Using domain modeling, and large datasets to discover when learning occurs and to provide scaffolding for struggling students. See http://www.learnlab.org/research/wiki/index.php/Computational_Modeling_and_Data_Mining.

* 21st Century Skills, Dispositions, and Opportunities. Re-examining the goals of education and assessment and considering transformative changes in how and where learning occurs.

* Opening Classroom Discourse. Studying how classroom talk contributes to domain learning and supports equity of learning opportunity. See LearnLab’s Social-Communicative Factors thrustwww.learnlab.org/research/wiki/index.php/Social_and_Communicative_Factors_in_Learning.

* Course-Situated Research. Running principle-testing experiments while navigating the complex waters of real-world classrooms. Seewww.learnlab.org/research/wiki/index.php/In_vivo_experiment.

* Motivation Interventions for Learning. Implementing theory based motivational interventions to target at risk populations to improve robust student learning. Seehttp://www.learnlab.org/research/wiki/index.php/Metacognition_and_Motivation

The substantive focus of the workshop is the use of current research and innovations to enhance achievement and equity at all levels of learning. Activities will include demonstrations of the diverse set of ongoing learning sciences research projects at LearnLab, and poster presentations or talks by participants. Participants will also meet with LearnLab faculty in research groups and various informal settings. We will provide information about becoming a part of the Carnegie Mellon or University of Pittsburgh learning science community.

In addition to these substantive themes, the workshop will provide participants with opportunities for professional development and the chance to gain a better understanding of the academic career ladder. These include mentoring that focuses on skills, strategies and “insider information” for career paths. Sessions will include keynote speakers and LearnLab senior faculty discussing professional development topics of interest to the attendees. These may include the tenure and promotion process, launching a research program, professionalism, proposal writing, among other topics. There is no cost to attend this workshop

We are very pleased to announce that the workshop will have two distinguished keynote speakers:

Nora S. Newcombe, Ph.D. is the James H. Glackin Distinguished Faculty Fellow and Professor of Psychology at Temple University. Dr. Newcombe is the PI of the Spatial Intelligence and Learning Center (SILC), headquartered at Temple and involving Northwestern, the University of Chicago and the University of Pennsylvania as primary partners. Dr. Newcombe was educated at Antioch College, where she graduated with a major in psychology in 1972; and at Harvard University, where she received her Ph.D. in Psychology and Social Relations in 1976. She taught previously at Penn State University.

A nationally recognized expert on cognitive development, Dr. Newcombe’s research has focused on spatial development and the development of episodic and autobiographical memory. Her work has been federally funded by NICHD and the National Science Foundation for over 30 years. She is the author of numerous scholarly chapters and articles on aspects of cognitive development, and the author or editor of five books, including Making Space: The Development of Spatial Representation and Reasoning (with Janellen Huttenlocher) published by the MIT Press in 2000.

Tammy Clegg, Ph.D. is an assistant professor in the College of Education with a joint appointment in the College of Information Studies at the University of Maryland. She received her PhD in Computer Science at Georgia Tech in 2010 and her Bachelor of Science in Computer Science from North Carolina State University in 2002. From 2010-2012 Tamara was a postdoctoral fellow at the University of Maryland with the Computing Innovations Fellows program. Her work focuses on developing technology to support life-relevant learning environments where children engage in science in the context of achieving goals relevant to their lives. Kitchen Chemistry is the first life-relevant learning environment she designed along with colleagues at Georgia Tech. In Kitchen Chemistry, middle-school children learn and use science inquiry to make and perfect dishes. Clegg uses participatory design with children to design these new technologies. Her work currently includes creating new life-relevant learning environments (e.g., Sports Physics, Backyard Biology) to understand how identity development happens across these environments. From this analysis, she aims to draw out design guidelines for life-relevant learning activities and technology in various contexts (e.g., sports, gardening).

About LearnLab

LearnLab is funded by the National Science Foundation (award number SBE-0836012). Our center leverages cognitive theory and computational modeling to identify the instructional conditions that cause robust student learning. Our researchers study robust learning by conducting in vivo experiments in math, science and language courses. We also support collaborative primary and secondary analysis of learning data through our open data repository LearnLab DataShop, which provides data import and export features as well as advanced visualization, statistical, and data mining tools.

To learn more about our cognitive science theoretical framework, read our Knowledge-Learning-Instruction Framework.

The results of our research are collected in our theoretical wiki which currently has over 400 pages. It also includes a list of principles of learning which are supported by learning science research. The wiki is open and freely editable, and we invite you to learn more and contribute.

Application Process

Applicants should email their CV, this demographic form, a proposed presentation title and abstract, and a brief statement describing their research interests to Jo Bodnar (jobodnar@cs.cmu.edu) by May 5, 2013. Please use the subject Application for LearnLab Summer Workshop 2013. Upon acceptance, we will let you know if you have been selected for a talk or poster presentation.

Costs

There is no registration fee for this workshop. However, attendance is limited so early applications are encouraged. Scholarships for travel are available. Scholarships will be awarded based on your application, including your research interests, future plans, and optional recommendation letter.

Visiting Indiana University this week

I’m visiting Indiana University this week, and giving two talks.  If any readers are in the Bloomington area, I hope you can stop by!

9:30 am Jan 29
Colloquium
Education 2140

Title: Improving Success in Learning Computer Science Using Lessons from Learning Sciences

Abstract: Learning computer science is difficult, with multiple international studies demonstrating little progress. We still understand too little about the cognitive difficulties of learning programming, but we do know that we can improve success by drawing on lessons from across learning sciences. In this talk, I will describe three examples, where we improve success in learning computer science through application of lessons and models from the learning sciences. We increased the retention of non-CS majors in a required CS course by increasing the relevance of the course (informed by Eccles’ model of achievement-related choices), though we are limited in how far we can go because legitimate peripheral participation is less relevant. We improved opportunities to learn in a collaborative forum by drawing on lessons from anchored instruction, but were eventually defeated by student perceptions of culture. We have improved learning and transfer of knowledge about programming by using subgoal labeling to promote self-explanations.

9 am Thursday Jan 31
SoIC Colloquium Series
IMU State Room East

Title: Three Lessons in Teaching Computing to Everyone

Abstract:  My colleagues and I have been studying how to teach computer science, to CS majors, to non-CS undergraduates, and to adult professionals.  In this talk, I’ll talk about some of what we’ve learned, organized around three lessons.  Lesson #1: We typically teach computer science too abstractly, and by teaching it in a context (e.g., media, robots, Nintendo GameBoys, Photoshop), we can dramatically improve success (retention and learning) for both traditional and non-traditional CS learners. Lesson #2: Collaboration can create opportunities for learning, but classroom culture (e.g., competition) trumps technology (Wikis).  Lesson #3: Our greatest challenge in computer science education is improving teaching, and that will require changes in high schools, in public policy, and in universities.

What are the cognitive skills needed for model-building?

Mylène is describing in the below blog post about how she’s helping her students develop a set of cognitive skills (including a growth mindset) to help them build models.  What I found fascinating in her post were the implicit points, obvious to her, about what the students didn’t know.  One student said, “I wish someone had told me this a long time ago.”  What are the cognitive skills necessary to enable people to build models, or program?  Causal thinking is absolutely critical, of course. What else is necessary that we haven’t identified?  We need to check if students have those skills, or if we need to teach them explicitly.

Last year I found out in February that my students couldn’t consistently distinguish between a cause and a definition, and trying to promote that distinction while they were overloaded with circuit theory was just too much.  So this year I created a unit called “Thinking Like a Technician,” in which I introduced the thinking skills we would use in the context of everyday examples.

via Growth-Mindset Resource Could Support Model-Building « Shifting Phases.

Is learning a programming language like becoming bilingual?

Fascinating question! Bilingual people have some additional executive control. Does learning a programming language give a similar benefit in executive control? The study described below is suggestive but not conclusive. If we could find evidence for it, it would be another benefit of learning to program.

If computer programming languages are languages, then people who spoke one language and could programme to a high standard should be bilingual. Research has suggested that bilingual people perform faster than monolingual people at tasks requiring executive control – that is, tasks involving the ability to pay attention to important information and ignore irrelevant information (for a review of the “robust” evidence for this, see Hilchey & Klein, 2011). So, I set out to find out whether computer programmers were better at these tasks too. It is thought that the bilingual advantage is the result of the effort involved in keeping two languages separate in the brain and deciding which one to use. I noticed that novice computer programmers have difficulty in controlling “transfer” from English to programming languages (e.g. expecting the command “while” to imply continuous checking; see Soloway and Spohrer, 1989), so it seemed plausible that something similar might occur through the learning of programming languages.

via Evidence suggesting that young computer programmers have “bilingual brains” « thecodingbrain.

Teaching CS in Schools with Meaning: Contexts and problems come first

Richard Hake relates a story from Alan Schoenfeld:

One of the problems on the NAEP [National Assessment of Educational Progress] secondary mathematics exam, which was administered to a stratified sample of 45,000 students nationwide, was the following: An army bus holds 36 soldiers. If 1128 soldiers are being bused to their training site, how many buses are needed?

Seventy percent of the students who took the exam set up the correct long division and performed it correctly. However, the following are the answers those students gave to the question of ‘how many buses are needed?’: 29% said…31 remainder 12; 18% said…31; 23% said…32, which is correct. (30% did not do the computation correctly).

It’s frightening enough that fewer than one-fourth of the students got the right answer. More frightening is that almost one out of three students said that the number of buses needed is ‘31 remainder 12’.

The problem that Hake and Schoenfeld are both pointing out is that we teach mathematics (and much else in our curriculum) completely divorced from the contexts in which the mathematics make sense. The children taking the NAEP knew how to do the mathematics, but not why, and not nearly enough about how the mathematics helps to solve a problem.  They knew mathematics, but now what it was for.

Hake relates this story in an article about Louis Paul Benezet, an educator who ran a radical experiment in the 1930′s. Benezet saw how mindlessly young children were performing mathematics, so he made a dramatic change: Almost entirely remove mathematics from grades 1-5. Start teaching mathematics in grade 6, with a focus on problem-solving (e.g., start from estimation, so that you have a sense of when an answer is reasonable). Sixth graders can understand the problems for which one should use mathematics.  The point is not to introduce the solution, until students understood the problem.  Remarkably, the experimental 6th graders completely caught up in just four months to the 6th graders who had had mathematics all five previous years.

The experiment was radical then, and as far as I know, has not been replicated — even though evaluations suggest it worked well.  It runs against our intuition about curriculum.  Mathematics is important, right?  We should do more of it, and as early as possible.  How could you remove any of it?  Benezet argued that, instead, young children should do more reading and writing, saving the mathematics for when it made sense.

Hake uses Benezet (and the evaluation of Benezet’s approach by Berman) to argue for a similar radical approach to physics education — teaching some things to kids to build up intuition, but with a focus on using physics to solve problems, and introducing the problems only when the students can understand them. There are lessons here for computing education, too.

• First, problems and contexts always come first! Teaching a FOR loop and arrays before teaching a problem in which they are useful just leads to rote learning, brittle knowledge which can’t be applied anywhere, let alone transferred.
• Second, the answer to the question “What should be removed from our overly-packed curriculum to squeeze computer science in?” may be “Get rid of the overly-packed curriculum.”  There may be things that we’re teaching at the wrong time, in the wrong way, which really is just a waste of everyone’s time.
• Finally, just how young should we be teaching programming? Several people sent me the link to the report about Estonia teaching all first graders to program (quoted and linked below). Sure, you can teach first graders to program — but will they understand why they’re programming? What problems will first graders recognize as problems for which programming is a solution?

I do applaud the national will in Estonia to value computing education, but I do wonder if teaching programming so young leads to rote learning and the idea that “31 remainder 12″ is a reasonable number of buses.

We’re reading today that Estonia is implementing a new education program that will have 100 percent of publicly educated students learning to write code.

Called ProgeTiiger, the new initiative aims to turn children from avid consumers of technology (which they naturally are; try giving a 5-year-old an iPad sometime) into developers of technology (which they are not; see downward-spiraling computer science university degree program enrollment stats).

ProgreTiiger education will start with students in the first grade, which starts around the age of 7 or 8 for Estonians. The compsci education will continue through a student’s final years of public school, around age 16. Teachers are being trained on the new skills, and private sector IT companies are also getting involved, which makes sense, given that these entities will likely end up being the long-term beneficiaries of a technologically literate populace.

via Guess who’s winning the brains race, with 100% of first graders learning to code? | VentureBeat.

Experience drives learning: Implications for CS Ed

I taught educational technology in the Spring, and it gave me a chance to re-read classic texts (I still love Cognitive Apprenticeship) and reflect on some of the key principles of learning sciences.  One of these is that all learning is built on existing knowledge — Piagetian assimilation and accommodation are still the main two learning mechanisms that we know.  That’s why culture matters, and past experience matters.

The piece linked below from NYTimes highlights how different that prior experience can be, even with students attending the same classroom, and how those different experiences lead to different learning outcomes.

I wonder about the implications for CS Ed.  What are the key experiences that lead students to have the prior knowledge to succeed in CS1?  If a student has never built a spreadsheet with formulas, then that student may not have the same understanding of specifying instructions for another agent and for using a formal notation to be interpreted by machine, compared to a student who has.  A student who has never used Photoshop or looked at a color chooser may have a harder time understanding hierarchy of data representations (e.g., red, green, and blue numbers inside a pixel, which is arranged in two dimensions to make up a picture).  Studies in the past have looked at background experiences like how much mathematics a student has had.  With the pervasiveness of computing technology today, we might be able to look at more “near transfer” kinds of activities.

When a new shipment of books arrives, Rhonda Levy, the principal, frets. Reading with comprehension assumes a shared prior knowledge, and cars are not the only gap at P.S. 142. Many of the children have never been to a zoo or to New Jersey. Some think the emergency room of New York Downtown Hospital is the doctor’s office.

The solution of the education establishment is to push young children to decode and read sooner, but Ms. Levy is taking a different tack. Working with Renée Dinnerstein, an early childhood specialist, she has made real life experiences the center of academic lessons, in hopes of improving reading and math skills by broadening children’s frames of reference.

via For Poorer Students, an Attempt to Let New Experiences Guide Learning – NYTimes.com.

Khan Academy offers kind-of-scaffolded computer science learning: Doing away with the teacher

With a bold claim, “Khan Academy Launches the Future of Computer Science Education,” TechCrunch described Khan’s new foray into computer science.  They’ve had CS videos in the past, but now they have a powerful text editor in which students can edit JavaScript, or manipulate variables like in Bret Victor’s cool demo.

The TechCrunch article actually cites research (see below), a paper by Cindy Hmelo.  Cindy’s paper is actually on problem-based learning, but it does describe scaffolding — as defined in a Hmelo & Guzdial paper from 1996! How about that!

What I see in the Khan Academy offering is one of the kinds of scaffolding that Cindy and I talked about.  Scaffolding is an idea (first defined by Wood, Bruner, and Ross) which does involve letting students explore, but under the guidance of a tutor.  A teacher in scaffolding doesn’t “point out novel ways of accomplishing the task.”  Instead, the teacher models the process for the student, coaches the student while they’re doing it, and gets the student to explain what they’re doing.  A key part of scaffolding is that it fades — the student gets different kinds of support at different times, and the support decreases as the student gets more expert.  I built a form of adaptable scaffolding in my 1993 dissertation project, Emile, which supported students building physics simulations in HyperTalk.  Yes, students using Emile could click on variables and fill in their values without directly editing the code, but there was also process guidance (“First, identify your goals; next, find your components in the Library”) and prompts to get students to reflect on what they’re doing.  And the scaffolding could be turned on or off, depending on student expertise.

I wouldn’t really call what Khan Academy has “scaffolding,” at least, not the way that Cindy and I defined it, nor in a way that I find compatible with Wood, Bruner, and Ross’s original definition.  There’s not really a tutor or a teacher.  There are videos as I learned from this blog post, and later found for myself.  The intro video (currently available on the main Khan Academy page) says that students should just “intuit” how the code works.  Really?  There’s a lot more of this belief that students should just teach themselves what code does.  The “scaffolding” in Khan Academy has no kind of process modeling or guidance, nothing to explain to students what they’re doing or why, nothing to encourage them to explain it to themselves.

It is a very cool text editor.  But it’s a text editor.  I don’t see it as a revolution in computer science education — not yet, anyway.  Now, maybe it’s way of supporting “collaborative floundering” which has been suggested to be even more powerful than scaffolding as a learning activity.  Maybe they’re right, and this will be the hook to get thousands of adolescents interested in programming.  (I wonder if they tested with any adolescents before they released?)  Khan has a good track record for attracting attention — I look forward to seeing where this goes.

The heart of the design places a simplified, interactive text editor that sits adjacent to the code’s drawing output, which updates in real time as students explore how different variables and numbers change the size, shapes, and colors of their new creation. An optional video guides students through the lesson, step-by-step, and, most importantly, can be paused at any point so that they can tinker with the drawing as curiosity and confusion arise during the process.

This part is key: learning is contextual and idiosyncratic; students better absorb new material if they can learn at their own pace and see the result of different options in realtime.

The pedagogy fits squarely into what educators called “scaffolded problem-based learning” [PDF]; students solve real-life problems and are encouraged to explore, but are guided by a teacher along the way, who can point out novel ways of accomplishing the task. Scaffolded learning acknowledges that real-life problems always have more than one path to a solution, that students learn best by doing, and that curiosity should drive exploration. This last point is perhaps the most important, since one of the primary barriers to boosting science-related college majors is a lack of interest.

via Khan Academy Launches The Future of Computer Science Education | TechCrunch.

National Academies Report Defines ’21st-Century Skills’

I looked up this report, expecting to see something about computation as a ’21st-century skill.’ The report is not what I expected, and probably more valuable than what I was looking for.  Rather than focus on which content is most valuable (which leads us to issues like the current debate of whether we ought to teach algebra anymore), the panel emphasized “nonacademic skills,” e.g., the ability to manage your time so that you can graduate and intra-personal skills.  I also appreciated how careful the panel was about transfer, mentioning that we do know how to teach for transfer within a domain, but not between domains.

Stanford University education professor Linda Darling-Hammond, who was not part of the report committee, said developing common definitions of 21st-century skills is critical to current education policy discussions, such as those going on around the Common Core State Standards. She was pleased with the report’s recommendation to focus more research and resources on nonacademic skills. “Those are the things that determine whether you make it through college, as much as your GPA or your skill level when you start college,” she said. “We have tended to de-emphasize those skills in an era in which we are focusing almost exclusively on testing, and a narrow area of testing.”

The skill that may be the trickiest to teach and test may be the one that underlies and connects skills in all three areas: a student’s ability to transfer and apply existing knowledge to a problem in a new context. “Transfer is the sort of Holy Grail in this whole thing,” Mr. Pellegrino said. “We’d like to believe we can create Renaissance men who are experts in a wide array of disciplines and can blithely transfer skills from one to the other, but it just doesn’t happen that way.”

via Education Week: Panel of Scholars Define ’21st-Century Skills’.

Science Education Research: Misconceptions are suppressed, not supplanted

Very interesting report from Neil Brown. Here’s the question I’d like to know: So what are students intuitions about computing as they enter the classroom? Are they suppressed or supplanted through instruction? My guess is that it’s different for computing than for science. We live our lives for many years, 24 hours a day, in the real world before we enter school. That’s a lot of time to invent science hypotheses about the world. Not so much for computing. While we may increasing live our lives in a computing world, it’s a constructed, designed world — a world in which the computer science is explicitly hidden. I bet that students only make up theories about computing in times of break down, when they have to invent a theory to explain what went wrong. How often does that happen? What theories do they develop?

The paper title here says it all: Scientific knowledge suppresses but does not supplant earlier intuitions. A consistent theme across the research described in this post is that when you are explaining science to pupils, you are not adding totally new knowledge, in the way that you might when explaining a lesser-known historical event. When you explain forces to someone, they will already have an idea about the way the world works (drop something, and it falls to the ground), so you are trying to adjust and correct their existing understanding (falling is actually due to gravity), not start from scratch. The paper suggests that the old knowledge is generally not replaced, but merely suppressed, meaning people carry their original misconceptions with them forever-after.

via Science Education Research | Academic Computing.

A Report on Worked Examples and Self-Explanations in Media Computation

I should give you a little report on how my worked examples/self-explanation intervention worked in my Media Computation class.  I have nothing close to real data, and you shouldn’t believe me if I offered any.  This is a rarified class: 22 students, meeting four days a week for 90 minutes, plus office hours for 90 minutes twice each week (that most of the students have come to), and the teacher (who is the author of the textbook) attends breakfast and dinner with the students.  I think it would be hard to get more student-teacher interaction than in this model.

That said, I would definitely do it again.  I was quite surprised at how seriously the students took the task of explaining these programs!  In retrospect, I shouldn’t have been surprised.  In most classes, aren’t students asked to analyze and explain situations, even asked to make sense of some text?  That’s exactly what I asked these students to do, and they really worked at it.  I had students coming to office hours to ask about their assigned programs, so that they could write up their one paragraph of explanation. There were things that I had to teach them about this process, e.g., teaching them to try a program with different data sets, to make sure that the odd result they got wasn’t an anomaly.  I gave them feedback (every single student, on every single program) about the quality of their explanations, and the explanations definitely got better over time.

The real benefit was that they were trying to understand some relatively complicated code before it was their own code that they were trying to understand (while also designing and debugging it, all before a deadline).   With the worked examples tasks, they were just trying to understand.  There clearly was a reduction in cognitive load.  Variations on the below program had lots of students coming to see me — combining sounds at different rates was a challenging idea, but students did a good job of getting a grasp on it:

def modifysound2(sound):
 retsound = makeEmptySound(2*getLength(sound))
 newsound = makeSound(getMediaPath("bassoon-c4.wav"))
 trgi = 0
 nsi = 0
 for i in range(getLength(sound)):
 value = getSampleValueAt(sound,i)
 if nsi < getLength(newsound):
 nsvalue = getSampleValueAt(newsound,int(nsi))
 else:
 nsvalue = 0
 setSampleValueAt(retsound,trgi,value+nsvalue)
 trgi = trgi + 1
 nsi = nsi + 0.5
 return resound

Because there four labs (that just involved explaining programs) and two homework’s (that involved typing in, executing, and explaining programs), the first real programming assignment was the collage assignment.  Everybody did it.  Everybody turned in a working program.  And some of these were huge.  This one (by Savannah Andersen) was over 100 lines of code:

This one, by Julianne Burch, is over 200 lines of code.  I’m posting shrunk versions here: Julianne’s is about 4000 pixels across, representing the travel portion of this study abroad program.

I suspect that the worked examples and self-explanations gave the students more confidence than they normally have when facing their first programs.  It’s unusual in my experience for students to be willing to write 50-200 lines of working code for their first programming assignment.

But some of these students were also getting it.  A few of my students realized that they could make their collages more easily by using a copy() method to reduce the complication of composing pictures.  I did prompt them to do that, and a few did — most just went with hard-coded FOR loops, because that was easier for them to understand.  When I described how to do that, one student asked, “Aren’t you just naming some of those lines of code?” Yes! Nice way to start thinking about functions and abstract: it’s about naming chunks of code.  One of my students, without prompting, also decided to create a copy() method for her sound collage.  They’re starting to grapple with abstraction.  Given that this is the third week of class, when none of them had any previous programming experience (all my students are liberal arts and management students), I think that they’re doing quite well at moving from notation into abstraction.

They’re working on their first midterm exam now, a take-home exam (to save classroom time.)  I think it’s significantly challenging for a first exam, but it doesn’t have much coding.  It has a lot of analysis of code, because that’s one of the key learning objectives.  I want them to be able to look at a piece of code and predict its behavior, to trace (if necessary) what’s going on.  For me, that’s a more important outcome from a first course than being able to write a lot of code.

Inventing a Worked Examples and Self-Explanation Method for CS Courses

I sent this idea to the mediacomp-teach mailing list, and got a positive response.  I thought I’d share it here, too.

I’m trying a worked examples + self-explanations approach in my Media Computation Python class that started Monday (first time I’ve taught it in seven years!) and in my “Computational Freakonomics” class (first time I’ve taught it in six years).  Whether you’re interested in this method or not, you might like to use the resource that I’ve created.

As I mentioned here, I’m fascinated by the research on worked examples and on self-explanations. The idea behind worked examples is that we ought to have students see more fully worked out examples, with some motivation to actually study them. The idea behind self-explanations is that learning and retention is improved when students explain something to themselves (or others), in their own words.  Pete Pirolli did studies where he had students use worked examples to study computer science (explicitly, recursion), and with Mimi Recker, prompted CS students to self-explain then studied the effect.  In their paper, Pirolli and Recker found:

“Improvement in skill acquisition is also strongly related to the generation of explanations connecting the example material to the abstract terms introduced in the text, the generation of explanations that focus on the novel concepts, and spending more time in planning solutions to novel task components. We also found that self-explanation has diminishing returns. “

Here’s the critical idea: Students (especially novices) need to see more examples, and they need to try to explain them.  This what I’m doing at key points in the class:

• Each team of two students gets one worked example in class. They have to type it in (to make sure that they notice all the details) and explain it to themselves – what does it do? how does it work?
• Each team then explains it to the teams on either side of them.
• At the end of the class, each individual takes one worked example, and does the process themselves: Types it in, pastes it into a Word document (with an example of the output), and explains what the program does.  I very explicitly encourage them to do with this others, and talk about their programs with one another.  I want students to see many examples, and talk about them.

Sure, our book has many examples in it, but how many students actually look at all those examples? How many type them in and try them?  Explain to themselves?

I’m doing this at four points in the MediaComp class: for images with getPixels, images with coordinates, sounds, and text and lists. For my CompFreak class, students are supposed to have had some CS1, and most of them have seen Python at least once, so I’m only doing this at the beginning of the class, and only on text and lists.  There are 22 students in my MediaComp class, so I needed 11 examples in class, then 22 examples one-for-each-person. Round it off to 35 examples. That’s 140 working examples. A lot of them vary in small ways — that’s on purpose. I wanted two teams to say, “I think our program is doing about the same thing as yours — what’s different?”

I did discover some effects that surprised me. For example, try this:

def changesound(sound):
   for sample in getSamples(sound):
     value = getSampleValue(sample)
     if value > 0:
       setSampleValue(sample, 4 * value)
     if value <= 0:
       setSampleValue(sample,0)

Turns out if you zero out all the negative samples, you can still hear the sound pretty clearly.  I wouldn’t have guessed this.

Whether you want to try this example-heavy approach or not, you might find useful all these examples.  I’ve put all 140 examples on the teacher MediaComp sharing site (http://home.cc.gatech.edu/mediacomp/9 – email me if you want the key phrase and don’t have it). I started creating these in Word, but that was tedious to format well. I switched to LaTeX, because that nicely formatted the Python without much effort on my part. I’ve uploaded both the PDF and the LaTeX, since the LaTeX provides easy copy-paste text.

My CompFreak students are doing their assignment now (due tonight), and we just did it for the first time in the MediaComp class today (the take-home portion due in two days).  I was pleased with the feedback.  I got lots of questions about details that students don’t normally ask about at the second lecture (e.g., “makeColor is doing something different than setRed, setGreen, and setBlue differently? What’s the difference between colors and pixels?”).  My hope is that, when they start writing their own code next week, they won’t be stymied by stupid syntax errors, because they will have struggled with many of the obvious ones while working with complete code.  I’m also hoping that they’ll be more capable in understanding (and thus, debugging) their own code.  Most fun: I had to throw the students out of class today.  Class ended at 4:10, and we had a faculty meeting at 4:30.  Students stayed on, typing in their code, looking at each others’ effects.  At 4:25, I shooed them off.

I am offering extra credit for making some significant change (e.g., not just changing variable names) to the example program, and turning that in, too (with explanation and example).  What I didn’t expect is that they’re relating the changes to code we’ve talked about, like in this comment from a student that just got turned in:

“I realized I made an error in my earlier picture so I went back and fixed it. I also added in another extra credit picture. I made a negative of the photo. It looks pretty cool!”

It’s interesting to me that she explicitly decided to “make a negative” (and integrated the code to do it) rather than simply adding/changing a constant somewhere to get the extra credit cheaply.

All my MediaComp students are Business and Liberal Arts students (and is 75% female — while CompFreak is 1 female and 9 males).  I got a message from one of the MediaComp students yesterday, asking about some detail of the class, where she added: “We all were pleasantly surprised to have enjoyed class yesterday!”  I take the phrase “pleasantly surprised” to mean that the expectations are set pretty low.

Defining: What does it mean to understand computing?

In the About page for this blog, I wrote, “Computing Education Research is about how people come to understanding computing, and how we can facilitate that understanding.”  Juha Sorva’s dissertation (now available!) helped me come to an understanding of what it means to “understand computing.”  I describe a fairly technical (in terms of cognitive and learning sciences) definition, which basically is Juha’s.  I end with some concrete pedagogical recommendations that are implied by this definition.

A Notional Machine:  Benedict DuBoulay wrote in the 1980′s about a “notional machine,” that is, an abstraction of the computer that one can use for thinking about what a computer can and will do.  Juha writes:

Du Boulay was probably the first to use the term notional machine for “the general properties of the machine that one is learning to control” as one learns programming. A notional machine is an idealized computer “whose properties are implied by the constructs in the programming language employed” but which can also be made explicit in teaching (du Boulay et al., 1981; du Boulay, 1986).

The notional machine is how to think about what the computer is doing.  It doesn’t have to be about the CPU at all. Lisp and Smalltalk each have small, well-defined notional machines — there is a specific definition of what happens when the program executes, in terms of application of S-expressions (Lisp) and in terms of message sending to instances of classes (Smalltalk).  C has a different notional machine, which isn’t at all like Lisp’s or Smalltalk’s.  C’s notional machine is closer to the notional machine of the CPU itself, but is still a step above the CPU itself (e.g., there are no assignment statements or types in assembly language). Java has a complicated notional machine, that involves both object-oriented semantics and bit-level semantics.

A notional machine is not a mental representation.  Rather, it’s a learning objective.  I suggest that understanding a realistic notional machine is implicitly a goal of computational thinking.  We want students to understand what a computer can do, what a human can do, and why that’s different.  For example, a computer can easily compare two numbers, can compare two strings with only slightly more effort, and has to be provided with an algorithm (that is unlikely to work like the human eye) to compare two images.  I’m saying “computer” here, but what I really mean is, “a notional machine.”  Finding a route from one place to another is easy for Google Maps or my GPS, but it requires programming for a notional machine to be able to find a route along a graph.  Counting the number of steps from the top of the tree to the furthest leaf is easy for us, but hard for novices to put in an algorithm.  While it’s probably not important for everyone to learn that algorithm, it’s important for everyone to understand why we need algorithms like that — to understand that computers have different operations (notional machines) than people.  If we want people to understand why we need algorithms, and why some things are harder for computers than humans, we want people to understand a notional machine.

Mental Models:  A mental model is a personal representation of some aspect of the world.  A mental model is executable (“runnable” in Don Norman’s terms) and allows us to make predictions.  When we turn on and off a switch, we predict that the light will go on and off.  Because you were able to read that sentence and know what I meant, you have a mental model of a light which has a switch. You can predict how it works.  A mental model is absolutely necessary to be able to debug a program: If you have to have a working expectation of what the program was supposed to do, and how it was supposed to get there, so that you can compare what it’s actually doing to that expectation.

So now I can offer a definition, based on Juha’s thesis:

To understand computing is to have a robust mental model of a notional machine.

My absolutely favorite part of Juha’s thesis is his Chapter 5, where he describes what we know about how mental models are developed.  I’ve already passed on the PDF of that chapter to my colleagues and student here at Georgia Tech.  He found some fascinating literature about the stages of mental model development, about how mental models can go wrong (it’s really hard to fix a flawed mental model!), and about the necessary pieces of a good mental model.  DeKleer and Brown provide a description of mental models in terms of sub-models, and tell us what principles are necessary for “robust” mental models.  The first and most important principle is this one (from Juha Sorva’s thesis, page 55):

• The no-function-in-structure principle: the rules that specify the behavior of a system component are context free. That is, they are completely independent of how the overall system functions. For instance, the rules that describe how a switch in an electric circuit works must not refer, not even implicitly, to the function of the whole circuit. This is the most central of the principles that a robust model must follow.

When we think about a switch, we know that it opens and closes a circuit. A switch might turn on and off a light. That would be one function for the switch. A switch might turn on and off a fan. That’s another function for a switch. We know what a switch does, completely decontextualized from any particular role or function.  Thus, a robust mental model of a notional machine means that you can talk about what a computer can do, completely apart from what a computer is doing in any particular role or function.

A robust mental model of a notional machine thus includes an understanding of how an IF or WHILE or FOR statement works, or what happens when you call a method on an object in Java (including searching up the class hierarchy), or how types do – completely independently of any given program.  If you don’t know the pieces separately, you can’t make predictions, or understand how they work a particular function in a particular program.

It is completely okay to have a mental model that is incomplete.  Most people who use scissors don’t think about them as levers, but if you know physics or mechanical engineering, you understand different sub-models that you can use to inform your mental model of how scissors work.  You don’t even have to have a complete mental model of the notional machine of your language.  If you don’t have to deal with casting to different types, then you don’t have to know it.  Your mental model doesn’t have to encompass the notional machine.  You just don’t want your mental model to be wrong.  What you know should be right, because it’s so hard to change a mental model later.

These observations lead me to a pedagogical prediction:

Most people cannot develop a robust mental model of a notional machine without a language.

Absolutely, some people can understand what a computer can do without having a language given to them.  Turing came up with his machine, without anyone telling him what the operations of the machine could do.  But very few of us are Turings.  For most people, having a name (or a diagram — visual notations are also languages) for an operation (or sub-model, in DeKleer and Brown terms) makes it easier for us to talk about it, to reference it, to see it in the context of a given function (or program).

I’m talking about programming languages here in a very different way than how they normally enter into our conversation.  In much of the computational thinking discussion, programming is yet another thing to learn.  It’s a complexity, an additional challenge.  Here, I’m talking about languages as a notation which makes it easier to understand computing, to achieve computational thinking.  Maybe there isn’t yet a language that achieves these goals.

Here’s another pedagogical recommendation that Juha’s thesis has me thinking about:

We need to discuss both structure and function in our computing classes.

I suspect that most of the time when I describe “x = x + 1″ in my classes, I say, “increment x.”  But that’s the function.  Structurally, that’s an assignment statement.  Do I make sure that I emphasize both aspects in my classes?  They need both, and to have a robust mental model, they probably need the structure emphasized more than the function.

We see that distinction between structure and function a lot in Juha’s thesis.  Juha not only does this amazing literature review, but he then does three studies of students using UUhistle.  UUhistle works for many students, but Juha also explores when it didn’t — which may be more interesting, from a research perspective.  A common theme in his studies is that some students didn’t really connect the visualization to the code.  They talk about these “boxes” and do random walks poking at graphics.  As he describes in one observation session (which I’m leaving unedited, because I enjoyed the honesty of Juha’s transcripts):

What Juha describes isn’t unique to program visualization systems. I suspect that all of us have seen or heard something pretty similar to the above, but with text instead of graphics.  Students do “random walks” of code all the time.  Juha talks a good bit about how to help his students better understand how UUhistle graphical representations map to code and to the notional machine.

Juha gives us a conceptual language to think about this with.  The boxes and “incomprehensible things” are structures that must be understood on their own terms, in order to develop robust mental models, and understood in terms of their function and role in a program. That’s a challenge for us as educators.

So here’s the full definition:  Computing education research is about understanding how people develop robust models of notional machines, and how we can help them achieve those mental models.