What’s generally good for you vs what meets a need: Balancing explicit instruction vs problem/project-based learning in computer science classes

Lauren Margulieux has posted another of her exceptionally interesting journal article summaries (see post here). Her post summarizes recent article asking which is more effective: Direct instruction or learning through problem-solving-first (like in project-based learning or problem-based learning — or just about any introductory computer science course in any school anywhere)? Direct instruction won by a wide margin.

Lauren points out that there are lots of conditions when problem-solving-first might make sense. In more advanced classes where students have lots of expertise, we should use a different teaching strategy than what we use in introductory classes. When the subject matter isn’t cognitively complex (e.g., memorizing vocabulary words), there is advantages to having the students try to figure it out themselves first. Neither of these conditions are true for introductory computer science.

This is an on-going discussion in computing education. Felienne Hermans had a keynote at the 2019 RStudio Conference where she made an argument for explicit direct instruction (see link here). I made an argument for direct instruction in Blog@CACM last November (see post here). Back in 2017, I recommended balancing direct instruction and projects (see post here), because projects are clearly more motivating and authentic for computer science students, while the literature suggest that direct instruction leads to better learning — even of problem-solving skills.

Lately, I’ve been thinking about this question with a health metaphor. Let me try it here:

Everybody should exercise, right? Exercise provides a wide variety of benefits (listed in a fascinating blog post from Freakonomics from this June), including cardiovascular improvements, better aging, better sleep, and less stress. But if you have a heart problem, you’re going to get treatment for that, right? If you’re having high cholesterol, you should continue to exercise (or even increase it), but you might also be prescribed a statin.  If you have a specific need (like a vitamin deficiency), you address that need.

Students in computing should work on projects. It’s authentic, it’s motivating, and there are likely a wide range of benefits. But if you want to gain specific skills, e.g., you want to achieve learning objectives, teach those directly. Don’t just assign a big project and hope that they learn the right things there. If you want to see specific improvement in specific areas, teach those. So sure, assign projects — but in balance. Meet the students’ needs AND give them opportunities to practice project skills.

And when you teach explicitly: Always, ALWAYS, ALWAYS use active learning techniques like peer instruction. It’s simply unethical to lecture without active learning.

September 16, 2019 at 7:00 am 1 comment

Come talk about the Role of Authentic STEM Learning Experiences in Developing Interest and Competencies for Technology and Computing #STEMforCompTech

I’m on a National Academies committee to write a report about the role of authentic STEM learning experiences in promoting interest and ability in computing.  We’re having an open meeting/workshop (I don’t really know what it’s about yet) in November in DC. Visit this link for more information.

Save_The_Date__November_4th_Workshop- Role_of_Authentic_STEM_Learning_Experiences_in_Developing_Interest_and_Competencies_for_Technology_and_Computing

September 13, 2019 at 10:00 am Leave a comment

Where a Notional Machine Doesn’t Help: JavaScript and the DOM

At the Dagstuhl Seminar on Notional Machines and Programming Language Semantics, I came to a new understanding about where notional machines are not useful and another kind of support is needed.

I joined a breakout group on “Notional Machines for everything else.” There had been discussions about notional machines for popular programming languages for Scratch and Python, and one breakout group was formed around that goal. This group was about exploring where else notional machines could be useful, like trying to understand machine learning, generating proofs, and JavaScript for Web development. Joe Politz was our group leader.

After the first round of discussion, we all decided to focus on JavaScript. We had some serious experts on JavaScript and the DOM in our group, like Shriram Krishnamurthi and Titus Barik. The discussion was amazing! I was learning so much, and I took pages and pages of notes. Everyone noticed my feverish note-taking, so I got elected to report back to the group. That’s this blog post — it’s the work of the whole group (not just me). I just happened to be the guy who made the slides.

We had already come to the realization that there isn’t just one notional machine per language. A notional machine is about helping students understand a computational process, and there can be lots of processes in a given language. So we picked a specific scenario that we were aiming to explain: You (as a student) want to turn the background yellow when the button is clicked.

But the student makes a mistake.

And I came to realize that, without a lot of support:

The problem is that window.bg = "yellow" isn’t wrong. Because there isn’t a previously defined bg property, this assignment simply creates a property bg for the window object. No error. It just doesn’t do what the students wanted. How does the student figure out that the desired property is backgroundColor? Get a list of all bindings on window? There are hundreds or even thousands of them. How do you find backgroundColor among all of those?

The breakout group started listing on the blackboard the things we might need to explain to students to help them understand when went wrong or what might go wrong with clicking on a button to turn the background yellow. It was a long list.

You probably can’t read all of those, so I’ll list a few of them here:

  • What happens if you have two DOM elements with the same ID?
  • Where did window as an object get defined at all?
  • If you have event handlers defined on an object, and you delete the object, what happens to the event handler?
  • What happens if objects higher up in the DOM modify the event triggered by the event click?
  • Something I heard students ask: If my JavaScript changed the DOM, how come my HTML file didn’t change?
  • And Many More

I really had no idea just how complicated JavaScript and the DOM were! Andy Ko looked up the JavaScript definition of what == means (see link here). This isn’t the formal semantics. This is meant to be understandable. It’s insanely complicated.

At this point, Ben Shapiro raised a really interesting side question: What’s the cost of JavaScript’s overly-complicated rules? Is there a way of measuring the lost productivity of bad programming language design?

So, what’s the answer?

I realized that the answer is not a notional machine. The problem is that long list Andy found for us. I can teach part of that list as a notional machine, but I can’t teach all of it with a simplified model. Any simplification I create would be insufficient for the complexity of the reality. And even having a notional machine wouldn’t help if a student typed window.bg = "yellow". The student needs IDE’s and other supports to figure out errors that never trigger an error.

The solution is to reduce complexity to make it teachable.  In an earlier talk at Dagstuhl, Ben Shapiro explained how Shriram and Joe and their collaborators did this with Pyret.  In Pyret, they explicitly disallow some things that Python allows but are way too complicated to explain.

Probably the best idea for JavaScript, following Racket’s lead, is to have language levels. We should teach students a strict subset of JavaScript, where the really complicated things are simply disallowed. The goal is to help students to learn a real subset, then grow the subset.  TypeScript offers an alternative model, because it offers a more sane way of doing JavaScript.  For example, TypeScript’s type checking might help figure out the window.bg bug. There are lots of other languages that are more reasonable and compile to JavaScript — but those are avoiding JavaScript entirely.

The very best idea would be to fix JavaScript and the DOM, but it’s probably too late for that.

This working group was useful to me for two reasons. First, I really do have to teach JavaScript and the DOM (again) in January 2020, and now I have a new sense of the challenges and my options. Second, this was a great example of where a notional machine is not the answer to a pedagogical problem.

Thanks to members of the group who reviewed an earlier draft of this summary.  They’re not responsible for where I still didn’t get the details right.

 

September 9, 2019 at 7:55 am 13 comments

How the Cheesecake Factory is like Healthcare and CS Education

Stephen Dubner of the Freakonomics Podcast did an interview with Atul Gawande, author of the Checklist Manifesto. Atul is a proponent of a methodical process — from a behavioral economist perspective, not from an optimization perspective. People make mistakes, and if we’re methodical (like the use of checklists), we’re less likely to make those mistakes. If we can turn our process into a “recipe,” then we will make fewer mistakes and have more reliable results. Here’s a segment of the interview where he argues for that process in healthcare.

DUBNER: Okay, what’s the difference between a typical healthcare system and say, a restaurant chain like the Cheesecake Factory?

GAWANDE: You’re referring to the article I wrote about the Cheesecake Factory…Basically what I was talking about was the idea that, here’s this restaurant chain. And yes, it’s highly caloric, but the Cheesecake Factorys here have as much business as a medium-sized hospital — $100 million in business a year. And they would cook to order every meal people had. And in order to make that happen, they have to run a whole process that they have real cooks, but then they have managers.

I was talking to one of the managers there about how he would make healthcare work. And his answer was, “Here’s what I would do, but of course you guys do this. I would look to see what the best people are doing. I would find a way to turn that into a recipe, make sure everybody else is doing it, and then see how far we improve and try learning again from that.” He said, “You do that, right?” And we don’t. We don’t do that.

Here’s where Gawande’s approach ties to education. Later in the interview:

DUBNER: And do you ever in the middle of, let’s say, a surgery think about, “Oh, here’s what I will be writing about this day?”

GAWANDE: You know, I don’t really; I’m in the flow. One of the things that I love about surgery is: it is, I have to confess, the least stressful thing I do, because at this point I’ve done thousands of the operations I do. Ninety-seven to 98 percent go pretty much as expected, and the 2 percent that don’t, I know the 10 different ways that are most common that they’ll go wrong and I have approaches to it. So it’s kind of freeing in a certain way.

I think about teaching like this. I have been teaching computing since 1980. I actively seek out new teaching methods. Teaching CS is fun for me — because I know lots of ways to do it, I have choices which keeps it interesting. I have mentioned the fascinating work on measuring teacher PCK (Pedagogical Content Knowledge) by checking how accurately teachers can predict how students will get exam questions wrong. I know lots of ways students can get computer science wrong. I still get surprises, but because I’m familiar with how students can get things wrong, I have a starting place for supporting students in constructing stronger conceptions.

We can teach this. The model of expertise that Gawande describes is achievable for CS teachers. We can teach new CS teachers methods that they can choose from. We can teach them common student mental models of programming, how to diagnose weaker understandings, and ways to help them improve their mental models. We can make a checklist of what a CS teacher needs to know and be able to do.

September 2, 2019 at 7:00 am 8 comments

Holding ourselves to a higher standard: “Language-independent” just doesn’t cut it

My CACM blog post this month (see link here) is a retraction of the term “language-independent” in our work on the FCS1 and SCS1:

There is no language independence here. The FCS1 and SCS1 are multi-lingual which is a remarkable achievement. We might also call them pseudcode-based assessments, which is how they can be multi-lingual, but since a pseudocode-based test isn’t necessarily validated across other languages, “multi-lingual” is a stronger claim than “pseudocode-based.” We do not cover all of any of those languages (Java, MATLAB, or Python), but we do cover the subset most often appearing in an introductory CS course.

They are clearly not language independent. In the great design space of programming languages, Java, MATLAB, and Python cluster together pretty closely. There are much more different programming languages than these — I’m sure it’ll take any reader here just a few moments to generate a half-dozen candidates whose learners would score poorly on the FCS1 and SCS1, from Scratch to Haskell to Prolog.

I only vaguely remember the discussion about using the term “language independence” with Allison many years ago.  I remember her asking me if we should worry about the (relatively few) classes that used languages other than Python, MATLAB, and Java.  I think I told her she needed to graduate. I judged from the perspective of what was being published at the SIGCSE Symposium — Python, MATLAB, and Java was “language independent” enough for the paper to be seen as valuable to SIGCSE reviewers. I don’t remember the details, but I’ll accept the blame for the decision to call FCS1 (and SCS1 later) language independent.

That was a long time ago, before the International Computing Education Research Conference (ICER) was invented.  Since then, we have a computing education research community that aims to answer questions about how people learn computing — period. We’re not just about undergraduate introductory computer science classes. Even at the undergraduate level, we should study the classes (no matter how few) doing something different to see what’s powerful and interesting about them. We should explicitly be exploring unusual (even purpose-invented) languages to understand more of the interaction between programming languages and human cognition.  An insightful PPIG paper from Clayton Lewis (see link here) was recently circulated on Twitter (see tweet) that makes great points about the complexity of measuring that interaction:

The PPIG community should be proud that cognitive dimensions analysis emerged from the work of people in its ranks, Thomas Green, Marian Petre, Alan Blackwell, and others. We should be skeptical of calls to replace its use with A-B trials or other quantitative methods that cannot cope with the complexity of the language design landscape. When results of A-B trials and similar studies are presented, we should diplomatically ask for the mechanisms that are involved to be described. Colleagues who present the results of such trials should be prepared to respond to this request, so that the generalizability of their results can be assessed.

We should not be driven by what’s in classrooms today (see previous post making that argument). We should hold ourselves to a higher standard. Our goal is to create a lasting record of exploration and research for a research community.

That’s why it’s past time for this retraction.

 

August 26, 2019 at 7:00 am Leave a comment

Summarizing findings about block-based programming in computing education

As readers of my blog know, I’m interested in alternative modalities and representations for programming. I’m an avid follower of David Weintrop’s work, especially the work comparing blocks and text for programming (e.g., as discussed in this blog post).

David wrote a piece for CACM summarizing some of his studies on block-based programming in computing education. It has just been published in the August issue.  Here’s the link to the piece — I recommend it.

To understand how learners make sense of the block-based modality and understand the scaffolds that novice programmers find useful, I conducted a series of studies in high-school computer science classrooms. As part of this work, I observed novices writing programs in block-based tools and interviewed them about the experience. Through these interviews and a series of surveys, a picture emerged of what the learners themselves identified as being useful about the block-based approach to programming. Students cited features discussed here such as the shape and visual layout of blocks, the ability to browse available commands, and the ease of the drag-and-drop composition interaction. They also cited the language of the blocks themselves, with one student saying “Java is not in English it’s in Java language, and the blocks are in English, it’s easier to understand.” I also surveyed students after working in both block-based and text-based programming environment and they overwhelmingly reported block-based tools as being easier. These findings show that students themselves see block-based tools as useful and shed light as to why this is the case.

August 19, 2019 at 7:00 am Leave a comment

Social studies teachers programming, when high schools choose to teach CS, and new models of cognition and intelligence in programming: An ICER 2019 Preview

My group will be presenting two posters at ICER this year.

  • Bahare Naimipour (Engineering Education Research PhD student at U-Michigan) will be presenting our participatory design session with social studies educators, Helping Social Studies Teachers to Design Learning Experiences Around Data–Participatory design for new teacher-centric programming languages. We had 18 history and economics teachers building data visualizations in either Vega-Lite or JavaScript with Google Charts. Everyone got the starter visualization running and made changes that they wanted in less than 20 minutes. Those who started in Vega-Lite also tried out the JavaScript code, but only about 1/4 of the JS groups moved to Vega-Lite successfully.
  • Miranda Parker (Human-Centered Computing PhD student at Georgia Tech) will be presenting her quantitative model explaining about half of the variance in whether Georgia high schools taught CS in 2016, A Statewide Quantitative Analysis of Computer Science: What Predicts CS in Georgia Public High School. The most important factor was whether the school taught CS the year before, suggesting that overcoming inertia is a big deal — it’s easier to sustain a CS program than start one. She may talk a little about her new qualitative work, where she’s studying four schools as case studies about their factors in choosing to teach CS, or not.

Barbara is co-author on a paper, A Spaced, Interleaved Retrieval Practice Tool that is Motivating and Effective, with Iman Yeckehzaare and Paul Resnick . This is about a spaced practice tool that 32% of the students in an introductory programming course used more than they needed to, and the number of hours of use had a measurable positive effect on the final exam grade.

All of our other papers were rejected this year, but we’re in good company — the accept rate was around 18%. But I do want to talk about a set of papers that will be presented by others at ICER 2019. These are papers that I heard about, then I asked the authors for copies. I’m excited about all three of them.

How Do Students Talk About Intelligence? An Investigation of Motivation, Self-efficacy, and Mindsets in Computer Science by Jamie Gorson and Eleanor O’Rourke (see released version of the paper here)

One of the persistent questions in computing education research is why growth mindset interventions are not always effective (see blog post here). We get hard-to-interpret results. I met Jamie and Nell at the Northwestern Symposium on Computer Science and the Learning Sciences in April (amazing event, see here for more details). Nell worked with Carol Dweck during her graduate studies.

Jamie and Nell found mixed mindsets among the CS students that they studied. Some of the students they studied had growth mindsets about intelligence, but their talk about programming practices showed more fixed mindset characteristics. Other students self-identified as having some of both growth and fixed mindset beliefs.

In particular, some students talked about intelligence in CS in ways that are unproductive when it came to the practice of programming. For example, some students talked about the best programmers as being able to write the whole code in one sitting, or never getting any errors. A more growth mindset approach to programming would be evidenced by talking about building programs in pieces, expecting errors, and improving through effort over time.

This is a really helpful finding. It gives us new hypotheses to explore about why growth mindset interventions haven’t been as successful in CS as in other disciplines. Few disciplines have this strong distinction between their knowledge and their practice as acutely as we do in CS. It’s no wonder that we see these mixed mindsets.

Toward Context-Dependent Models of Productive Knowledge in Programming Cognition, by Brian A. Danielak

I’ve known Brian since he was a PhD student, and have been hoping that he’d start to publish some of his dissertation work. I got to read one chapter of it, and found it amazingly insightful. Brian explained how what we might see as a “random walk” of syntax was actually purposeful and rational behavior. I was excited to hear about this paper, and I enjoyed reading it.

It’s such an unusual paper for ICER! It’s empirical, but has no methods section. A big part of it is connecting to prior literature, but it’s not about a formal literature review.

Brian is making an argument about how we characterize knowledge and student success in CS. He points out that we often talk about students being wrong and having misconceptions, which is less productive than figuring out what they understand and where their alternative conceptions work or fail. I see his work following on to the work of Rich et al. (mentioned in this blog post) on CS learning trajectories. There are so many things to learn in CS, and sometimes, just getting started on the trajectory is a big step.

Spatial Encoding Strategy Theory: The Relationship between Spatial Skill and STEM Achievement by Lauren Margulieux.

Lauren is doing some impressive theoretical work here. She’s considering the work exploring the relationship between spatial reasoning and CS learning/performance, then constructs a theory explaining the observed results. Since it’s Lauren, the theory is thorough and covers well the known results in this space. I wrote her that I didn’t think that theory explains things that we expect are related to spatial reasoning, but we don’t yet have empirical evidence to support it. For example, when programmers simulate a program in their mind, their mental models may have a spatial component to them, but I don’t know of empirical work that explores that dimension of CS performance. But again, since it’s Lauren, I wouldn’t be surprised if her presentation addresses this point, beyond what was in the paper. (Also, read Lauren’s own summary of the paper here.)

I am looking forward to the discussion of these papers at ICER!

August 12, 2019 at 7:00 am Leave a comment

Older Posts


Enter your email address to follow this blog and receive notifications of new posts by email.

Join 6,309 other followers

Feeds

Recent Posts

Blog Stats

  • 1,681,484 hits
September 2019
M T W T F S S
« Aug    
 1
2345678
9101112131415
16171819202122
23242526272829
30  

CS Teaching Tips