How to reduce the defensive climate, and what students really need to understand code: ITICSE 2019 Preview

This year, we’re presenting two papers at the 2019 ACM SIGCSE Innovation and Technology in CS Education (ITICSE) conference in Aberdeen, Scotland. I’ve been to ITICSE several times and enjoy it every time, but I can’t always justify the trip. This year, Barbara Ericson, Katie Cunningham, and I are all in Germany for a Dagstuhl Seminar on Notional Machines the week before, so we took the opportunity to submit and are fortunate to be invited to present.

Making CS Learning Visible: Case Studies on How Visibility of Student Work Supports a Community of Learners in CS Classrooms by Amber Solomon, Vanessa Oguamanam, Mark Guzdial, and Betsy DiSalvo

When I taught CS Ed Research this last semester (see the blog post about open questions from the class), the students so resonated with Lecia Barker’s papers about defensive climate (the classic paper is here). The story about how CS classes are “characterized by competitiveness rather cooperation, judgments about others, superiority, and neutrality rather than empathy” still rings true, 17 years after first written. Several of my students proposed research to follow-up on the original study.

Amber and Vanessa are also motivated by the concerns about defensive climate in CS classes, but they don’t want to measure it. They are suggesting an intervention.

They suggest that a community of learners approach would reduce defensive climate. Key to creating a community of CS learners, they propose, is making student work and process visible. Vanessa works in marker-oriented curricula, where student work is physical and the work process is visible. Amber did the evaluation of the AR design studio classroom that I’ve written about previously. In both of these case studies, they observed student communication patterns that were different from the defensive climate studies and more in keeping with a community of learners culture. Defensive climate is still a problem, and changing culture and community is the way to address it.

Novice Rationales for Sketching and Tracing, and How They Try to Avoid It by Katie Cunningham, Shannon Ke, Mark Guzdial, and Barbara Ericson

At ICER 2017, Katie presented a study of how students trace their programs on paper (see ICER 2017 paper here, and my blog post summary here). She had some fascinating and surprising results. For example, if students traced their programs only part way and then guessed at the final result, they were more likely to get the problem wrong than if they’d never traced before. But why? Why would students trace the program part way and then stop — only to get it wrong?

That’s what Katie explored in this follow-up paper. She had an innovative experimental design. She replicated her original tracing experiment, then pulled out about a dozen interesting participants, and invited them back for a retrospective interview. She could show them the original problem and what they traced — and then ask them why? Here’s one of the answers: They thought that they “got the hang of it.” They recognized a goal or pattern. They just recognized wrong.

One of my favorite parts of the paper is completely visual. Katie had a terrific idea — let’s ask the teacher of the class to trace the problems. Here’s one of the traces that the teacher did:

Here are some examples of what students did:

Notice a distinct lack of similarity? Why? Why don’t students trace like the instructor did?

This is a qualitative study, so it’s rich with interview data. I recommend reading the whole paper. There’s a neat part at the end where Katie points out, “Program visualizations do X. When students trace, they do Y. Why are these things so different?”