No, Really – Programming is Hard and CS Flipped Classrooms are Complicated: ITICSE 2016 Award-Winning Papers

September 30, 2016 at 7:22 am 8 comments

I only recently started digging into the papers from the ITICSE 2016 conference (see Table of Contents link at ACM Digital Library here).  There were two papers that caught my attention.

First, the best paper award went to one of my former PhD students, Brian Dorn: An Empirical Analysis of Video Viewing Behaviors in Flipped CS1 Courses, by  Suzanne L. Dazo, Nicholas R. Stepanek, Robert Fulkerson, and Brian Dorn.  Brian has this cool piece of technology where students can view videos, annotate them, be challenged to answer questions from specific places, and have discussions.  They used this for teaching a flipped CS1 class, where students were required to watch videos before class and then engage in more active learning opportunities in class.  The real trick, as you might imagine and that the paper goes into detail on, is getting students to watch the video.  I liked both the techniques for prodding students to watch videos and the fascinating results showing the relationship between watching the videos and learning.

ITICSE 2016 recognized two “commended” papers this year.  I haven’t found the listing of which papers they were, but I did learn that one of them is Learning to Program is Easy by Andrew Luxton-Reilly.  I enjoyed reading the paper and recommend it — even though I disagree with his conclusions, captured in the paper title.  He does a good job of exploring the evidence that programming is hard (and even uses this blog as a foil, since I’ve claimed several times that programming is hard), and overall, is a terrific synthesis of a bunch of computing education papers (40 references is a lot for a six page ITICSE paper).

His argument that programming is easy has two parts:

  • First, children do it. As he says in the abstract, “But learning to program is easy — so easy that children can do it.”  That’s a false comparison — what children do in programming is not the same definition of “programming” that is in most of the literature that Andrew cites.  The evidence that programming is hard is coming mostly from higher-ed CS classes.  What is going on in introductory University CS classes and what children do is dramatically different.  We saw that in the WIPSCE 2014 Fields and Kafai paper, and those results were recently replicated in a recent ICER 2016 paper.  These are two different activities.
  • Second, what higher-education CS teachers expect at the end of the first course is too much.  He presents significant evidence that what CS teachers expect is achieved by students, but at the end of the second course.  The paper from Morrison, Decker, and Margulieux supports the argument that students think and work very differently and much more successfully by the end of the second CS course than in the first course.

I see Andrew’s argument as evidence that programming is hard.  The problem is that Andrew doesn’t define the target.  What level of ability counts as “programming”?  I believe that level of ability described by the McCracken Working Group, by the FCS1/SCS1 exams, and by most teachers as the outcomes from CS1 (these are all cited by Andrew’s paper) is the definition of the lowest level of “programming ability.”  That it takes two courses to reach that level of ability is what I would call hard.

I’ve been reading a terrific book, Proust and the Squid: The Story and Science of the Reading Brain by Maryanne Wolf.  It’s the story of how humans invented reading, how we teach reading, and how reading changes our brains (physically and in terms of cognitive function).  Oral language is easy.  We are literally wired for that.  Reading is hard.  We are not wired for that, and much of the invention of reading is about inventing how to teach reading.  Unless you can teach reading to a significant part of your population, you don’t develop a literate culture, and your written language doesn’t succeed.

Much of the invention of written language is about making it easier to learn and teach because learning to read is so hard.  Have you ever thought about why our Latin alphabet is ordered?  Why do we talk about the “ABC”‘s and sing a song about them?  We don’t actually need them to be ordered to read.  Ordering the alphabet makes it easier to memorize, and learning to read is a lot about memorization, about drill-and-practice to make the translation of symbols to sounds to words to concepts effortless (or at least, all System 1 in Kahneman terms).  This makes it easier, but the task of learning to read and write is still a cognitively complex task that takes a significant amount of time to master. It’s hard.

Programming is hard like written language is hard.  It’s not possible to program unless you know how to read.  Programming is particularly hard because the concepts that we’re mapping to are unfamiliar, are not part of our daily experience.  We only see it as easy because we have expert blind-spot.  We have already learned these concepts and made those mappings.  We have constructed understandings of iteration and conditional execution and variable storage.  It is difficult for experts to understand how hard it is to develop those concepts.  The evidence of children programming suggests that most children who program don’t have those concepts.

I remain unconvinced by Andrew’s argument, but I recommend the paper for a great summary of literature and an interesting read.


Entry filed under: Uncategorized. Tags: , , , , .

Making learning effective, efficient, and engaging: An Interview With an Educational Realist and Grumpy Old Man, Paul Kirschner How to Write a Guzdial Chart: Defining a Proposal in One Table

8 Comments Add your own

  • 1. Matt  |  September 30, 2016 at 9:33 am

    Dear Mark,

    “First, the best paper award went to one of my former PhD students, Brian Dorn”

    Actually, the first author on the paper was Brian’s student, Suzanne Dazo. She is a woman in computing. It is difficult, sometimes, to be constantly aware of the invisible injustices we do to women in computing, but we do need to be aware. Highlighting the male supervisor instead of the female first author is, I would guess, one of those long-standing issues in academia… we highlight the work of the man instead of the work of the woman.

    You might update your first paragraph:

    “First, the best paper award went to Suzanne Dazo, a graduate student who is advised by one of my former PhD students, Brian Dorn:”

    Admittedly, we’d probably highlight that Suzanne is a graduate of Berea College, but that’s just a local bit of pride for those of us at Berea College.

    More of the paragraph needs to change after the first sentence changes, but I think those changes follow naturally.

    Many thanks,

  • 2. Diana  |  September 30, 2016 at 10:05 am

    I have a different take on the “Programming is Easy” paper. The author, intentionally, takes an extreme view. Mark compares programming to language acquisition. I agree with that, and I think the author would, too. But many people think language is easy. Why? Because we spend so much time and energy doing it incrementally that we don’t notice it’s hard. How about if we taught language in one year and expected people to be proficient by then, fluent in two, and using it as merely a tool for learning many other things in year three. Then those who don’t pick up language really quickly will quit, thinking they’re incapable. Instructors will be tempted to weed early so students don’t “waste time” pursuing something they can’t do. When people think something is that hard, they make judgments about who can and can’t do it, and those judgments are heavily tainted by bias. Maybe, if we see it as “easy” – not easy like no one needs to work at it, but easy like if we just teach a little a year, incrementally, and expect that everyone can learn it, then maybe we’ll make a better environment and bring in underrepresented populations.

    That’s the vision of “Easy” that I see, and I think it rings a lot more true than the current version of “Hard/Impossible for many” that too many people believe right now.

    I agree that the article overstates the case.

    • 3. Bonnie  |  September 30, 2016 at 12:55 pm

      If you are thinking of language as merely oral, then this makes sense. But programming is probably more like learning to read and write. I don’t think the incremental approach works all that well for those skills either. Look at the many students who make it to college without good reading or writing skills, despite having studied those subjects incrementally since kindergarten. I relaly don’t think we have any more understanding of how to effectively teach reading and writing than we do programming.

  • […] The key to making Computer Science (CS) and Computational Thinking (CT) as widely taught as Math and Science is to provide teachers with the tools and the training that they need to implement CS and CT classes. Teacher training for CS and CT classes has to encompass not just CS and CT principles and practices, but also pedagogy. This is because novice programmers face challenges that students in a math or science class would not encounter. […]

  • 5. Andrew Kay  |  July 10, 2017 at 8:19 pm

    As I understand it, Luxton-Reilly’s argument has two parts:

    #1. Since children can successfully learn to program (to a level below that expected of undergraduates) then there is at a lower non-zero level we could set CS1 expectations at, which would be attainable by many more students, if we choose to do so.
    #2. We should choose to do so, because then our expectations will be more realistic and many more students will meet them.

    I can think of arguments against #2, but since you call McCracken/FCS1 the definition of the lowest level of ability, it sounds like you dispute #1. I’d be interested if you could elaborate on your response.

    • 6. Mark Guzdial  |  July 11, 2017 at 6:17 am

      Andrew, would you accept a revision to #1 that only “SOME children can successfully learn to program”? We are still lacking evidence that we can teach computing to ALL students. I’m teaching an HCI class this semester where we talk about Universal Design, and because of my own interests, I’m exploring Universal Design for Learning. It’s quite clear that we are not teaching computing in a universal manner. For example, we know relatively little about teaching students with cognitive difficulties, low intelligence, or behavioral difficulties, and that’s significant part of ALL (10-15% in some estimates).

      In the UK, there is a clear problem with motivation, too. The existing GCSE and A-level curricula are not being successful at attracting a large and diverse proportion of students (see Roehampton report). If you can’t engage them, you can’t teach them, which is another limitation to ALL.

      The challenge of #2 is getting agreement. We really need more studies of what CS1 instructors value and implement in their classes. SIGCSE attendees are only a subset of all CS1 instructors. I wonder why we have so many CS1’s in the United States that teach the intro course with C++ and passive lecture. That suggests to me that there are CS1 instructor goals other than being able to teach everyone, teach effectively, and teach for engagement.

  • 7. gasstationwithoutpumps  |  July 11, 2017 at 12:08 pm

    “I wonder why we have so many CS1’s in the United States that teach the intro course with C++ and passive lecture.” That’s pretty easy to answer—it is the cheapest way to teach large numbers of students, even if it isn’t the most effective one. With the continued defunding of public universities and the increasing demands that faculty spend all their time on pursuing grants, there are no resources for other, more labor-intensive teaching methods. (The MOOC approach seems to be even less effective than the megalecture.)

  • […] to be more influential. I’m not saying that programming isn’t hard. There are (1) good (2) arguments for it being hard. Probably, most of my papers make and/or use that argument. What I am wondering […]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 10,184 other subscribers


Recent Posts

Blog Stats

  • 2,048,345 hits
September 2016

CS Teaching Tips

%d bloggers like this: