Posts tagged ‘computing for everyone’
No, Really – Programming is Hard and CS Flipped Classrooms are Complicated: ITICSE 2016 Award-Winning Papers
I only recently started digging into the papers from the ITICSE 2016 conference (see Table of Contents link at ACM Digital Library here). There were two papers that caught my attention which led to me to explore these papers.
First, the best paper award went to one of my former PhD students, Brian Dorn: An Empirical Analysis of Video Viewing Behaviors in Flipped CS1 Courses, by Suzanne L. Dazo, Nicholas R. Stepanek, Robert Fulkerson, and Brian Dorn. Brian has this cool piece of technology where students can view videos, annotate them, be challenged to answer questions from specific places, and have discussions. They used this for teaching a flipped CS1 class, where students were required to watch videos before class and then engage in more active learning opportunities in class. The real trick, as you might imagine and that the paper goes into detail on, is getting students to watch the video. I liked both the techniques for prodding students to watch videos and the fascinating results showing the relationship between watching the videos and learning.
ITICSE 2016 recognized two “commended” papers this year. I haven’t found the listing of which papers they were, but I did learn that one of them is Learning to Program is Easy by Andrew Luxton-Reilly. I enjoyed reading the paper and recommend it — even though I disagree with his conclusions, captured in the paper title. He does a good job of exploring the evidence that programming is hard (and even uses this blog as a foil, since I’ve claimed several times that programming is hard), and overall, is a terrific synthesis of a bunch of computing education papers (40 references is a lot for a six page ITICSE paper).
His argument that programming is easy has two parts:
- First, children do it. As he says in the abstract, “But learning to program is easy — so easy that children can do it.” That’s a false comparison — what children do in programming is not the same definition of “programming” that is in most of the literature that Andrew cites. The evidence that programming is hard is coming mostly from higher-ed CS classes. What is going on in introductory University CS classes and what children do is dramatically different. We saw that in the WIPSCE 2014 Fields and Kafai paper, and those results were recently replicated in a recent ICER 2016 paper. These are two different activities.
- Second, what higher-education CS teachers expect at the end of the first course is too much. He presents significant evidence that what CS teachers expect is achieved by students, but at the end of the second course. The paper from Morrison, Decker, and Margulieux supports the argument that students think and work very differently and much more successfully by the end of the second CS course than in the first course.
I see Andrew’s argument as evidence that programming is hard. The problem is that Andrew doesn’t define the target. What level of ability counts as “programming”? I believe that level of ability described by the McCracken Working Group, by the FCS1/SCS1 exams, and by most teachers as the outcomes from CS1 (these are all cited by Andrew’s paper) is the definition of the lowest level of “programming ability.” That it takes two courses to reach that level of ability is what I would call hard.
I’ve been reading a terrific book, Proust and the Squid: The Story and Science of the Reading Brain by Maryanne Wolf. It’s the story of how humans invented reading, how we teach reading, and how reading changes our brains (physically and in terms of cognitive function). Oral language is easy. We are literally wired for that. Reading is hard. We are not wired for that, and much of the invention of reading is about inventing how to teach reading. Unless you can teach reading to a significant part of your population, you don’t develop a literate culture, and your written language doesn’t succeed.
Much of the invention of written language is about making it easier to learn and teach because learning to read is so hard. Have you ever thought about why our Latin alphabet is ordered? Why do we talk about the “ABC”‘s and sing a song about them? We don’t actually need them to be ordered to read. Ordering the alphabet makes it easier to memorize, and learning to read is a lot about memorization, about drill-and-practice to make the translation of symbols to sounds to words to concepts effortless (or at least, all System 1 in Kahneman terms). This makes it easier, but the task of learning to read and write is still a cognitively complex task that takes a significant amount of time to master. It’s hard.
Programming is hard like written language is hard. It’s not possible to program unless you know how to read. Programming is particularly hard because the concepts that we’re mapping to are unfamiliar, are not part of our daily experience. We only see it as easy because we have expert blind-spot. We have already learned these concepts and made those mappings. We have constructed understandings of iteration and conditional execution and variable storage. It is difficult for experts to understand how hard it is to develop those concepts. The evidence of children programming suggests that most children who program don’t have those concepts.
I remain unconvinced by Andrew’s argument, but I recommend the paper for a great summary of literature and an interesting read.
Every University Student should Learn to Program: Guzdial Arguing for CS for All in Higher Education
A colleague recently approached me and said, “It would be useful if Universities got involved in this CS for All effort. All Universities should offer courses aimed at everyone on campus. There should be a systematic effort to get everyone to take those classes.”
I agree, and have been making this argument for several years now. I spent a few minutes gathering the papers, blog posts, and book where I’ve made that argument over the last decade and a bit.
In 2002, Elliot Soloway and I argued in CACM that we needed a new way to engage students in intro programming: Teaching the Nintendo Generation to Program.
In 2003, I published the first paper on Media Computation: A media computation course for non-majors.
In 2004, Andrea Forte led the team studying the Media Computation class at GT:Computers for communication, not calculation: Media as a motivation and context for learning and A CS1 course designed to address interests of women.
In 2005, Andrea Forte and I presented empirical evidence about the courses we’d designed for specific audiences: Motivation and nonmajors in computer science: identifying discrete audiences for introductory courses. I published a paper in CACM about how the courses came to be at Georgia Tech: Teaching computing to everyone.
In 2008, I offered the historical argument for teaching everyone to program: Paving the Way for Computational Thinking.
We’ve published several papers about our design process: Imagineering inauthentic legitimate peripheral participation: an instructional design approach for motivating computing education and Design process for a non-majors computing course.
My 2013 ICER paper was a review of a decade’s worth of research on Media Computation: Exploring hypotheses about media computation
My keynote at VL/HCC 2015 was on how computing for all is a requirement for modern society: Requirements for a computing-literate society
My 2015 book is, to a great extent: an exploration of how to achieve CS for All: Learner-Centered Design of Computing Education: Research on Computing for Everyone.
In blog posts, it’s been a frequent topic of conversation:
- In 2011, I argued that it makes more sense to require CS at universities before pushing into K-12, because then all pre-service teachers have some CS which makes later PD much easier and cheaper: https://computinged.wordpress.com/2015/11/30/require-cs-at-universities-before-k-12-computational-community-for-everyone/ and https://computinged.wordpress.com/2011/05/17/if-you-want-cs-in-high-school-require-cs-in-college/
- In 2013, I pointed out that CS is becoming increasingly valuable outside of CS: https://computinged.wordpress.com/2013/12/10/why-are-english-and-lots-of-other-majors-studying-computer-science/
- One of my earlier Blog@CACM posts was on how students learn things in MediaComp that informs them about their world, not just about CS: http://cacm.acm.org/blogs/blog-cacm/26343-media-computation-for-creativity-and-surprises/fulltext
- On how CS is a value-added to a liberal education: http://cacm.acm.org/blogs/blog-cacm/101738-computer-science-as-value-added-to-a-liberal-education/fulltext
I don’t know how to convince University CS departments to do just about anything, but here are my contributions to the dialogs that I hope are happening at Colleges and Universities worldwide about how to prepare students to engage in computational literacy.
Ruthe Farmer at the Office of Science and Technology Policy (OSTP) has been working furiously towards today’s announcements. The Obama Administration is aiming to achieve the goal of CSforAll, and with only a few months left before the new Administration takes off, they’re showing what they’ve put in place today. The full details on all the announcements are here. There’s a webcast at 1 pm EDT today here. The biggest deal to me is the establishment of the CSforAll Consortium (see website here) which is meant to carry on the initiative, no matter who wins in November.
To mark this progress, and celebrate new commitments in support of the President’s initiative, the White House is hosting a summit on Computer Science for All. Key announcements being made today include:
More than $25 million in new grants awarded from the National Science Foundation (NSF) to expand CS education;
A new CSforAll Consortium of more than 180 organizations, which will connect stakeholders with curriculum and resources, as well as track progress towards the goal of Computer Science for All; and
New commitments from more than 200 organizations, ranging from expanded CS offerings within the Girl Scouts of the USA that could reach 1.4 million girls per year, to Code.org supporting professional development for 40,000 additional teachers, to new collaborations to bring CS to students in a variety of settings from African-American churches to family coding nights to tribal Head Start programs to students as Chief Science Officers.
ICER 2016 was just held in Melbourne, Australia, so I found the article linked at the bottom (and from which these images come from) particularly relevant and interesting.
Australia is facing a boom in primary school students, which creates additional demand for teachers. As has been mentioned here previously, there is a shortage of teachers. The shortage isn’t distributed across fields. In particular, over 30% of computing teachers in Australia are teaching without qualification (see image below). When considering other shortages (e.g., declining number of computing teachers in Scotland, as described in the last post), it’s clear that the pipeline of CS teachers is going to be an impediment to CS for all.
But an influx of new students isn’t the only problem our school system needs to address.Shortages in specific subject areas mean that many students are being taught by teachers working outside of their qualifications.
When I visited Mumbai for LaTICE 2016, I mentioned meeting Yogendra Pal. I was asked to be a reader for his thesis, which I found fascinating. I’m pleased to report that he has now graduated and his thesis, A Framework for Scaffolding to Teach Vernacular Medium Learners, is available here: https://www.cse.iitb.ac.in/~sri/students/#yogendra.
I learned a lot from Yogendra’s thesis, like what “vernacular medium learners” means. Here’s the problem that he’s facing (and that Yogendra faced as a student). Students go through primary and secondary school learning in one language (Hindi, in Yogendra’s personal case and in his thesis), and then come to University to study Computer Science. Do you teach them (what Yogendra calls “Medium of Instruction” or MoI) in English, or in Hindi? Note that English is pervasive in Computer Science, e.g., almost all our programming languages use English keywords.
Here’s Yogendra’s bottomline finding: “We find that self-paced video-based environment is more suitable for vernacular medium students than a classroom environment if English-only MoI are used.” Yogendra uses a design-based research methodology. He measures the students, tries something based on his current hypothesis, then measures them again. He compares what he thought would happen to what he saw, and revises his hypothesis — and then iterate. Some of the scaffolds he tested may seem obvious (like using a slower pace), but a strength of the thesis is that he develops rationale for each of his changes and tests them. Eventually, he came to this surprising (to me) and interesting result: It’s better to teach with Hindi in the classroom, and in English when students are learning from self-paced videos.
The stories at the beginning of the thesis are insightful and moving. I hadn’t realized what a handicap it is to be learning English in a class that uses English. It’s obvious that the learners would be struggling with the language. What I hadn’t realized was how hard it is to raise your hand and ask questions. Maybe you have a question just because you don’t know the language. Maybe you’ll expose yourself to ridicule because you’ll post the question wrong.
Yogendra describes solutions that the Hindi-speaking students tried, and where the solutions didn’t work. The Hindi-speaking students used English-to-English dictionaries. They didn’t want English-Hindi dictionaries, because they wanted to become fluent in English, but they needed help with the complicated (especially technical) words. They tried using online videos for additional explanations of concepts, but most of those were made by American or British speakers. When you’re still learning English, switching from an Indian accent to another accent is a barrier to understanding.
The middle chapters are a detailed description of Yogendra’s attempts to scaffold student learning. He tried to teach in all-Hindi but some English technical terms like “execute” don’t have a direct translation in Hindi. He selected other Hindi words to represent the technical terms, but the words he selected as the Hindi translation were unusual and not well-known to the students. Perhaps the most compelling insight for me in these chapters was how important it was to both the students and the teachers that the students learn English — even when the Hindi materials were measurably better for learning in some conditions.
In the end, he found that Hindi language screencasts led to better learning (statistically significantly) when the learners (who had received primary and secondary school instruction in Hindi) were in a classroom, but that the English language screencasts led to better learning (again, statistically significantly) when the learners were watching the screencasts self-paced. When the students are self-paced, they can rewind and re-watch things that are confusing, so it’s okay to struggle with the English. In the classroom, the lecture just goes on by. It works best if it’s in Hindi for the students who learned in Hindi in school.
Yogendra tells a convincing story. It’s an interesting question of how these lessons transfer to other contexts. For example, what are the issues for Spanish-speaking students learning CS in the United States? In a general form, can we use the lessons from this thesis to make CS learning accessible to more ESL (English as a Second Language) learners?
When I give talks about teaching computer to everyone, I often start with Alan Perlis and C.P. Snow in 1961. They made the first two public arguments for teaching computer science to everyone in higher education. Alan Perlis’s talk was the most up-beat, talking about all the great things we can think about and do with computer. He offered the carrot. C.P. Snow offered the stick.
C.P. Snow foresaw that algorithms were going to run our world, and people would be creating those algorithms without oversight by the people whose lives would be controlled by them. Those who don’t understand algorithms don’t know how to challenge them, to ask about them, to fight back against them. Quoting from Martin Greenberger’s edited volume, Computers and the World of the Future (MIT Press, 1962), we hear from Snow:
Decisions which are going to affect a great deal of our lives, indeed whether we live at all, will have to be taken or actually are being taken by extremely small numbers of people, who are nominally scientists. The execution of these decisions has to be entrusted to people who do not quite understand what the depth of the argument is. That is one of the consequences of the lapse or gulf in communication between scientists and non-scientists. There it is. A handful of people, having no relation to the will of society, have no communication with the rest of society, will be taking decisions in secret which are going to affect our lives in the deepest sense.
I was reminded of Snow’s quote when I read the article linked below in the NYTimes. Increasingly, AI algorithms are controlling our lives, and they are programmed by data. If all those data are white and male, the algorithms are going to treat everyone else as outliers. And it’s all “decisions in secret.”
This is fundamentally a data problem. Algorithms learn by being fed certain images, often chosen by engineers, and the system builds a model of the world based on those images. If a system is trained on photos of people who are overwhelmingly white, it will have a harder time recognizing nonwhite faces.
A very serious example was revealed in an investigation published last month by ProPublica. It found that widely used software that assessed the risk of recidivism in criminals was twice as likely to mistakenly flag black defendants as being at a higher risk of committing future crimes. It was also twice as likely to incorrectly flag white defendants as low risk.
The reason those predictions are so skewed is still unknown, because the company responsible for these algorithms keeps its formulas secret — it’s proprietary information. Judges do rely on machine-driven risk assessments in different ways — some may even discount them entirely — but there is little they can do to understand the logic behind them.
One of our superstar alumna, Joy Buolamwini, wrote about a similar set of experiences. She’s an African-American woman who works with computer vision, and the standard face-recognition libraries don’t recognize her. She lays the responsibility for fixing these problems on the backs of “those who have the power to code systems.” C.P. Snow would go further — he’d say that it’s all our responsibility, as part of a democratic process. Knowing about algorithms and demanding transparency when they effect people’s lives is one of the responsibilities of citizens in the modern world.
The faces that are chosen for the training set impact what the code recognizes as a face. A lack of diversity in the training set leads to an inability to easily characterize faces that do not fit the normal face derived from the training set.
So what? As a result when I work on projects like the Aspire Mirror (pictured above), I am reminded that the training sets were not tuned for faces like mine. To test out the code I created for the Aspire Mirror and subsequent projects, I wore a white mask so that my face can be detected in a variety of lighting conditions.
The mirror experience brings back memories from 2009. While I was working on my robotics project as an undergraduate, I “borrowed” my roommate’s face so that I could test the code I was writing. I assumed someone would fix the problem, so I completed my research assignment and moved on.
Several years later in 2011, I was in Hong Kong taking a tour of a start-up. I was introduced to a social robot. The robot worked well with everyone on the tour except for me. My face could not be recognized. I asked the creators which libraries they used and soon discovered that they used the code libraries I had used as an undergraduate. I assumed someone would fix the problem, so I completed the tour and moved on.
Seven years since my first encounter with this problem, I realize that I cannot simply move on as the problems with inclusion persist. While I cannot fix coded bias in every system by myself, I can raise awareness, create pathways for more diverse training sets, and challenge us to examine the Coded Gaze — the embedded views that are propagated by those who have the power to code systems.
Yasmin Kafai has been a friend and mentor to me for years — she introduced me to my PhD advisor, Elliot Soloway. Her book with Quinn Burke, Connected Code, updates thinking about the role of computing and programming in schools. They emphasize an idea they call Computational Participation as a contrast with computational thinking. I asked Yasmin to do a CACM Viewpoint on the idea, and it’s published this month. Yasmin has shared the paper on Academia.edu.
In the 1980s many schools featured Basic, Logo, or Pascal programming computer labs. Students typically received weekly introductory programming instruction. These exercises were often of limited complexity, disconnected from classroom work, and lacking in relevance. They did not deliver on promises. By the mid-1990s most schools had turned away from programming. Pre-assembled multimedia packages burned onto glossy CD-ROMs took over. Toiling over syntax typos and debugging problems were no longer classroom activities.
Computer science is making a comeback in schools. We should not repeat earlier mistakes, but leverage what we have learned. Why are students interested in programming? Under what circumstances do they do it, and how? Computational thinking and programming are social, creative practices. They offer a context for making applications of significance for others, communities in which design sharing and collaboration with others are paramount. Computational thinking should be reframed as computational participation.