Changing Computer Science Education to eliminate structural inequities and in response to a pandemic: Starting a Four Part Series

July 13, 2020 at 7:00 am 12 comments

George Floyd’s tragic death has sparked a movement to learn about race and to eliminate structural inequities and racism. My email is flooded with letters and statements demanding change and recommending actions. These include a letter from Black scholars and other members of the ACM to the leadership of the ACM (see link here), the Black in Computing Open Letter and Call to Action (see link here, and the Hispanics in Computing supportive response link here). The letter about addressing institutional racism in the SIGCHI community from the Realizing that All Can be Equal (R.A.C.E) is powerful and enlightening (see link here).

I’m reading daily about race. I’m not an expert, or even particularly well-informed yet. One of the books I’m reading is Me and White Supremacy by Layla F. Saad (see Amazon link here) where the author warns against:

Using perfectionism to avoid doing the work and fearing using your voice or showing up for antiracism work until you know everything perfectly and can avoid being called out for making mistakes.

This post is the start of a four part series about what we should be changing in computing education towards eliminating structural inequities. We too often build computing education for the most privileged, for the majority demographic groups. It’s past time to support alternative pathways into computing. Even if you’re not driven by concerns about racial injustice, I ask you to take my proposals seriously because of the pandemic. We don’t know how to teach CS remotely at this enormous scale over the next year, and the least-privileged students will be hurt the most by this. We must CS teach differently so that we eliminate the gap between the most and least privileged of our students. Here’s what we need to do.

Learn about Race

Amber Solomon, a PhD student working with Betsy DiSalvo and me, reviewed my first two posts about race in CS Education (at Blog@CACM and here a few weeks ago). Amber has written on intersectionality in CS education, and is writing a dissertation about the role of embodied representations in CS education (see a post here about her most recent paper). She recommended more on learning about race:

  • Whiteness as Property by Cheryl Harris (see link here). Harris, Andre Brock, and some race scholars, argue that to understand racism, you should understand whiteness, not Blackness.
  • The Matter of Race in Histories of American Technology by Herzig (see link here). She has the clearest explanation of “race and/as technology” that I’ve read. And she also does a great job explaining why we can’t just say that race is a social construct.

Two videos:

  • Repurposing Our Pedagogies: Abolitionist Teaching in a Global Pandemic (see YouTube link here).
  • Data, AI, Public Health, Policing, the Pandemic, and Un-Making Carceral States (see YouTube link here). it’s about data, but they get into what it means to be racially Black, white, etc.; and Ruha says something super interesting “rather than collecting racial data, think about what it would mean to collect data on racism.”

Think about the words you use, like “Underrepresented Minority”

Tiffani Williams wrote a Blog@CACM post in June that made an important point about the term “underrepresented minority” (see link here). She argues that it’s a racist term and we should strike it from our language.

Reason #1: URM is racist language because it denies groups the right to name themselves.

Reason #2: URM is racist language because it blinds us to the differences in circumstances of members in the group.

Reason #3: URM is racist language because it implies a master-slave relationship between overrepresented majorities and underrepresented minorities.

In Me and White Supremacy, Saad uses BIPOC (Black, Indigenous, and People of Color), but points out that that is mostly a shorthand for “people lacking white privilege.” She argues, as does Williams, that the term BIPOC ignores the differences in experience between people in those groups. I am striving to be careful in my language and be thoughtful when I use terms like “BIPOC” and “underrepresented.”

Change Computing Departments

Amy Ko has made some strong and insightful posts in the last month about the injustice and exclusion in CS education (see Microsoft presentation slides here). She wrote a powerful post about why her undergraduate major in Information Technology at the University of Washington is racist (see link here). Obviously, her point is that it’s not just her program — certainly the vast majority of computing majors are racist in the ways that she describes. There are mechanisms that are better, like the lottery I described recently to reduce the bias in admission to the major. Amy’s points are inspiring this blog post series.

Manuel Pérez-Quiñones has started blogging, with posts on what CS departments should do to dismantle racism (see link here) and about why CS departments should create more student organizations to combat racism (see link here). His first post has a quote that inspires me:

First, it should come as no surprise that many things we assume to be fair, standard, or just plain normal in reality are not. Even our notion of “fair” has been constructed from a point of view that prioritizes fairness for certain groups. Not only is history written by the victors; laws, structures, and other pieces of society are developed by them too. To expect them to be fair or equitable is naive at best.

Chad Jenkins shared with me a video of his keynote from the RSS 2018 Conference where he suggests that CS departments need to change their research focus, too, to incorporate a value for equity and human values.

The Chair of our department’s Diversity, Equity, and Inclusion (DEI) Committee, Wes Weimer, is pushing for all Computer Science departments to be transparent about how they’re doing on their goals to make CS more diverse and equitable, and what their plans are. The Computer Science & Engineering division at the University of Michigan is serving as an example by making its annual DEI report publicly available here. In the comments, please share your department’s DEI report. Let’s follow Wes’s lead and makes this the common, annual practice.

Change how we teach Computing

Manuel’s point isn’t just about departments. We as individual teachers of computer science and computing make choices which we think are “fair, standard” but actually support and enforce structural inequities. We have to change how we teach. CS for All has published a statement on anti-racism and injustice (see link here) where they say:

We pledge to repeatedly speak out against our historical pedagogies and approaches to computer science instruction that are grounded and designed to weed out all but a small prerogative subset of the US population.

Chad Jenkins mailed me the statement from the University of Maryland’s CS department about their recommendations to improve diversity and inclusion. I loved this quote, which will be the theme for this series of posts:

Creating a task force within the Education Committee for a full review of the computer science curriculum to ensure that classes are structured such that students starting out with less computing background can succeed, as well as reorienting the department teaching culture towards a growth mindset

We currently teach computer science in ways that “weed out all but a small prerogative subset of the US population” (CS for All). We need to teach so that “students starting out with less computing background can succeed” (UMdCS). We teach in ways that assume a fixed mindset — we presume that some students have a “Geek Gene” and there’s nothing much that teaching can do to change that. We know the opposite — teaching can trump genetics.

Even if you don’t care about race or believe that we have created structural inequities in CS education, I ask you to change because of the pandemic. Teaching on-line will likely hurt our students with the least preparation (see post here). We have to teach differently this year when students will have fewer resources, and we are literally inventing our classes anew in remote forms. If we don’t teach differently, we will increase the gap between those more or less computing background.

While I am just learning about race, I have been studying for years how to teach computing to people with less computing background. This is what this series of posts is about. In the next three posts, I make concrete recommendations about how we should teach differently to reduce inequity. I hope that you are inspired by the desire to eliminate racial inequities, but if not, I trust that you will recognize the need to teach differently because of the pandemic.

First step: Stop using pseudocode on the AP CS Principles Exam

Here’s an example of a structural inequity that weeds out students with less computing background. The AP CS Principles exam (see website here) is meant to be agnostic about what programming language the students are taught, so the programming problems on the actual exam are given in a pseudocode — either text or block-based (randomly). There is no interpreter generally available for the pseudocode, so students learn one language (maybe Snap! or Scratch or MIT App Inventor) and answer questions in another one.

Advanced Placement (AP) classes are generally supposed to replicate the experience of introductory courses at College. AP CSP is supposed to map to a non-CS majors’ intro to computing course. How many of these teach in language X, but then ask students to take their final exam in language Y which they’ve never used?

Allison Elliott Tew’s dissertation (see link here) is one of the only studies I know where students completed a validated instrument both in a pseudocode and in whatever language they learned (Java, MATLAB, and Python in her study). She found that the students who scored the best on the pseudocode exam had the closest match in scores between the pseudocode exam and the intro language exam — averaging a difference in 2.31 answers out of the 27 questions on the exams (see table below.). But the average difference increases dramatically. For the bottom two quartiles, the difference is 17% (4.8 questions out of 27) and 22% (6.2 questions out of 27). It’s not too difficult for students in the best-performing quartile to transfer their knowledge to the pseudocode, but it’s a significant challenge for the lowest-performing two quartiles. These results suggest predict that giving the AP CSP exam in a pseudocode is a barrier, which is easily handled by the most prepared students and is much more of a barrier for the least prepared students.

I went to a bunch of meetings around the AP CSP exam when it was first being set up. At a meeting where the pseudocode plans were announced, I raised the issue of Allison’s results. The response from the College Board was that, while it was a concern, it was not likely going to be a significant problem for the average student. That’s true, and I accepted it at the time. But now, we’re aware of the structural inequities that we have erected that “weed out all but a small prerogative subset of the US population” (CS for All). It’s not acceptable that switching to a pseudocode dramatically increases the odds that students in the bottom half fail the AP CSP exam, when they might have passed if they were given the language that they learned.

Further, studies of block-based and text languages in the context of AP CSP support the argument that students overall do better in a block-based language (see this post here as an example). Every text-based problem decreases the odds that female, Black, and Hispanic students will pass the exam (using the race labels the students use to self-identify on the exam). If the same problem was in a block-based language, they would likely do better.

Using pseudocode on the AP CSP exam is like a tax. Everyone has to manage a bit more difficulty by mapping to a new language they’ve never used. But it’s a regressive tax. It’s much more easily handled by the most privileged and most prepared students.

AP CSP is an important program that is making computing education available to students who otherwise might never access a CS course. We should grow this program. But we should scrap the AP CSP exam in its current form. I understand that the College Board and the creators of the AP CSP exam were aiming with the pseudocode, mixed-modality exam to create freedom for schools and teachers to teach with whatever the language and curriculum they wanted. However, we now know that that flexibility comes at a cost, and that cost is greater for students with less computing background, with less preparation, and who are female, Black, or Hispanic. This is structural inequity.

Entry filed under: Uncategorized. Tags: , , , .

Paradigm shifts in education and educational technology: Influencing the students here and now Proposal #1 to Change CS Education to Reduce Inequity: Teach computer science to advantage the students with less computing background

12 Comments Add your own

  • 1. Mr. Bartucz  |  July 13, 2020 at 9:02 am

    Hi Mark, thank you for posting this. It contains a lot of information that will take quite a while to digest. Two thoughts come to mind: first, instead of completely scrapping the CSP exam, why can’t we have different versions for different languages that the students know? It doesn’t seem that difficult to substitute the few questions that are in pseudocode with whatever language the student prefers. Second, I think you’ve finally inspired me to start my own blog with a thought I’ve been having for years – why is there no focus on teaching the students we have about how to use computer science for anti-racism and anti-bias? Why are we only focused on increasing the percentage of BIPOC computer scientists? Are we expecting that will solve the problem? It is 100% our responsibility to teach the students we have to be anti-racists and we can do that through the lens of computer science.

    • 2. Mark Guzdial  |  July 13, 2020 at 9:06 am

      I would be fine with replacing the pseudocode with the different languages, but it won’t meet the standards of the ETS psychometricians. ETS builds excellent standardized tests, which means that the test items are carefully developed, measured, and calibrated. Simply swapping out languages doesn’t result in the same items. (And it doesn’t really work — Snap! is more of a functional language which doesn’t map easily to the imperative pseudocode.) It will be expensive to do that testing with every item in every CSP language. They’re going to have to pick a few standard languages.

      I hope you enjoy the rest of the series. I’ll be addressing some of those themes over the next four weeks. You should do your blog! It’s a great idea!

  • 3. Richard Ladner  |  July 13, 2020 at 12:55 pm

    Hi Mark,

    Why would you say “Further, studies of block-based and text languages in the context of AP CSP support the argument that everyone does better in a block-based language…?” First, the popular block-based languages are not screen reader accessible, making them not usable by blind learners. Second, all these papers show a statistical difference, not that “everyone” does better.

    • 4. Mark Guzdial  |  July 13, 2020 at 1:33 pm

      Hi Richard,

      I was based the claim of “everyone” on the analysis in the Weintrop et al. ICLS paper where they say “Overall” with an n>5000 of AP CSP students. It is absolutely true that they don’t call out visually-disabled learners, so that’s a fair criticism that I don’t know “everyone.” I don’t understand your second point. How is a statistical difference not the same as doing better?

      • 5. cdsmith  |  July 13, 2020 at 3:36 pm

        I suspect that Richard’s point (and it may be my point, too) is that there are almost certainly students who performed better on the text-based questions. It’s just that they did not form a majority of any of the specific groupings considered in the Weintrop et al paper. For instance, Weintrop et al did not even break down students by whether they learned a block-based language in their AP CSP class prior to taking the exam! I’d definitely want to see that obvious dimension before I agreed with a statement that all students do better in one format.

        That said, judging only by the one example in the Weintrop et al paper of the modalities, it looks like what they are calling block-based is really the exact same text-based code, but layered with visual cues about the block structure. If that’s the case, of course most students do better in that format. And sure, there’s no harm in always providing that extra visual aid. So I think I reach the same conclusion as you: there is no reason not to standardize on the format that makes the structure of the code more obvious, since parsing the code isn’t the skill being assessed.

      • 6. Richard Ladner  |  July 13, 2020 at 5:20 pm

        To be clear: “does better” is not the same as “everyone does better.” The latter is what you said in your blog. The former could be interpreted as “on average, does better” or some stronger statistical way of saying it.

        • 7. Mark Guzdial  |  July 13, 2020 at 5:29 pm

          Yup, I see what you mean. Thanks for pointing it out — I’ve reworded to be more accurate.

  • 8. Briana Morrison  |  July 13, 2020 at 5:47 pm

    So it will be interesting when the data for CSP comes out for this year – no multiple choice exam so the pseudo-code will have been removed. Instead, this year’s results will be based entirely on the Create and Explore Tasks – which do not involve pseudo-code. But my guess is that the results for students will still be similar to those we’ve seen in the past – and won’t likely improve the performance of students in the lower quartiles.
    Also, CSP is the first course to be adopted in reverse – high school courses were in place before colleges created similar courses. So arguing that colleges don’t “teach in language X but test in language Y” is only because the preferred language isn’t standardized.

  • 9. kirkpams  |  July 14, 2020 at 3:10 pm

    “How many of these teach in language X, but then ask students to take their final exam in language Y which they’ve never used?”

    One of my worst experiences of grad school was my Programming Languages course. We learned about things like Lambda Calculus, real-time garbage collection, etc. In addition to taking the final for the class, we were required to pass a number of Qualifying Exams. Basically, they would add a few extra questions that would be graded pass/fail. The PL qual when I took it was 100% on Pi Calculus…which we had never seen or discussed. (Narrator’s voice: It didn’t go well.)

  • […] about how we have to change how we teach CS education to reduce inequity. I started this series with this post, making an argument based on race, but might also be made in terms of the pandemic. We have to […]

  • […] her PhD career, Amber has written about her experience of being a Black woman in CS. She taught me what intersectionality is about. I am grateful that she has been both a CS education researcher and activist during her PhD. I am […]

  • 12. A New Awareness - Computer Science Teachers Association  |  March 7, 2023 at 8:32 am

    […] structural inequities and in response to a pandemic: Starting a Four Part Series” […], and then I watched the conference keynote by Ruha Benjamin,  Professor of African […]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 10,186 other subscribers


Recent Posts

Blog Stats

  • 2,060,872 hits
July 2020

CS Teaching Tips

%d bloggers like this: