The goal of the first CS course should be to promote confidence if we’re going to increase diversity in CS: Paying off on a bet

This should be a thing: If you make a public bet on Twitter, and lose, you should have to write a blog post explaining how you got it wrong.

Let me set the stage for the bet. There are studies suggesting that the Advanced Placement (AP) Computer Science A exam has a significantly different impact on students’ majors than other AP exams. (For non-US readers: AP tests provide an opportunity for secondary school students to earn post-secondary school credit.) AP CS A exam-takers are more likely to go on to take more CS courses or become a CS major — more likely than, say, students taking AP Calculus or AP US History exams to become mathematics or history majors. But does that extend to the newer AP CS exam, AP CS Principles? AP CS Principles was designed to be less about the kinds of programming that CS majors do in their first year, and more about a broader understanding of computing and its effects (see College Board site here). There were several of us talking about this in the Spring. On April 1, 2019, I tweeted to Jeff Forbes (see link): “I bet that AP CS Principles has no impact on CS or STEM majors. It’s such a different course (eg doesn’t map to CS courses on most campuses).” He took that bet, and he was right. A study released by the College Board shows that there is a causal relationship between taking AP CS Principles and majoring in CS in undergraduate (see report link here). The impact is large. Overall, students who take AP CS Principles are three times more likely to major in computer science in college. AP CSP students who are female are twice as likely to major in CS.

I wasn’t crazy for expecting that AP CS Principles would not have such a big impact on recruitment and retention. At SIGCSE 2020, Joanna Goode and co-authors published a paper showing that (see blog post link here) AP CS Principles is effectively recruiting much more diverse students than the AP CS A course (which is mostly focused on Java programming). But, AP CS A students end up with more confidence in computing and much more interest in computing majors and tech careers. ACM TOCE in 2019 published a paper using NCWIT Aspirations award winners (see blog post link here) showing that taking the CS Advanced Placement A exam was one of the best predictors of persistence three years after the high school survey in both CS and other technology-related majors. The TOCE paper authors made a particular emphasis on the importance of programming: “It seems that involvement in general tech-related fields other than programming in high school does not transfer to entering and persisting in computer science in college for the girls in our sample.”

So I had good reason to believe that non-programming-intensive courses might not have a big impact on recruitment into the CS major and retention. But I accept the evidence that I was wrong. What else is going on?

Here’s another recent piece of evidence that supports Jeff’s belief that AP CS Principles (and classes like that) could be having a big impact. Philip Boda and Steve McGee have a paper coming out in SIGCSE 2021 showing that the Exploring CS course (see website here) is having a significant impact in driving up AP CS A participation and diversity (see paper here), which continues to have a large impact on majoring in CS. Exploring CS, like AP CS Principles, is de-emphasizes programming in favor of a broader understanding of computing and helping students to see themselves as successful at CS.

Neither of these papers offers an explanation for why AP CSP or ECS is having this positive impact. They’re both large scale quantitative stories. You’d think that I might have learned my lesson from this last failed bet. Nah. I’ve got guesses. My guesses might be wrong, as they were in this case. I’m a post-positivist. I don’t think we’ll ever get to the place where we know the complete truth, but we should keep trying, keeping making hypotheses, and we can keep getting closer.

Here’s my hypothesis for what’s going on, stated as a prediction:

A first course will be successful at promoting recruitment into CS as a major or career and at retaining students in CS if it increases students self-efficacy about programming tasks.

The critical part is for students to increase their confidence that they can be successful at programming tasks. AP CS A easily does this, which is why it has such great results in recruitment and retention. Not all classes or experiences do, as the NCWIT study suggests. AP CS Principles and ExploringCS are all about increasing student confidence, helping them to see themselves as successful at computing. I don’t know how little programming a student needs to do to increase their self-efficacy. Maybe it’s enough to see programs and what programming is about.

Recent research in computing education has been focusing on self-efficacy as one of the most important variables predicting student recruitment and retention in CS. Alex Lishinski and his co-authors showed that self-efficacy had different relationships for female and male CS students (see paper link here) and that programming projects influenced students’ sense of self-efficacy, which in turn influenced performance in the CS class (see paper link here). Jamie Gorson and Nell O’Rourke found (in an ICER 2020 paper that I blogged about here) that CS students had deflated self-efficacy, in part, because they had unreasonable expectations of what real programmers do. Dr. Katie Cunningham, soon to be a post-doc joining Nell’s lab, showed in her dissertation how students simply give up on programming tasks that they don’t think that they’ll be successful at (see blog post on Katie’s dissertation defense). Self-efficacy is likely an important variable in recruitment and retention, particularly of female students, and it’s one that we can manipulate with better designed education.

I’m not the first person to to suggest this relationship. In a study with over 5 million participants, Peter Kemp and colleagues suggest that female participation in secondary school computer science in England is being negatively impacted because of female students’ low self-efficacy in CS — and that this is because of the CS classes (see paper link here). In England, curriculum in Information and Communications Technology is being faded out in favor of a Computer Science focus. They write in their paper “Female Performance and Participation in Computer Science: A National Picture”:

The move to introduce CS into the English curriculum and the removal of the ICT qualifications look to be having a negative impact on female participation and attainment in computing. Using the theory of self-efficacy, we argue that the shift towards CS might decrease the number of girls choosing further computing qualifications or pursuing computing as a career. Computing curriculum designers and teachers need to carefully consider the inclusive nature of their computing courses.

I made my bet because I thought that the programming-light focus of AP CS Principles (or even ExploringCS) would have less of an impact on CS recruitment and retention than the programming-intensive focus of AP CS A. I now believe I was wrong. I would now bet that the amount of programming probably isn’t the critical variable at all. It’s whether students come out of these courses saying, “I can do this. I can program.” That’s the critical variable for recruitment and retention that I believe AP CS Principles and Exploring CS are influencing successfully.