How White and Male the AP CS Really Is: Measuring Quality as well as Quantity
Updated August 22: See note at bottom
We spent a significant amount of time this summer discussing with NSF our proposal to create an alliance around Expanding Computing Education Pathways (ECEP). One of the issues that we got pressed on was how to not just improve the numbers of women and members of under-represented minorities entering computer science, but to improve the quality of their learning and of their performance on metrics like the Advanced Placement Computer Science exam. Barbara Ericson started digging into the AP CS data at the College Board site, and found some pretty amazing things. I’m helping with some of the statistics (using my new “Computational Freakonomics” knowledge). We’re not sure what we’re going to do with this yet (SIGCSE paper, perhaps?), but Barb agreed that I could share some of the stats with you. The results in this post are Barb’s analysis of the AP CS results from 2006-2011, the years in which “Georgia Computes!” and CAITE were both in existence.
Nationally, here are the pass rates per year. The gap from the blue line at top and the red line below is explained by the gender gap. In 2011, the pass rate was 63.7% overall, 57.6% for females. The even larger gap from those two lines down to the rest is the race/ethnicity gap: 31.7% for Blacks, and 37.2% for Hispanics in 2011. I didn’t expect this: Hispanic females do statistically significantly better than Black females at passing the AP CS over this time frame (t-test, one-tailed, p=.01). (I’m using “Black” because that’s the demographic category that the College Board gives us. We are collapsing “Mexican American,” “Other Hispanic,” and “Puerto Rican” into the “Hispanic” category.) There’s still a big gap between the orange Hispanic line (37.2% in 2011) and the light blue Hispanic females line (25% in 2011).
While Hispanics are doing better than Blacks on AP CS, I was still surprised at this: No Hispanic female has scored a passing grade (3, 4, or 5) on the AP CS test in Georgia, Michigan, Indiana, South Carolina, or Alabama in the last six years. Only one Hispanic female has passed in Massachusetts in the same time frame. Why these states? ECEP is starting from Georgia and Massachusetts, next involving California and South Carolina, and we want to compare to states of similar size or similarly sized minority populations. We haven’t looked at all 50 states — the College Board doesn’t make it easy to grab these numbers.
The Black pass rate is quite a bit smaller than the Hispanic, in part because the participation rate is so low. Michigan has 1.4 million Blacks (out of 9.8 million overall population, so 14% Black), but only 2 Black men have passed the AP CS in the last six years. In 2011, 389 students took the AP CS in Michigan, only 2 of whom were Black. Only one Black female has even taken the AP CS in Michigan in the last six years. (No, she didn’t get a passing grade.)
Considering the population of the state is really important when considering these numbers. Last year, Georgia had 884 people take the AP CS Level A test (the most ever), 79 of whom were Black (about 9%). 17 passed. for a 21.5% pass rate. In contrast, California had a 51.7% pass rate among Black test-takers, 15 of the 29 test takers. That’s 29 test-takers out of 3101 AP CS Level A tests in California (0.9%)! California has an enormous test-taking population, but few Blacks and relatively few Hispanics (230 Hispanic test takers (49 female) out of the 3101 overall test takers). California has 37.6 million people, and 2.2 million Blacks (5.8%). Georgia has 9.8 million people, 2.9 million Blacks (30%). Bottomline: Georgia had many more Black test-takers than California, with a similarly-sized Black population. Georgia’s test-taking numbers aren’t representative of the population distribution overall (9% vs. 30%), but California’s are even more out-of-whack (0.9% vs. 5.8%).
Barb’s still digging into the numbers (e.g., to compare regionally, as well as by similarly sized). If we get ECEP, this is the first step — to know where we are, so we can measure how we do.
Updated August 22: When I wrote this up, I didn’t realize that Barb had created several datasets. She has data back into the 1990’s, but the dataset she gave me was just 2006-2011, the years in which our NSF BPC Alliances existed. So my claims of “ever” in the original post were too strong. We don’t know that the claims are wrong, but we haven’t actually checked back further than 2006 yet. My sincere apologies for mis-stating the scope of my claims! I’m glad that we discovered this problem when it’s just a blog post, not a paper submitted for publication. I’ve updated the text of the post to reflect the claims that I can actually make.