How White and Male the AP CS Really Is: Measuring Quality as well as Quantity
August 21, 2012 at 7:52 am 34 comments
Updated August 22: See note at bottom
We spent a significant amount of time this summer discussing with NSF our proposal to create an alliance around Expanding Computing Education Pathways (ECEP). One of the issues that we got pressed on was how to not just improve the numbers of women and members of under-represented minorities entering computer science, but to improve the quality of their learning and of their performance on metrics like the Advanced Placement Computer Science exam. Barbara Ericson started digging into the AP CS data at the College Board site, and found some pretty amazing things. I’m helping with some of the statistics (using my new “Computational Freakonomics” knowledge). We’re not sure what we’re going to do with this yet (SIGCSE paper, perhaps?), but Barb agreed that I could share some of the stats with you. The results in this post are Barb’s analysis of the AP CS results from 2006-2011, the years in which “Georgia Computes!” and CAITE were both in existence.
Nationally, here are the pass rates per year. The gap from the blue line at top and the red line below is explained by the gender gap. In 2011, the pass rate was 63.7% overall, 57.6% for females. The even larger gap from those two lines down to the rest is the race/ethnicity gap: 31.7% for Blacks, and 37.2% for Hispanics in 2011. I didn’t expect this: Hispanic females do statistically significantly better than Black females at passing the AP CS over this time frame (t-test, one-tailed, p=.01). (I’m using “Black” because that’s the demographic category that the College Board gives us. We are collapsing “Mexican American,” “Other Hispanic,” and “Puerto Rican” into the “Hispanic” category.) There’s still a big gap between the orange Hispanic line (37.2% in 2011) and the light blue Hispanic females line (25% in 2011).
While Hispanics are doing better than Blacks on AP CS, I was still surprised at this: No Hispanic female has scored a passing grade (3, 4, or 5) on the AP CS test in Georgia, Michigan, Indiana, South Carolina, or Alabama in the last six years. Only one Hispanic female has passed in Massachusetts in the same time frame. Why these states? ECEP is starting from Georgia and Massachusetts, next involving California and South Carolina, and we want to compare to states of similar size or similarly sized minority populations. We haven’t looked at all 50 states — the College Board doesn’t make it easy to grab these numbers.
The Black pass rate is quite a bit smaller than the Hispanic, in part because the participation rate is so low. Michigan has 1.4 million Blacks (out of 9.8 million overall population, so 14% Black), but only 2 Black men have passed the AP CS in the last six years. In 2011, 389 students took the AP CS in Michigan, only 2 of whom were Black. Only one Black female has even taken the AP CS in Michigan in the last six years. (No, she didn’t get a passing grade.)
Considering the population of the state is really important when considering these numbers. Last year, Georgia had 884 people take the AP CS Level A test (the most ever), 79 of whom were Black (about 9%). 17 passed. for a 21.5% pass rate. In contrast, California had a 51.7% pass rate among Black test-takers, 15 of the 29 test takers. That’s 29 test-takers out of 3101 AP CS Level A tests in California (0.9%)! California has an enormous test-taking population, but few Blacks and relatively few Hispanics (230 Hispanic test takers (49 female) out of the 3101 overall test takers). California has 37.6 million people, and 2.2 million Blacks (5.8%). Georgia has 9.8 million people, 2.9 million Blacks (30%). Bottomline: Georgia had many more Black test-takers than California, with a similarly-sized Black population. Georgia’s test-taking numbers aren’t representative of the population distribution overall (9% vs. 30%), but California’s are even more out-of-whack (0.9% vs. 5.8%).
Barb’s still digging into the numbers (e.g., to compare regionally, as well as by similarly sized). If we get ECEP, this is the first step — to know where we are, so we can measure how we do.
Updated August 22: When I wrote this up, I didn’t realize that Barb had created several datasets. She has data back into the 1990’s, but the dataset she gave me was just 2006-2011, the years in which our NSF BPC Alliances existed. So my claims of “ever” in the original post were too strong. We don’t know that the claims are wrong, but we haven’t actually checked back further than 2006 yet. My sincere apologies for mis-stating the scope of my claims! I’m glad that we discovered this problem when it’s just a blog post, not a paper submitted for publication. I’ve updated the text of the post to reflect the claims that I can actually make.
Entry filed under: Uncategorized. Tags: APCS, BPC, CAITE, computing education research, ECEP, GaComputes, NCWIT.
Computing Ed Research - Guzdial's Take 
1.
Ellen Zegura | August 21, 2012 at 8:14 am
Interesting data. Why isn’t the gender gap the difference between the male passing rate and the female passing rate? As you described and show it, the gap is between the overall passing rate and the female passing rate.
2.
Mark Guzdial | August 21, 2012 at 8:49 am
Yes, you’re right, Ellen. The gender gap is the difference between the male passing rate and the female passing rate. I wasn’t exact in what I wrote above. The explanation for the gap between the overall passing rate and the female passing rate is the difference between the gender passing rates. Overall, in 2011, 18.9% of AP CS test-takers were female.
3.
jkhuggins | August 21, 2012 at 8:24 am
One of the College Board’s standard “talking points” is that they want to see more students (of all backgrounds) simply participating in AP, regardless of whether or not they eventually pass the exam. While I’m sure that, in general, College Board would rather see students pass than fail, I’m wondering how they would react to pressure from folks like NSF to improve pass rates. This could lead to weird effects that we’ve seen occasionally at the high school level — e.g. schools that are so concerned about their AP pass rates that they *discourage* all but the few elite students from taking APCS.
4.
Mark Guzdial | August 21, 2012 at 8:51 am
This was an issue that we discussed repeatedly with NSF officials this summer, Jim. What does it mean to try to raise the pass rates on an international test which isn’t criterion referenced? What I do think is fair game is the difference in pass rates. Why should Black females be passing at such a low rate on AP CS, especially in comparison to other demographic groups?
5.
Jeff Rick | August 21, 2012 at 8:33 am
It would be interesting to compare these results to other subjects, such as AP Calculus and AP English. That would let you know how much of this effect is caused by general differences between men and women, between race and ethnicity, and the subject. It would also be nice to separate SES from race / ethnicity. The effect of SES on both rates of attempting and rates of passing could explain many of the other factors. For instance, high SES schools are likely the only ones providing AP computing classes. Those tend to be more white. That could also explain the difference between Georgia and California.
6.
Mark Guzdial | August 21, 2012 at 8:53 am
We’re trying to get those data, Jeff. Barb is working on a program to scrape data on the schools/districts where test-takers come from. We do know that AP CS is far more white and male than other subjects, but I don’t know much about demographic categories on those other tests. Just getting these data has taken Barb hours of work. As I said, the College Board doesn’t make this easy.
7.
Cecily | August 21, 2012 at 9:56 am
I am not sure I would believe district/school data if you had it. SES can vary widely in a school, and a school average would probably NOT mirror the AP test taking average for that school. If you can extract averages from programs that cater to first generation college students/low SES students, I would be more inclined to believe those comparisons. I suppose it is possible/maybe even probably that SES is more homogenous there in GA than it is in UT, but I would still be highly skeptical.
8.
Jeff Rick | August 27, 2012 at 4:28 am
My main point is that the selection bias of schools (based on SES) could be huge. For instance, Georgia has a 30% black population. As such, we’d like to see about a 30% black test population instead of 9%. How do we explain this? One explanation that I think is probable is access. What if we discovered that, of the schools that offered AP CS in GA, only 9% of their students were black. Then, the results suggest a serious access question. To increase the number of black participants, you need to get AP CS courses into predominantly black schools. If it is 30%, then access is not the main problem. If the data is accessible, it is worthwhile looking into it.
9.
Mark Guzdial | August 27, 2012 at 10:47 am
Barb told me a great story last week about a local, majority-AA high school. One of the students there is one of Barb’s summer camp assistants — he’s been going to our summer camps for years. He wanted AP CS to be offered at his school. The principal said that there wasn’t enough interest, but he’d be swayed by a petition of at least 30 names of students who would sign up for AP CS. The student came back with 60 names. The principal said it was too late, he couldn’t possibly get an AP CS teacher in time. Here’s the trick: There is a teacher at that school who has a Masters in CS, who has been taking our professional development for years. He could easily teach AP CS. What’s going on here? I’ve got two students who want to work with me this semester to make another run at getting permission to go into these schools and ask the question.
10.
Alfred Thompson | August 21, 2012 at 10:41 am
Is there data on Asian students? I’m just curious.
There is a lot of scary data there. I think that for one thing it highlights the risks of pushing quantity over quality which I see as a problem in society in general these days. I think (suspect) that a lot of the gender gap is in the way we teach CS. W’ll not everyone but most people.
11.
Mark Guzdial | August 21, 2012 at 11:35 am
There are — Barb didn’t disaggregate those. They’re not an under-represented minority in CS.
12.
Puzzled | September 2, 2012 at 12:27 am
That is a poor reason not to look at all of the data. You are only looking at what you are interested in finding? Also, combining Mexican American and other “hispanic” groups? Because Mexico and the rest are the same in what way? All ” blacks” are the same? Some one ho came over from somalia last year, and a family who are actually more european than african are still the same? Black americans who have history here going back 350 years are in a group with recent imugrants? If you really believe that race should be unimportant to how people are treated, and especially how they learn, then why would you ignorantly toss aside factors that are probably more important than skin color, or that two wildly different ” hispanic” groups are the “same” when it comes to learning. CS? The gender gap is an ” explanation”? Shoddy thinking, and that is where shoddy answers and shoddy policy will come from to muddy the waters in education more…..
13.
alanone1 | August 21, 2012 at 12:39 pm
Hi Mark
First, white males are not an “underrepresented minority in CS” either, and they are being shown, so I think for this and other reasons Asians should be shown.
“Divination” whether from chicken entrails or surveys like this usually suffers from lack of models, etc.
Another way to look at this would be to note that “white males” are not very interested in CS either, when the data are compared with the whole population. My instincts would be to try to find “parsimonious descriptors” of the white males who -are- interested, and see how they compare with those who aren’t.
One thing that is likely is that quite a bit of practice — even with poor programming languages — is required to be fluent enough to be flexible with questions. School is a terrible place to do this practicing. So another correlation I’d look for is to see what % of kids who pass did so primarily because of school, or because they had already put in years of “serious hobbying”.
(Those who were poor enough or interested enough to build their own ham radios in the 40s and 50s wound up with a terrific advantage in many high school and college areas from that experience.) There are more parallels and analogies lurking here.
Cheers,
Alan
14.
Mark Guzdial | August 21, 2012 at 12:49 pm
Hi Alan,
The data exist for Whites, Asians, and males, but that’s not what what NSF was asking us to explore. In the graph in this post, I’m showing overall pass rates, then pass rates for the BPC demographic groups. I’m not hiding White, Asian, or male data — it’s simply not part of the question we were asked.
I’m not understanding the “divination” comment. These aren’t survey results. These are simply counts of test-takers and their scores from AP CS, based on self-reported (is that what you meant by survey?) demographic categories. I’m not predicting the future here. I’m not even trying to explain these results. Barb and I are trying to find experts who can help us develop explanations for these results. I absolutely agree that we need models.
These data are useful as a spotlight. I was born in Detroit and grew up in Michigan. I had no idea that there so few African-American students taking AP CS there. The raw population numbers are similar between Michigan and Georgia, yet the results are so different. No, I don’t understand why. These data are phenomena that suggest things to explore.
The variables you identify as interesting to explore certainly are worth investigating.
Cheers,
Mark
15.
Joel | August 21, 2012 at 3:47 pm
Hi Mark,
Interesting post — I look forward to reading the paper!
Regarding the difference between Michigan and Georgia: Is it not the case that in Georgia the AP CS course satisfies one the state’s No Child Left Behind graduation requirements? AP CS does not satisfy any graduation requirement in Michigan, so I’m frankly surprised that there were as many Michigan test-takers are there were. We all really need to become advocates for http://www.computinginthecore.org.
-Joel.
16.
Mark Guzdial | August 21, 2012 at 3:55 pm
Hi Joel,
Nope, AP CS counts as a science towards high school graduation in Georgia, but I don’t think CS counts towards any NCLB areas anywhere. There are five states where AP CS counts towards high school graduation.
17.
Joel | August 21, 2012 at 5:06 pm
I guess I was misinformed. At least it counts towards something in GA; I don’t think it counts as anything — science or math — in Michigan, and that could be a factor in the difference why fewer students in MI take the AP CS test compared to GA. There’s little incentive for MI students to take the course.
-Joel.
18.
Mark Guzdial | August 21, 2012 at 5:35 pm
389 test-takers is not a small number. Michigan has a lot of test-takers. Just very few Black and Hispanic test-takers.
19.
Mark Guzdial | August 21, 2012 at 6:26 pm
I checked in the “Running on Empty” report. The states that accept AP CS as part of Science (Georgia) or Math are Missouri, New York, North Carolina, Oklahoma, Oregon, Rhode Island, Texas, and Virginia. Notice that California, with over 3K test-takers, is not on the list, nor Massachusetts, which had 750 test-takers in 2011 and is smaller than Michigan.
20.
Alex Gaynor | August 21, 2012 at 1:07 pm
This data seems to be taking as an underlying assumption that the AP CS test measures some real phenomenon, and that that phenomenon is important. To what extent have you investigated whether or not the AP CS course is good at all? That is, should we care about these rates, or are they mearly part of a pattern of poor education, and measuring it poorly.
21.
Mark Guzdial | August 21, 2012 at 1:21 pm
Alex, I do think AP CS can be a good course. I am pretty familiar with it. I’ve had one child take it, and another child taking it now. My wife has taught it, and is regularly a Reader for it. But I don’t think that’s really the critical question here.
If AP CS is inherently a bad course, then it’s bad for Whites, Blacks, and Hispanics alike, isn’t it? If “passing” is a poor measure, then shouldn’t it be a “poor” measure across different demographic groups? Why are there such statistically-significant differences? Is there something about the course that makes it effective or not for different demographic groups?
Maybe it’s really about accessibility? Or appeal? As I said to Alan, I don’t have explanations for these results. The key point is that they are dramatically different. As Jane Margolis et al. pointed out in “Stuck in the Shallow End,” access to computing education means access to some of the best-paying jobs in our economy. Lack of access can be preventing social mobility. Is this a question of lack of access? I don’t know, but these results suggest a big difference that is worth investigating and trying to explain.
22.
Puzzled | September 2, 2012 at 12:44 am
Ouch, more shoddy thinking. Alex is getting at weather or not AP computer science makes any difference in the real world. I ask myself this question all of the time with all of the resources put into “STEM” now. Legislators, school boards, principles, and the like back flashy programs that look good, regardless of their real world effect. We want more Americans is science and engineering. Is taking resources out of higher education , while pushing things like AP CS really worth anything? Why is computer science an AP only subject at many schools? And I do not mean “IT” , but actual programming classes? That is what allen is getting at. How many of the students in groups you are talking about, of any race or gender as that is your main concern, go on to become programmers, scientists, or engineers of any kind. That would be useful to know.
23.
Joy Buolamwini | September 25, 2013 at 9:58 am
I took AP Computer Science as a high school senior in Tennessee and scored a 5 in 2008. I loved it even though there were only 3 of us in it, and it was grouped with another class. The teacher (an amazing young woman ) would alternate between two sides of the room. I would spend my lunch break working on programming robots in java.
That class really gave me confidence in my abilities in Computer Science and time to explore / participate in programming challenges. The major factor in deciding which degree to pursue came down to what excited me the most given that it seemed I would be able to do well across a number of disciplines.
I was fortunate to go to Georgia Tech and study Computer Science. I was awarded a full scholarship in part because of my programming skills and entrepreneurial endeavors that led to early recognition. At Tech, I found the environment I needed to thrive. I was also a bit ahead of the curve because I had AP Computer Science in high school. I discovered there that Computer Science could be compassionate, a helping field. I was thrilled to work on Simon the robot, contributing to a code base derived from MIT used with robots I had grown up watching on PBS!
I was even more excited because we were exploring how social robotics could be employed in the diagnosis of autism. Still, I wanted to work on something where my programming skills would have immediate impact. This led me to Ethiopia to develop mobile surveying tools to help eradicate blinding Trachoma in one of the most endemic regions in the world in 2011. 3 years after taking AP Computer Science in highschool, I was using skills that were nurtured in that class as a technical consultant for the Carter Center at age 21. If interested, you can read our publication “A Novel Electronic Data Collection System for Large-Scale Surveys of Neglected Tropical Diseases ” here http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0074570
I would be classified as a “black female”, but more importantly I am an individual who was fortunate to have the right environment to foster my curiosity. I have a BS in Computer Science with Highest Honors from Georgia Tech. I have worked in Silicon Valley, been the Chief Technology Officer / Founder for multiple start-ups, and I recently completed a Fulbright Fellowship in Zambia where I established a Software Development and Training Organization.
http://vimeo.com/73768147 – Progress Video
http://vimeo.com/66479996 – Motivation Behind the Work
http://vimeo.com/74010152 – Song on building mobile applications
http://vimeo.com/72586980- Preview of Media- platform to encourage pursuing passions worldwide
Article Feature on Women’s Rights App Developed
http://women2.com/zamrize-helps-launch-womens-rights-app-wrapp-in-zambia/
Next month, I will move to the UK to begin my studies as a Rhodes Scholar pursuing two masters degrees – one in African Studies focused on fostering Innovation Ecosystems and enabling Technology Entrepreneurs in Frontier Markets. The other will be in Visual Material Museum Anthropology with a focus on empowering media.
What makes a difference in the real world are people who believe in pursuing their passions and those who provide the education, support, and encouragement to help dreams become reality. That is what I had from Ms. Connell when I took AP Computer Science, that is what I had at Georgia Tech with incredible professors and advisors (Mark and Barb included), and that it what I am working on making more accessible to other people.
It’s not about “promoting STEM”. It is about equipping ALL of humanity with tools and problem solving abilities to address the grand challenges of our time.
24.
Mark Guzdial | September 25, 2013 at 11:01 am
Thanks, Joy!
25. Quantity Is Not Enough - Microsoft in Education blog - Site Home - TechNet Blogs | August 23, 2012 at 3:30 pm
[…] have been looking at the statistics for AP CS test takers and finding some interesting things. (see How White and Male the AP CS Really Is: Measuring Quality as well as Quantity ) It appears that not only is there a gap between white and minority students it taking the exam […]
26. Quantity Is Not Enough | UpSearchLearn | August 25, 2012 at 6:56 pm
[…] have been looking at the statistics for AP CS test takers and finding some interesting things. (see How White and Male the AP CS Really Is: Measuring Quality as well as Quantity ) It appears that not only is there a gap between white and minority students it taking the exam […]
27.
Don Davis (@gnu_don) | September 30, 2012 at 5:01 pm
I was rereading this. I would be interested to know Latina (and Latino) scores without the collapse i.e. “Mexican American,” “Other Hispanic,” and “Puerto Rican” as these varying demographic groups fair significantly differently when comparing overall academic success.
Perreira, K., Harris, K., & Lee, D. (2006). Making it in America: High school completion by immigrant and native youth. Demography, 43(3), 511-536.
28.
Research questions on MOOCs: Who’s talking, who’s completing, and where’s the teaching? « Computing Education Blog | December 14, 2012 at 10:54 am
[…] or CS. Is that true for MOOCs as well? What are the demographics of completers in a MOOC? We already have a problem of CS being mostly White/Asian and Male. (Lots of reasons why that is a problem: From equity and fairness, to tapping into the fastest […]
29.
Meet Kyla McMullen: The 1st Black Woman to Graduate With a PhD In CS at U-Michigan « Computing Education Blog | January 17, 2013 at 10:57 am
[…] I just learned this fact at the NSF BPC/CE21 meeting from Jane Margolis’s talk. This last Fall 2012, the first female African-American CS PhD graduated from the University of Michigan. Michigan is 14% African-American. University of Michigan is a state institution. Really? 2012? I guess it’s not too surprising, when we know from the AP CS data that I talked about last year that few African-Americans get access to c…. […]
30.
First, Do No Harm: Inequality in American Education Will Not Be Solved Online « Computing Education Blog | January 21, 2013 at 1:44 am
[…] don’t think is being discussed enough. I predict that computer science MOOC completers are even more white and male than in existing computing education. Replacing more face-to-face CS courses with MOOCs may be […]
31.
Disaggregating Asian-American educational attainment | Computing Education Blog | June 25, 2013 at 1:06 am
[…] Computer science is mostly white or Asian and male. We have lots of data to support that. What I didn’t realize was how sub-groups within Asian-American differ markedly in their educational attainment. A new report from NYU and ETS disaggregates the data, and below is the startling graphic that Rick Adrion pointed me to. […]
32.
Rupert Murdoch wants to teach your kids in an AP CS MOOC | Computing Education Blog | July 24, 2013 at 1:41 am
[…] more available to more students. Still, I’m not excited too about a MOOC to teach AP CS. AP CS is already overwhelmingly white and male. The demographic data from existing CS MOOCs is even more white and male than our face-to-face […]
33.
Where are the black students in STEM in Georgia? | Computing Education Blog | September 25, 2013 at 1:33 am
[…] old) in high school mathematics. APS is Atlantic Public Schools. I live in Dekalb county. No wonder we can’t get more Black students into AP CS, if we can’t get past Sophomore year […]
34. Disaggregating Asian-American educational attainment Hmoob Ib Society | February 17, 2014 at 10:11 pm
[…] Computer science is mostly white or Asian and male. We have lots of data to support that. What I didn’t realize was how sub-groups within Asian-American differ markedly in their educational attainment. A new report from NYU and ETS disaggregates the data, and below is the startling graphic that Rick Adrion pointed me to. […]