A careful model-based analysis of AP CS 2013 exam data: CS commenters need to learn statistics

There has been a lot of media attention to Barb’s analysis of AP CS 2013 exam data, but not all of it has been well-informed.  We both really enjoyed reading the Gas Station Without Pumps analysis, quoted and linked below.  Not only is it a careful, model-based analysis, but it’s a nice explanation too.  I learned more about how to measure under-representation — recommended reading.

There are states that do have significant under-representation of women: for example, Utah had 103 test takers, only 4 of whom were women. With an expected number of about 51.5, this is p<1.4E-16. Even with 51× multiple hypothesis correction, this under-representation is hugely significant.  Looking nationwide, total counts were 5485 female test takers out of 29555 total test takers.  That’s p< 1.4E-1677. The highest percentage of female test takers was in Tennessee, with 73 out of 251, which is  p< 2.6E-7, again highly significant.

Tennessee also had a high proportion of black test takers with 25 out of 251.  With an expected number of 42.12, this is p<0.003 (still significantly under-represented).  To see if black students were under-represented nationwide, one would have to add up the expected numbers for each state and see how the actual number compared with the expected number.

via CS commenters need to learn statistics | Gas station without pumps.