“Georgia Computes!” (read: Barb) impact on CS1 in Georgia
September 1, 2010 at 2:10 pm 13 comments
I’m working on my (er, overdue) annual report to NSF on “Georgia Computes!” and just found this pretty remarkable tidbit. One of the big projects we’re doing as part of our two year extension on “Georgia Computes!” is to conduct a survey of all CS1 students in the whole state — what’s our real WDF rate? where are these students coming from? 29 schools in the University System of Georgia have computing programs, and 19 of those participated in our survey.
Here’s the factoid. Barbara (through the Institute for Computing Education) has now taught CS high school teachers at 152 of the 422 (about 36%) high schools in Georgia. Of the 1,349 students taking CS1 last year and participating in our study, 64% (865) went to a public high school in Georgia. (I thought that that was a surprisingly low percentage.) But here’s the kicker: 58% of those kids (498) came from the schools where Barbara trained the teachers.
If we’re only in 36% of the schools, but those are the ones generating 58% of the (responding) CS1 students, I’d say that Barb’s workshops are having a disproportionate impact on CS1 in the state.
Entry filed under: Uncategorized. Tags: GaComputes.
13 Comments Add your own
Leave a comment
Trackback this post | Subscribe to the comments via RSS Feed
Computing Ed Research - Guzdial's Take 
1.
gasstationwithoutpumps | September 1, 2010 at 2:40 pm
What De F** is WDF? Wikipedia says
“WDF may refer to:
* Windows Driver Foundation
* World Darts Federation”
The freedictionary has
WDF Web Design Factory (Czech Republic)
WDF Windows Driver Foundation (Microsoft)
WDF World Darts Federation
WDF Washington Department of Fisheries
WDF Waste Derived Fuel
WDF Wireless Data Forum
WDF World Development Federation
WDF Watershed Development Fund
WDF Women’s Development Federation
WDF Worth Dying For (band)
WDF WhenDarknessFalls (gaming)
WDF Women’s Development Foundation
WDF WhenDarknessFalls (Counter-Strike server network)
WDF World Dental Federation
WDF Wales Debating Federation
WDF Write Disturb Fault
WDF Wholesale Distribution Facility
none of which seem relevant.
2.
Mark Guzdial | September 1, 2010 at 2:45 pm
WDF is the rate of students withdrawing from the class or earning a failing grade (D or F). It’s a standard measure used in higher education. It’s been studied for CS courses, too, e.g., see Bennedsen and Caspersen, June 2007.
3.
gasstationwithoutpumps | September 1, 2010 at 3:06 pm
Thanks for the explanation. I’ve been in higher education for 28 years, and never seen the acronym WDF before. I’ve seen the concept with various names (include NP for “not passing”).
You need to be very careful about acronyms, as they are rarely as universally understood as the people who use them believe.
4.
gasstationwithoutpumps | September 1, 2010 at 2:42 pm
“If we’re only in 36% of the schools, but those are the ones generating 58% of the (responding) CS1 students, I’d say that Barb’s workshops are having a disproportionate impact on CS1 in the state.”
Or that your sampling methodology is biased, and you are preferentially getting responses from those classes where you have had personal contact.
5.
Mark Guzdial | September 1, 2010 at 3:00 pm
While we certainly could have made mistakes, I don’t immediately see where we could have made this one. We asked all 29 post-secondary schools to participate, and 19 did. Sure, those could be the 19 who most want to work with us. But then we ask their CS1 students what high school they went to. How we could have somehow biased the result toward high schools that worked with us? It’s certainly not the case that our high schools were somehow given preferential treatment in applications. I’m not making a claim about the 10 post-secondary schools that didn’t participate. Where’s the potential for bias?
6.
gasstationwithoutpumps | September 1, 2010 at 3:13 pm
The sampling strategy does seem to be as unbiased as you can make it. Which high schools you worked with may have been biased (going first to the high schools that provide a large number of students to the colleges would even be a sensible strategy).
Even if you offered the training to all high schools, those that teach a lot of students going on to take CS1 are more like to take up the training. So the bias is likely to be in choice of schools getting the training, rather than in the sampling of students.
This does not negate your conclusion that the workshops are having a “disproportionate” impact on the survey, but does provide a plausible non-causal explanation for the correlation.
7.
Mark Urban-Lurain | September 1, 2010 at 3:39 pm
Congratulations on the data in general, but along the line of other comments “Correlation ain’t causation.”
Perhaps Barbara’s at schools that are receptive because they value CS because they are better preparing their students in general and their receptivity to CS and Barb’s training is a function of those cultural values.
8.
Mark Guzdial | September 1, 2010 at 4:16 pm
I don’t think I claimed any causation in my post. I claimed impact. Ours is a pipeline effort. We need to be able to show that our efforts at the secondary school level are impacting students who enter the post-secondary level. I think our results show that. A surprisingly high percentage of CS1 students in Georgia were taught in schools where there were teachers we trained. We’d need other data to show that we influenced their decision to study CS, or that we made them into better CS students. Right now, I’m just establishing the influence on the pipeline.
9.
gasstationwithoutpumps | September 1, 2010 at 6:58 pm
Sorry about that. Two of us interpreted “impact” as implying causation, as I think many reviewer would. Having trained the teachers provides the potential for influence or impact, but does not establish it. I would have used a word like “access” to imply that there was the potential for impact.
10.
Mark Guzdial | September 2, 2010 at 9:10 am
Good thing this is a blog and not a conference publication! 🙂
11.
gasstationwithoutpumps | September 2, 2010 at 9:49 am
Yeah, I’m not always so nit-picky on blogs, but I thought the feedback might help you with your annual report, which either no one will read, or someone will read with the goal of finding nits to pick.
12.
Alfred Thompson | September 6, 2010 at 10:26 pm
I’m pretty ready to believe that Barb and the Georgia Computes program is having an impact. I’ve heard enough teachers talk about being inspired and influenced by the program. So my question is how do we get more programs like this and can we find more Barbs? The latter may be harder than the former.
13.
Mark Guzdial | September 7, 2010 at 2:33 pm
I agree that cloning is out. 🙂 I wonder if this is related to the discussion we’re having about the role of research in the university. If more faculty were more focused on teaching than research, would we have more programs like “Georgia Computes!” and more teaching-centered faculty like Barb? Is the cost of more-research that we have less-education-focus, and if we lessened research, we might get more education?