In my most recent recent Blog@CACM post on last month’s ACM Ed Council meeting, I mentioned that I gave a talk about the differences between computing education research and engineering education research (EER) and physics education research (PER). Let me spell these out a bit here.
The context was a panel on how to grow computing education research (CER). We were asked to consider the issue of getting more respect for computing education research (an issue I’ve written on before). I decided to explore the characteristics of CER that are important and that are not present in EER or PER. Engineering Education Research (EER) and Physics Education Research (PER) are better established and more well-respected in the United States. But I’ve come to realize that CER has characteristics that are different from what’s in EER and PER.
Engineering Education Research
I came to a new understanding of EER because of a cross-campus STEM Education Research seminar that we’re holding at Georgia Tech this semester. It’s given me the opportunity to spend a couple hours each week with people who publish in Journal of Engineering Education (see here), review for them, and edit for them. JEE is generally considered to top EER journal.
If you’re not familiar, engineering education research is a big deal in the United States. There are well-funded engineering education research centers. There are three academic departments of EER. It’s well-established.
In one of the early sessions, we talked about the McCracken Study (Mike McCracken has been coming to the sessions, which has been great), where an experimental assignment was used in five classes in four countries. Are there similar studies in EER? Our EER colleagues looked at one another and shrugged their shoulders. For the most part, EER studies occur in individual classes at individual institutions. Laboratory studies are rare. International collaborations are really rare.
I started digging into JEE. The last issue of JEE only had papers by American authors from American institutions. I’m digging further back. My colleagues are right — international authors and collaborations are unusual in JEE.
In contrast, I don’t think that the ACM Transactions on Computing Education has ever had an only-American issue. Our ICER conference is not even American-dominated. The ICER 2014 best paper award went to a paper by Leo Porter (American) who worked with Raymond Lister (Australian) using data collected from Daniel Zingaro’s classroom (at U. Toronto in Canada) to address a theory by Anthony Robins (New Zealand). We use classroom studies, laboratories studies, and frequently use multi-institutional, multi-national (MIMN) collaborative studies (and study how to conduct them well).
Physics Education Research
At the January workshop on CER that Steve Cooper organized (paper to appear in CACM next month — it’s where Eric Roberts gave a keynote that I wrote about here), Carl Wieman was the opening keynote speaker. He talked about the hot issues in physics education research.
After his talk, he was asked about how physics education researchers were dealing with the gender skew in physics and about improving access in K-12 to quality educational opportunities. If you look at Brian Danielak’s visualization of AP CS test data, you’ll see that CS is the most gender-skewed, but Physics follows closely after. (Click on the picture to get a bigger version, and look at the lower left-hand corner.)
Carl said that gender diversity just wasn’t a priority in PER. I dug into the PER groups around the US. From what I could find, he’s right. Eric Mazur’s group has one paper on this issue, from 2006 (see here). I couldn’t find any at U. Washington or at Boulder. There probably is work on gender diversity in physics education research, but it certainly doesn’t stand out like the broadening participation in computing effort in the United States (see papers listing from Google Scholar). The January workshop really brought home for me that a key characteristic of CER, particularly in comparison with PER, is an emphasis on broadening participation, on social justice, on improving the diversity of the field, and guaranteeing access to quality educational opportunities for all.
I don’t have a deep bottom-line here. It was only a few minute talk. My exploration of EER and PER gave me a new appreciation that CER has something special. It’s not as big or established as EER or PER, but we’re collaborative, international, working on hard and important problems, and using a wide variety of methods, from in-classroom to laboratory studies. That’s pretty cool.
April Heard at Georgia Tech built this map for us about where AP CS is taught in the state of Georgia. Some of it is totally to be expected. Most of the schools are in the Atlanta region, with a couple in Columbus, a handful in Macon, and a few more in Augusta and Savannah area.
But what’s disappointing is that huge swath in the south of the state with nothing. Not a single school south of Columbus and west of Brunswick. In terms of area, it’s about 1/3 of the state. Albany is home to Albany State University, the largest HBCU in Georgia. No AP CS at all there. And Georgia is one of the top states for having AP CS.
Sure, there might be some non-AP CS teachers in South Georgia, but we’re talking a handful. Not double, and certainly not a magnitude more than AP CS.
I suspect that much of the US looks like this, with wide stretches without a CS teacher in sight. April is continuing to generate these maps for states that we’re working with in ECEP. Here’s California, with big empty stretches.
Tom McKlin just generated this new map, which overlays the AP CS teacher data on top of mean household income in a school district. The correlation is very high — districts with money have AP CS, and those that don’t, don’t.
A few months ago, I wrote a post on Constructionism for Adults. I argued that we want constructionist learning for adults, but most constructionist learning environments are aimed at children. I suggested that adults have three challenges in constructionism that adults don’t have:
- Adults have a “face” (in the Goffman sense) that they want to preserve.
- Adults don’t necessarily have expertise in an area, but as adults, they are presumed to have expertise.
- Adults have less free time and more responsibilities than children.
I mentioned in that post that I was learning to play the ukulele, and that that experience was leading to new insights for me about adult education. I’m going to continue to use my ukulele learning to suggest a way to create constructionist learning opportunities for adults.
Legitimate Peripheral Participation for Adult Learning
From this point of view a very remarkable aspect of the Samba School is the presence in one place of people engaged in a common activity – dancing – at all levels of competence from beginning children who seem scarcely yet able to talk, to superstars who would not be put to shame by the soloists of dance companies anywhere in the world. The fact of being together would in itself be “educational” for the beginners; but what is more deeply so is the degree of interaction between dancers of different levels of competence. From time to time a dancer will gather a group of others to work together on some technical aspect; the life of the group might be ten minutes or half an hour, its average age five or twenty five, its mode of operation might be highly didactic or more simply a chance to interact with a more advanced dancer. The details are not important: what counts is the weaving of education into the larger, richer cultural-social experience of the Samba School.
So we have as our problem: to transfer the positive features of the Samba School into the context of learning traditional “school material” — let’s say mathematics or grammar. Can we solve it?
— Seymour Papert, “Some Poetic and Social Criteria for Education Design” (1975)
What Seymour was seeing in Samba schools is what Jean Lave and Etienne Wenger called a community of practice. My colleagues Jose Zagal and Amy Bruckman have a wonderful paper describing how Samba schools are a form of a community of practice, and how that model appears in the Computer Clubhouses that Yasmin writes about in her new book. In their influential 1991 book Situated learning: Legitimate Peripheral Participation, Lave and Wenger described several examples for how learning occurs in everyday settings, often with adults. Lave and Wenger point out
- There are the midwives who train their daughters who start out just going-along to help mother at births.
- There are the tailors who start out by delivering fabric and pieces between shops, and in that way, get to see many shops — without actually doing tailoring but still doing something useful to being a tailor.
- There are the attendees at Alcoholics Anonymous meetings who learn to tell their stories through listening to role models and getting feedback from others.
There are some key elements to these stories:
- Newcomers start out doing something useful, but on the periphery of the community — hence, legitimate peripheral participation. Jose and Amy point out that successful Samba schools are flexible to outsiders (anyone can become a newcomer).
- Everyone sees practice (story-telling, being a tailor, helping a birth, dancing at Samba school) at different levels. Jose and Amy talk about having a diversity of membership (socio-economic, age, race, and expertise) and that there are events for public to exhibit practice.
- There are some members of the community of practice who are clearly at the center. They serve as role models for others. From the newcomers to those practicing but not yet central, everyone strives to learn to become like those at the center of the community of practice.
Ukulele Meet-up As Samba School and Community of Practice
In my quest to learn to play ukulele, I’ve joined the Southeast Ukers, a group of ukulele players in Atlanta. I was fortunate to know a Uker who invited me to a meet-up. A meet-up is the experience I’ve had that is closest to how I understand a Samba school.
The meet-up is held at a local Hawaiian BBQ restaurant at 2 pm on the 1st and 3rd Sunday’s in a month. Ukers show up with a couple of Ukulele songbooks with literally hundreds of songs. (I happened to have one of them on my iPad when I first went, and had both by my second meet-up.)
For the first 90 minutes, it’s a “strum-along.” The leader calls out a page number, then after a count off, everyone plays the same song and sings along. This is a remarkably successful learning activity for me as a newcomer.
- It’s completely safe. If I can play along, I do. If I can’t, I just sing, or just watch. If I can play the chords but more slowly, I catch up on the second or third strum of a measure. I can immediately hear if I’m getting it right (right chord, right rhythm) or if I made a mistake. The people right next to me can hear me and can comment on my playing, but only those — it’s a big group.
- It’s a public opportunity for learning. I know what chords everyone is playing. I can look around and see how everyone else plays it.
- While everyone is strumming, the really good players are picking individual notes, or doing tricky rhythms. I can hear those, and watch them do it, and develop new goals for things I want to learn.
The gaps between the songs are when a lot of the learning happens for me. I get coaching (e.g., “You are doing really well!” or “I heard you stammer in your rhythm on that hard chord change”). I can ask specific questions and get specific advice. I’ve received tips on how to make D7 chords more easily, and different ways to do barre chords.
After 90 minutes, it’s open-mic time. Individual ukers sign up during the strum-along, and then go up to the corner stage to perform (a quality setup, with separate mics for singing and for playing and someone at a sound board). Here’s where we get to see those on their way or at the center of the community of practice. Those at the center of the community of practice reference other meet-ups and other performances, and often play their own compositions.
As a newcomer, I stare slack-jawed at the open-mic performances. They create music that I didn’t know could be made on a ukulele. Slowly, I’m starting to imagine myself playing at open-mic, even writing my own music. I’m starting to set a personal goal to become more central to this community of practice.
At a meet-up, I talk to my fellow ukers and get a sense of how much effort does it take to develop that level of expertise. I start to get a sense of how much effort it will take me to reach different levels of expertise. There’s no expectations set on me, and no presumption of expertise. I can decide for myself on how good I want to get and how much effort I can afford to put in. I can set my own pace for when I might one day sign up for an open-mic performance, and maybe even try to compose my own music. (But it won’t be soon.)
Creating a Computing Samba/Meet-Up Culture
Could we create an experience like the Samba school or like the meet-up for learning computing by adults, like undergraduates, end-user programmers, and high school teachers? What are the critical parts that we would need to duplicate?
It must be safe. People should be able to save face at the meet-up. Participants need to be able to talk with one another privately, without overhead (e.g., learning some complicated mechanism to open a private chat line). Newcomers need to be able to participate without expectation or responsibility, but be able to take on expectation and responsibility as they become more central to the community.
There must be legitimate peripheral participation. Newcomers have to be able to participate in a way that’s meaningful while working at the edge of the community of practice. Asking the noobs in an open-source project to write the docs or to do user testing is not a form of legitimate peripheral participation because most open source projects don’t care about either of those. The activity is not valued.
Everyone’s work must be visible. Newcomers should be able to see the great work of the more central participants just by looking around. This is probably the trickiest part. We tend to confuse accessibility with visibility. Yes, on an open source project, everyone’s contributions are accessible — if you can figure out github, and figure out which files are meaningful, and figure out who contributed which. Visible means that you can look around without overhead and see what’s going on.
I must be able to work alone. Everyone needs a lot of hours of practice to develop expertise. It can’t happen just in the meetup. There needs to be a way to develop one’s work alone, and share it in the meetup.
A Proposed Computing Meet-Up Context
Here are some early thoughts on what it might be like to create an environment for learning computing the way that the ukulele meetup works.
Years ago, the Kansas environment was implemented in the programming language Self. Kansas was remarkable. It was a shared desktop where all participants could see each other, see their cursors, and see their developing work.
Lex Spoon created a version of Kansas for the Squeak programming language called Nebraska (for another “large, flat, sparsely-populated space”). Nebraska in Squeak is particularly interesting for a meet-up because all the rich multi-media features of Squeak are available in both a programmable and a drag-and-drop form.
Here’s a sketch of what I propose, using a shared space like Kansas or Nebraska:
- Participants come to a physical space with their laptops. Physical co-location is key for safe and easy peer communication. Anew journal article on co-located viewing of MOOCs suggests that co-location may dramatically improve learning.
- The participants log on to a shared Kansas/Nebraska server, which is displayed an ultra-high resolution display.
- The participants work together to create a multimedia show.
- Newcomers can build the graphical or audio elements (perhaps some developed at home and brought to the meetup). Building can start in drag-and-drop form, but can develop into code elements. If something doesn’t work, it might not make it into the show, but it’s a contribution to the shared space, and it’s visible for comment and review.
- All participants can watch others work, and can walk over to them to ask questions.
- Participants can specialize, by focusing on different aspects of the performance (e.g., music, graphics, layout, synchronization).
- Those more central to the community can assemble components and choreograph the whole performance (much as in a Samba school).
Would this kind of meet-up be a way for adults to learn computation in a constructionist manner?
Interesting question: When do toys lead to changes in stereotypes, and when don’t they?
Why is including female scientists in toys for young kids so important? Because children begin making stereotypical assumptions of what a scientist looks like early in elementary school, and these stereotypes become more firmly implanted year-by-year. Starting in 1957, several research groups have asked school children to “draw a scientist” in order to probe these stereotypes. Normally, when someone is asked to draw a generic person they will draw a person of their own gender. This is not true when children are asked to draw scientists. In study after study, the vast majority of students of both genders draw a scientist as a white man in a lab coat.What is really fascinating is that several studies—although not all—have found that these stereotypes can be changed by simply exposing children to female and/or minority scientists. After interacting with diverse examples of scientists, children made more realistic images of scientists—portraying them without dangerous, smoking test tubes for example—and were more likely to draw female and minority scientists.
Fun and interesting blog addressing the belief that mathematics is necessary for programming, a misconception that Nathan Ensmenger claims has reduced the percentage of women in computing. Sarah Mei goes into some depth addressing (and dispensing with) each of the three claims below
Specifically, learning to program is more like learning a new language than it is like doing math problems. And the experience of programming today, in industry, is more about language than it is about math.And my next thought, of course, was why doesn’t anyone else think this? Why do we still have this idea that math skills indicate programming potential, while language skills mean you should go into poli sci?Well, when I feel out of my depth, I usually start by looking for “official” opinions. So I looked for relevant academic research.
I found absolutely none, which is pretty flabbergasting. I found a lot of opinions, both from computer science educators, and from people in industry. Perhaps within academia, the link between math and programming is considered such an obvious truth that it isn’t worth confirming with research.It seems more likely, though, that this research exists, but not under the search terms I tried. Please let me know if you are aware of relevant papers.In the meantime, if we can’t have data, we can at least examine the conversations people have on this topic. Here are some things people often say when asserting that people must be good at math to be good developers.Generally, they fall into three categories:
1. “You need to know math to be a good programmer.”
2. “You need to learn math to get the skills you need for programming.”
3. “Plenty of programming is still math!”
Coding Is the New Writing for Developing Self-Expression, Communication, Imagination and Solving Hard Problems
A really nice article by Dr. Idit Harel Caperton on why to learn to code. She really captures well the insight that computing is a medium, and the value of developing literacy in this medium.
To summarize, kids and adults too learn best by teaching, explaining, inventing and representing information to others. Coding is the new writing tool for young minds to do just that and especially well when integrated into a harder problem and a larger purpose, than learning coding for the sake of learning coding.
Do students fail intro CS at higher rates than in comparable classes (e.g., intro Physics, or Calculus, or History)? We’ve been trying to answer that question for years. I studied that question here at Georgia Tech (see my Media Computation retrospective paper at last year’s ICER). Jens Bennedsen and Michael Caspersen answered that question with a big international survey (see paper here). They recognized the limitations of their study — it was surveying on the SIGCSE member’s list and similar email lists (i.e., to teachers biased toward being informed about the latest in computing education), and they got few responses.
This last year’s ITiCSE best paper awardee tried to measure failure rates again (see link below), by studying published accounts of pass rates. While they got a larger sample size this way, it’s even more limited than the Bennedsen and Caspersen study:
- Nobody publishes a paper saying, “Hey, we’ve had lousy retention rates for 10 years running!” Analyzing publications means that you’re biasing your sample to teachers and researchers who are trying to improve those retention rates, and they’re probably publishing positive results. You’re not really getting the large numbers of classes whose results aren’t published and whose teachers aren’t on the SIGCSE members list.
- I recognized many of the papers in the meta-analysis. I was co-author on several of them. The same class retention data appeared in several of those papers. There was no funny business going on. We reported on retention data from our baseline classes. We then tried a variety of interventions, e.g., with Media Computation and with Robotics. The baseline then appears in both papers. The authors say that they made sure that that didn’t double count any classes that appeared in two papers, but I can’t see how they could possibly tell.
- Finally, the authors do not explicitly cite the papers used in their meta-analysis. Instead, they’re included on a separate page (see here). SIGCSE shouldn’t publish papers that do this. Meta-analyses should be given enough pages to list all their sources, or they shouldn’t be published. Including them on a separate page makes it much harder to check the work, to see what data got used in the analysis. Second, they are referencing work that won’t appear in any reverse citation indices or in the authors’ H-index calculations. I know some of the authors of those papers who are up for promotion or tenure decisions this coming year. Those authors are having impact through this secondary publication, but they are receiving no credit for it.
This paper is exploring an important question, and does make a contribution. But it’s a much more limited study than what has come before.
Whilst working on an upcoming meta-analysis that synthesized fifty years of research on predictors of programming performance, we made an interesting discovery. Despite several studies citing a motivation for research as the high failure rates of introductory programming courses, to date, the majority of available evidence on this phenomenon is at best anecdotal in nature, and only a single study by Bennedsen and Caspersen has attempted to determine a worldwide pass rate of introductory programming courses.In this paper, we answer the call for further substantial evidence on the CS1 failure rate phenomenon, by performing a systematic review of introductory programming literature, and a statistical analysis on pass rate data extracted from relevant articles. Pass rates describing the outcomes of 161 CS1 courses that ran in 15 different countries, across 51 institutions were extracted and analysed. An almost identical mean worldwide pass rate of 67.7% was found. Moderator analysis revealed significant, but perhaps not substantial differences in pass rates based upon: grade level, country, and class size. However, pass rates were found not to have significantly differed over time, or based upon the programming language taught in the course. This paper serves as a motivation for researchers of introductory programming education, and provides much needed quantitative evidence on the potential difficulties and failure rates of this course.