ICER 2015 at the University of Nebraska, Omaha was fantastic. Brian Dorn did a terrific job hosting all of us.
The Doctoral Consortium went really well. We had 20 students from US, Chile, Germany, and UK. Below is a picture from the “Up against the wall bubble sort” where experienced students went to one side, and newer students went to the other, and the former gave advice to the latter.
Georgia Tech had even more going on at ICER and RESPECT than I mentioned in my earlier blog posts (like here and here). The GVU Center did a nice write up about all of us here. The biggest thrill at ICER for the GT crowd was Briana Morrison receiving the Chairs Award (one of two best paper awards at ICER) for the paper that I blogged about here. Below is the whole GT contingent at ICER (including chair Brian Dorn, GT alum).
The other best paper award, the peoples’ choice John Henry Award, went to Kristin Searle and Yasmin Kafai (see paper here) about the e-textiles work with American Indians that I blogged about here. Kristin had so many interesting insights, like the boys in her project telling her that “I don’t own” the projects they made because they felt no ownership over the programming environment they were using.
The quality of the papers was very good (you can see the list of all of them here). My favorite paper from my review packet was presented Monday morning, Spatial Skills in Introductory Computer Programming. Steve Cooper and Sheryl Sorby with two undergraduates at Stanford did the study that I’ve been wanting to see for ages (see blog post where I talk about it). Training an experimental group in spatial skills improved performance over a control group. Surprisingly, SES and race differences disappeared in the experimental group! This is an important result.
But one session blew me away — it changed how I think about blocks programming.
- The first paper was from Thomas Price and Tiffany Barnes showing that students using blocks were able to achieve programming tasks faster than those using text, but with no difference in learning or attitudes afterwards (paper here). This was an interesting result, but it was a limited study (short intervention, no pre-test) so it mostly supported a finding from Chris Hundhausen from years previous that graphical, direct-manipulation languages lead to faster start-up than text languages (see paper here).
- David Weintrop presented his remarkable paper with Uri Wilensky (see paper here). Below is the graph that changed my thinking about blocks. David carefully developed an isomorphic test in blocks and text, and gave it to the same population. Students did much better on the blocks-based test. MODALITY MATTERS! Blocks and text are not equivalent. He did careful analyses at each level of the test. For example, David replicated the result that else clauses in text are really hard for novices (which I talked about here), but students perform much better in blocks-based if-else.
- Diana Franklin presented their paper describing fourth graders reading Scratch programs (see paper here). I was expecting a paper on program comprehension — it wasn’t. Instead, it was a paper about user interfaces, and how the user interface interfered or supported students exploring and coming to understand the program.
I came away from that three papers realizing that blocks programming is likely the best modality to use in elementary school programming, and perhaps even when starting to program in high school, and maybe even for end-user programmers. But even more important, I realized that Andy Ko’s comments about programming languages as being a powerful and unusable user interface (see his blog post here) is the critical insight about programming today. David showed us that blocks can dramatically increase readability of programs. Diana showed us that the user interface dramatically influences the readability of the blocks. At the novice programming level, blocks-based languages are the most promising direction today, and designing good blocks languages is as much a user interface design problem as it is a programming language design problem.
Nice piece about the Makerspace that Ben Shapiro helped create to draw Haitian girls into STEM. Ben’s quote is great — it’s about changing attitudes and identities, not just teaching tech.
The tools, computers and gadgets in Nedlam’s Workshop might imply that the Makerspace’s main purpose is to teach students technical skills, but that’s just one benefit it provides.
“At the heart of it, it’s not so much about any specific technology as how we can develop kids’ identity as people who can do this kind of stuff,” Shapiro said. “And how we can change the perception of teachers and administrators in the process.”
Demographically, young white males populate most Makerspaces in the United States, but Nedlam’s Workshop is used mostly by Haitian girls–recent U.S. immigrants, many of whom do not speak English fluently, and struggle with traditional classes. Nedlam’s Workshop offers an alternative mode of learning and expression.
Our second RESPECT paper is by Barbara Ericson and Tom McKlin. Barb has been developing this cool intervention based on the Texas-based Advanced Placement Incentive Program (APIP) from AP Strategies (see paper about that work) and Betsy DiSalvo’s Glitch (see blog post here). Barb is offering financial incentives to African American students to encourage them to take advantage of additional learning opportunities (e.g., attend webinars and face-to-face workshops), and then pass the AP CS exam. NMSI has also offered grants to states to replicate the Texas APIP project (e.g., blog post at NMSI).
Barb has published on Project Rise Up 4 CS at SIGCSE (see my blog post on that paper). This new paper, “Helping African American Students Pass Advanced Placement Computer Science: A Tale of Two States,” describes Barb’s efforts to replicate the project in another state, and Tom’s efforts to measure what happened.
The bottomline is that in both states where she tried this, the participants had significant improvements in attitudes towards computing from pre to post. Probably the most important attitude change is that the participants had a significant increase in their perceived ability to pass the exam, and some of the students said that they couldn’t have passed the AP CS exam without Project Rise Up 4 CS. Both states had their highest-ever African American AP CS pass rates, though it would be hard to ever make a causal argument that this is due to Project Rise Up 4 CS.
The significant contribution of the paper is the deep understanding of what the project meant to the students, based on interviews. Students talked about their classes and teachers, what worked in Project Rise Up 4 CS, and how the project helped their confidence and knowledge. Barb used undergraduate students to host the webinars and workshops, who served as “near-peer” mentors and role models for the students. Those near-peer mentors were a critical piece in making Project Rise Up work.
Barb’s paper is being highlighted as one of four “Exemplary” papers at RESPECT.
On Friday, August 14, the first RESPECT conference will be held in Charlotte, NC — the first international meeting of the IEEE Special Technical Community on Broadening Participation with technical co-sponsorship by the IEEE Computer Society (see conference website here). RESPECT stands for Research on Equity and Sustained Participation in Engineering, Computing, and Technology.
We have two papers in RESPECT which I’ll summarize in a couple of blog posts. I’m less familiar with IEEE rules on paper referencing and publishing, so I’ll make a copy available as soon as I get the rules sorted out.
Miranda Parker has just finished her first year as a Human-Centered Computing PhD student at Georgia Tech, working with me. She’s done terrific work in her first year which I hope to be talking more about as she publishes. At RESPECT 2015, she’ll be presenting her first paper as a PhD student, “A critical research synthesis of privilege in computing education.”
Miranda defines privilege as:
Privilege is an unearned, unasked-for advantage gained because of the way society views an aspect of a student’s identity, such as race, ethnicity, gender, socioeconomic status, and language.
Her short paper is a review of the literature on how we measure privilege, where its impact has been measured in other STEM fields, and where there are holes in the computing education literature. She’s using studies of privilege in other STEM fields to help define new research directions in computing education. It’s just the sort of contribution you’d want a first year PhD student to make. She’s surveying literature that we don’t reference much, and using that survey to identify new directions — for her, as well as the field.
Briana Morrison is presenting the next stage of our work on subgoal labeled worked examples, with Lauren Margulieux. Their paper is “Subgoals, Context, and Worked Examples in Learning Computing Problem Solving.” As you may recall, Lauren did a terrific set of studies (presented at ICER 2012) showing how adding subgoal labels to videos of App Inventor worked examples had a huge effect on learning, retention, and transfer (see my blog post on this work here).
Briana and Lauren are now teaming up to explore new directions in educational psychology space and new directions in computing education research.
- In the educational psychology space, they’re asking, “What if you make the students generate the subgoal labels?” Past research has found that generating the subgoal labels, rather than just having them given to the students, is harder on the students but leads to more learning.
- They’re also exploring what if the example and the practice come from the same or different contexts (where the “context” here is the cover story or word problem story). For example, we might show people how to average test grades, but then ask them to average golf scores — that’s a shift in context.
- In the computing education research space, Briana created subgoal labeled examples for a C-like pseudocode.
One of the important findings is that they replicated the earlier study, but now in a text-based language rather than a blocks-based language. On average, subgoal labels on worked examples improve performance over getting the same worked examples without subgoal labels. That’s the easy message.
The rest of the results are much more puzzling. Being in the same context (e.g., seeing averaging test scores in the worked examples, then being asked to average test scores in the practice) did statiscally worse than having to shift contexts (e.g., from test scores to golf scores). Why might that be?
Generating labels did seem to help performance. The Generate group had the highest attrition. That make sense, because increased complexity and cognitive load would predict that more participants would give up. But that drop-our rate makes it hard make strong claims. Now we’re comparing everyone in the other groups to only “those who gut it out” in the Generate group. The results are more suspect.
There is more nuance and deeper explanations in Briana’s paper than I’m providing here. I find this paper exciting. We have an example here of well-established educational psychology principles not quite working as you might expect in computer science. I don’t think it puts the principles in question. It suggests to me that there may be some unique learning challenges in computer science, e.g., if the complexity of computer science is greater than in other studies, then it’s easier for us to reach cognitive overload. Briana’s line of research may help us to understand how learning computing is different from learning statistics or physics.
ICER 2015 (see website here) is August 9-13 in Omaha, Nebraska. The event starts for me and Barbara Ericson, Miranda Parker, and Briana Morrison on Saturday August 8. They’re all in the Doctoral Consortium, and I’m one of the co-chairs this year. (No, I’m not a discussant for any of my students.) The DC kickoff dinner is on Saturday, and the DC is on Sunday. My thanks to my co-chair Anthony Robins and to our discussants Tiffany Barnes, Steve Cooper, Beth Simon, Ben Shapiro, and Aman Yadav. A huge thanks to the SIGCSE Board who fund the DC each year.
We’ve got two papers in ICER this year, and I’ll preview each of them in separate blog posts. The papers are already available in the ACM digital library (see listing here), and I’ll put them on my Guzdial Papers page as soon as the Authorizer updates with them.
I’m very excited that the first CSLearning4U project paper is being presented by Barbara on Tuesday. (See our website here, the initial blog post when I announced the project here, and the announcement that the ebook is now available). Her paper, “Analysis of Interactive Features Designed to Enhance Learning in an Ebook,” presents the educational psychology principles on memory and learning that we’re building on, describes features of the ebooks that we’re building, and presents the first empirical description of how the Runestone ebooks that we’re studying (some that we built, some that others have built) are being used.
My favorite figure in the paper is this one:
This lists all the interactive practice elements of one chapter of a Runestone ebook along the horizontal axis (in the order in which they appear in the book left-to-right), and the number of users who used that element vertically. The drop-off from left-to-right is the classic non-completion rate that we see in MOOCs and other online education. Notice the light blue bars labelled “AC-E”? That’s editing code (in executable Active Code elements). Notice all the taller bars around those light blue bars? That’s everything else. What we see here is that fewer and fewer learners edit code, while we still see learners doing other kinds of learning practice, like Parsons Problems and multiple choice problems. Variety works to keep more users engaged for longer.
A big chunk of the paper is a detailed analysis of learners using Parsons Problems. Barbara did observational studies and log file analyses to gauge how difficult the Parsons problems were. The teachers solved them in one or two tries, but they had more programming experience. The undergraduate and high schools students had more difficulty — some took over 100 tries to solve a problem. Her analysis supports her argument that we need adaptive Parsons Problems, which is a challenge that she’s planning on tackling next.
Important new paper in Nature that makes the argument for active learning in all science classes, which is one of the arguments I was making in my Top Ten Myths blog post. The image and section I’m quoting below are about a different issue than learning — turns out that active learning methods are important for retention, too.
Active learning is winning support from university administrators, who are facing demands for accountability: students and parents want to know why they should pay soaring tuition rates when so many lectures are now freely available online. It has also earned the attention of foundations, funding agencies and scientific societies, which see it as a way to patch the leaky pipeline for science students. In the United States, which keeps the most detailed statistics on this phenomenon, about 60% of students who enrol in a STEM field switch to a non-STEM field or drop out2 (see ‘A persistence problem’). That figure is roughly 80% for those from minority groups and for women.