High school students learning programming do better with block-based languages, and the impact is greatest for female and minority students

I learned about this study months ago, and I was so glad to see it published in ICLS 2018 this last summer.  The paper is “Blocks or Text? How Programming Language Modality Makes a Difference in Assessing Underrepresented Populations” by David Weintrop, Heather Killen, and Baker Franke.  Here’s the abstract:

Broadening participation in computing is a major goal in contemporary computer science education. The emergence of visual, block-based programming environments such as Scratch and Alice have created a new pathway into computing, bringing creativity and playfulness into introductory computing contexts. Building on these successes, national curricular efforts in the United States are starting to incorporate block-based programming into instructional materials alongside, or in place of, conventional text-based programming. To understand if this decision is helping learners from historically underrepresented populations succeed in computing classes, this paper presents an analysis of over 5,000 students answering questions presented in both block-based and text-based modalities. A comparative analysis shows that while all students perform better when questions are presented in the block-based form, female students and students from historically underrepresented minorities saw the largest improvements. This finding suggests the choice of representation can positively affect groups historically marginalized in computing.

Here’s the key idea as I see it. They studied over 5,000 high school students learning programming. They compared students use block-based and text-based programming questions.  Everyone does better with blocks, but the difference is largest for female students and those from under-represented groups.

Here’s the key graph from the paper:


So, why wouldn’t we start teaching programming with blocks?  There is an issue that students might think that it’s a “toy” and not authentic — Betsy DiSalvo saw that with her Glitch students. But a study with 5K students suggests that the advantages of blocks swamp the issues of inauthenticity.

The International Conference on the Learning Sciences (ICLS) 2018 Proceedings are available here.

August 20, 2018 at 7:00 am 5 comments

CRA Memo on Best Practices for Engaging Teaching Faculty in Research Computing Departments

I’m excited to see this memo from the Computing Research Association on the status of teaching faculty in computing departments. Computing departments are increasingly relying on teaching faculty, and it’s important to give them fair and equitable treatment.

I wrote in 2016 that “CS Teaching Faculty are like Tenant Farmers.” This memo addresses some of the issues I raised, though some are buried in the text of the memo.  I argued that teaching faculty should be involved in hiring for both traditional and teaching faculty, and that teaching faculty should serve in upper-level leadership positions.  The report does state halfway down the report, “Similarly, teaching faculty should be broadly included in faculty governance on matters related to their roles in the department, including participation in faculty meetings, voting rights on matters impacting the education mission, inclusion in evaluation of the teaching performance of other faculty, and input on hiring decisions.”  This memo is a step in the right direction.

To achieve their educational mission, computing departments at research universities increasingly depend on full-time teaching faculty who choose teaching as a long-term career. This memo discusses the need for teaching faculty, explores the impact of teaching faculty, and recommends best practices.

Essential best practices for departments include:

  • Departments should provide teaching faculty with equitable rights and resources, except in limited areas where differing job responsibilities make that inappropriate.

  • Departments should encourage teaching faculty to be equal and active partners on projects and committees with the goal of contributing to the department’s educational mission.

  • Departments should set course, preparation, student, and service loads of teaching faculty at a level that allows for innovation and quality instruction.


Source: Laying a Foundation: Best Practices for Engaging Teaching Faculty in Research Computing Departments

August 17, 2018 at 7:00 am 4 comments

How computing education researchers and learning scientists might better collaborate

Lauren Margulieux has started a blog which is pretty terrific.  I wrote about Lauren’s doctoral studies here, and I last blogged about her work (a paper comparing learning in programming, statistics, and chemistry) here.

In her blog, Lauren is explaining in lay terms papers from learning sciences, educational psychology, and educational technology.  She’s an interdisciplinary researcher, and she’s blogging to help others connect across disciplines.

Her most recent blog post is about an issue I’ve been thinking about a lot lately. I wrote a blog post in the summer about the challenge of bridging the modes of science and truth-seeking in (computing) education vs. computer science. Lauren summarizes a paper by Peffer and Renken about concrete strategies to be used between discipline-based education researchers (like math education researchers, science education researchers, or computing education researchers) and learning scientists. Quoting part of it below:

Challenges in Interdisciplinary Research: Collaboration within a field can be difficult as people attempt to reconcile different ideas towards one goal. Collaboration between fields, each with its own traditions in theory and methodology, can seem like a minefield. Below are some common challenges that DBERers and learning scientists face.

  1. Differences in hard and soft sciences – researchers in the hard sciences can often feel frustrated by the lack of predictability in human-subjects research, and researchers in social sciences can become frustrated when those in the hard sciences have unrealistic expectations or view research in the soft sciences as non-scientific.

  2. Differences in theories and frameworks – What constitutes a theory or framework can be different in different domains, confusing what is often a fundamental building block of research.

  3. Differences in research methodologies – those unfamiliar with human-subjects research can find its methodologies complex, varied, and full of uncertainty, and those who have endured countless hours of training in these methodologies can find it difficult to describe or justify methodological decisions in a concise way.

See more at https://laurenmarg.com/2018/07/29/peffer-renken-2016-dber-and-learning-sciences-collaboration-strategies/

August 12, 2018 at 11:00 pm 1 comment

Adaptive Parsons problems, and the role of SES and Gesture in learning computing: ICER 2018 Preview


Next week is the 2018 International Computing Education Research Conference in Espoo, Finland. The proceedings are (as of this writing) available here: https://dl.acm.org/citation.cfm?id=3230977. Our group has three papers in the 28 accepted this year.

“Evaluating the efficiency and effectiveness of adaptive Parsons problems” by Barbara Ericson, Jim Foley, and Jochen (“Jeff”) Rick

These are the final studies from Barb Ericson’s dissertation (I blogged about her defense here). In her experiment, she compared four conditions: Students learning through writing code, through fixing code, through solving Parsons problems, and through solving her new adaptive Parsons problems. She had a control group this time (different from her Koli Calling paper) that did turtle graphics between the pre-test and post-test, so that she could be sure that there wasn’t just a testing effect of pre-test followed by a post-test. The bottom line was basically what she predicted: Learning did occur, with no significant difference between treatment groups, but the Parsons problems groups took less time. Our ebooks now include some of her adaptive Parsons problems, so she can compare performance across many students on adaptive and non-adaptive forms of the same problem. She finds that students solve the problems more and with fewer trials on the adaptive problems. So, adaptive Parsons problems lead to the same amount of learning, in less time, with fewer failures. (Failures matter, since self-efficacy is a big deal in computer science education.)

“Socioeconomic status and Computer science achievement: Spatial ability as a mediating variable in a novel model of understanding” by Miranda Parker, Amber Solomon, Brianna Pritchett, David Illingworth, Lauren Margulieux, and Mark Guzdial

(Link to last version I reviewed.)

This study is a response to the paper Steve Cooper presented at ICER 2015 (see blog post here), where they found that spatial reasoning training erased performance differences between higher and lower socioeconomic status (SES) students, while the comparison class had higher-SES students performing better than lower-SES students. Miranda and Amber wanted to test this relationship at a larger scale.

Why should wealthier students do better in CS? The most common reason I’ve heard is that wealthier students have more opportunities to study CS — they have greater access. Sometimes that’s called preparatory privilege.

Miranda and Amber and their team wanted to test whether access is really the right intermediate variable. They gave students at two different Universities four tests:

  • Part of Miranda’s SCS1 to measure performance in CS.
  • A standardized test of SES.
  • A test of spatial reasoning.
  • A survey about the amount of access they had to CS education, e.g., formal classes, code clubs, summer camps, etc.

David and Lauren did the factor analysis and structural equation modeling to compare two hypotheses: Does higher SES lead to greater access which leads to greater success in CS, or does higher SES lead to higher spatial reasoning which leads to greater success in CS? Neither hypothesis accounted for a significant amount of the differences in CS performance, but the spatial reasoning model did better than the access model.

There are some significant limitations of this study. The biggest is that they gathered data at universities. A lot of SES variance just disappears when you look at college students — they tend to be wealthier than average.

Still, the result is important for challenging the prevailing assumption about why wealthier kids do better in CS. More, spatial reasoning is an interesting variable because it’s rather inexpensively taught. It’s expensive to prepare CS teachers and get them into all schools. Steve showed that we can teach spatial reasoning within an existing CS class and reduce SES differences.

“Applying a Gesture Taxonomy to Introductory Computing Concepts” by Amber Solomon, Betsy DiSalvo, Mark Guzdial, and Ben Shapiro

(Link to last version I saw.)

We were a bit surprised (quite pleasantly!) that this paper got into ICER. I love the paper, but it’s different from most ICER papers.

Amber is interested in the role that gestures play in teaching CS. She started this paper from a taxonomy of gestures seen in other STEM classes. She observed a CS classroom and used her observations to provide concrete examples of the gestures seen in other kinds of classes. This isn’t a report of empirical findings. This is a report of using a lens borrowed from another field to look at CS learning and teaching in a new way.

My favorite part of of this paper is when Amber points out what parts of CS gestures don’t really fit in the taxonomy. It’s one thing to point to lines of code – that’s relatively concrete. It’s another thing to “point” to reference data, e.g., when explaining a sort and you gesture at the two elements you’re comparing or swapping. What exactly/concretely are we pointing at? Arrays are neither horizontal nor vertical — that distinction doesn’t really exist in memory. Arrays have no physical representation, but we act (usually) as if they’re laid out horizontally in front of us. What assumptions are we making in order to use gestures in our teaching? And what if students don’t share in those assumptions?

August 10, 2018 at 7:00 am Leave a comment

CS educators listen to authority more than evidence: Time to move on

My CACM Blog post for July starts from Stuart Reges’ inflammatory blog post in June “Why Women Don’t Code.”  I use his post and other writing as a foil to critique how we make arguments in computing education.  They tend to be arguments from authority, not from evidence.

Why is that? Why do CS educators use evidence and research less than (as quoted in the CACM post) Physics educators?  Is it because of the youth of the field, so when we grow up we’ll think more about research on how to teach well?  Is it because of the economics of the field?  Getting a CS background is so lucrative that students are desperate to succeed in the classes. We don’t have to teach well — student motivation will make up for where our teaching lacks. Or is it something else — is it something about CS in its nature that leads to opposition to using evidence and research when making educational decisions?

In June, Stuart Reges, principal lecturer in Computer Science and Engineering at the University of Washington, published a blog post Why Women Don’t Code that led to several articles and blog posts in response (e.g., Seattle Times and GeekWire). Reges argues that women are simply never going to enter computing at significant numbers, and 20% is about all that we’re ever going to get.

Our community must face the difficult truth that we aren’t likely to make further progress in attracting women to computer science. Women can code, but often they don’t want to. We will never reach gender parity. You can shame and fire all of the Damores you find, but that won’t change the underlying reality.

It’s time for everyone to be honest, and my honest view is that having 20 percent women in tech is probably the best we are likely to achieve. Accepting that idea doesn’t mean that women should feel unwelcome. Recognizing that women will be in the minority makes me even more appreciative of the women who choose to join us.

Hank Levy, Director of the U-W CSE School, wrote a great statement in response (see here). Levy disagrees with Reges’s conclusions, but supports Reges’s right to make his argument. Levy puts the current gender ratio in computer science in context by comparing to other disciplines.

I was most struck by the 20% claim. That’s easily proven wrong. There are many CS educational programs in the US with more than 20% female (like Computational Media at Georgia Tech). There are countries where CS is more than 50% female. How can Reges claim that 20% is the best that we can possibly do?

Here’s something important about Stuart Reges that people outside of CS education might not know — he’s a rockstar. He packs the house when he speaks at education conferences. He publishes regularly in the field. He has written a popular book on how to teach Java in introductory computer science (see Building Java Programs). Students love him, and teachers want to be like him. When Stuart Reges speaks, CS educators listen.

In this post, I want to step back and consider how Reges is making his argument, because it says something about how we make decisions in computing education. I am going to characterize the argument style in computing education as argument from authority which Wikipedia describes as “a claimed authority’s support is used as evidence for an argument’s conclusion.” We need to recognize the form before we can move beyond it.

Click here to read the rest of the CACM Blog Post.

August 6, 2018 at 7:00 am 8 comments

A Rawlsian Argument to provide computing education beyond MOOCs

I’m still not moved, but the moving process is no longer consuming all my waking hours (and some of the hours when I wished I was asleep). We spent hours clearing out stuff from our house that we’d accrued over 25 years while raising three kids. We have a contract on a house in Ann Arbor, and a contract on our house in Decatur. Children have places to live, and most of their stuff is moved out of the house. We have dates for moving our stuff.

I can make some time to blog again.

Amy Bruckman and I wrote a piece for CACM that appears this month. Amy is an expert in ethical implications of computing, and I worry about MOOCs. Together, we wrote an article about the implications of John Rawls’ definitions of justice for computing education.

We used to think MOOCs were going to change higher education and would democratize education. In 2012, a reasonable person might have seen development of MOOCs as a way to bridge social and economic inequities. By creating MOOCs, CS departments could reasonably claim they were using their privilege to provide great benefit to the least-advantaged members of society.

Today, we have evidence MOOCs do not work like that.

People who take MOOCs already have access to education and tend to be wealthy.

We now know that MOOCs as we have used them so far violate Rawls’ Difference Principle—we are further advantaging the already advantaged. We have an ethical mandate to do better.

Note that there’s a typo in the CACM article: MOOC participants are .45 standard deviations wealthier than the average, not 45 standard deviations.

You can find the whole article here: Providing Equitable Access to Computing Education

It’s a particularly important article for me since it’s my first publication with a University of Michigan byline.

August 3, 2018 at 7:00 am 4 comments

We might want naive and delusional PhD students

We’re in the midst of cleaning out 25 years of accumulated stuff in our house in order to sell this house, buy a new house in Ann Arbor, and move to the University of Michigan by September 1.

As I was cleaning, I found the below — my original statement of purpose that I submitted to the University of Michigan in 1988 to start my doctorate.

I shared it with some friends, ruefully.  It felt silly, as well as grammatically flawed. I really did think that I was going to get a faculty position in “Computer Science and Education” when I graduated in the early 1990’s.  I was naive, maybe even delusional. I had no idea what academic CS was like when I applied. The reality is far different than what I imagined.  At the Home4CS event just this last April, I mentioned that it would be great if we had CS Education faculty slots in Schools of Education today.  As Diane Levitt reported on Twitter, the audience roared with laughter.  How crazy was I to think that we’d have some in the 1990’s?

But now, some positions like that do exist.  There are faculty who have been hired at US higher-education institutions to focus on CS Ed.  My new job at the University of Michigan is a joint position between CS and their Engineering Education Research program.  It took 25 years, but yeah, I’m going to have the kind of job for which I earned my PhD.

Some friends encouraged me to share this statement. Maybe it’s a good thing to have naive new PhD students.  Maybe that’s what we want in PhD students. We want PhD students to think long term, i.e., to have bought into a goal, a set of research questions, or a vision — and be willing to work at it for decades.  Eventually, if the student is really lucky and others are working on similar visions at the same time, the vision doesn’t seem not quite so naive, not quite so delusional.

I’ll be taking some time off from the blog while making the move to Michigan. I may post some guest contributions over the next few weeks, but for now, I’m putting the blog on hiatus.

June 29, 2018 at 7:00 am 5 comments

Older Posts

Recent Posts

August 2018
« Jun    


Blog Stats

  • 1,538,641 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 5,301 other followers

CS Teaching Tips