High school students learning programming do better with block-based languages, and the impact is greatest for female and minority students

August 20, 2018 at 7:00 am 9 comments

I learned about this study months ago, and I was so glad to see it published in ICLS 2018 this last summer.  The paper is “Blocks or Text? How Programming Language Modality Makes a Difference in Assessing Underrepresented Populations” by David Weintrop, Heather Killen, and Baker Franke.  Here’s the abstract:

Broadening participation in computing is a major goal in contemporary computer science education. The emergence of visual, block-based programming environments such as Scratch and Alice have created a new pathway into computing, bringing creativity and playfulness into introductory computing contexts. Building on these successes, national curricular efforts in the United States are starting to incorporate block-based programming into instructional materials alongside, or in place of, conventional text-based programming. To understand if this decision is helping learners from historically underrepresented populations succeed in computing classes, this paper presents an analysis of over 5,000 students answering questions presented in both block-based and text-based modalities. A comparative analysis shows that while all students perform better when questions are presented in the block-based form, female students and students from historically underrepresented minorities saw the largest improvements. This finding suggests the choice of representation can positively affect groups historically marginalized in computing.

Here’s the key idea as I see it. They studied over 5,000 high school students learning programming. They compared students use block-based and text-based programming questions.  Everyone does better with blocks, but the difference is largest for female students and those from under-represented groups.

Here’s the key graph from the paper:


So, why wouldn’t we start teaching programming with blocks?  There is an issue that students might think that it’s a “toy” and not authentic — Betsy DiSalvo saw that with her Glitch students. But a study with 5K students suggests that the advantages of blocks swamp the issues of inauthenticity.

The International Conference on the Learning Sciences (ICLS) 2018 Proceedings are available here.

Entry filed under: Uncategorized. Tags: , , , , .

CRA Memo on Best Practices for Engaging Teaching Faculty in Research Computing Departments In last five years, little progress in increasing the fraction of American CS BS degree recipients who are African Americans

9 Comments Add your own

  • 1. alanone1  |  August 20, 2018 at 7:19 am

    Well, not at all surprising. The largest point of the Etoys scripting interface (from which the blocks part of both Scratch and Alice came) was (a) to deal with cognitive load (a huge part of pretty much everything), and also (b) to have chunking be done in terms of larger “phrases of meaning” (following some of Joe Becker’s ideas about how humans really “do language”).

    I was hoping that both Scratch and Alice would incorporate much stronger “process ideas” than they do. We thought this was critical because despite the history around “algorithmic programming” and “procedures and data structures”, most important computing — for both beginners and adults — is about making and modifying systems of processes (and having processes be made of systems of processes).

    The cautionary part of this that success with a weak set of ideas can make it very difficult for the stronger ideas to even be seen, let alone adopted. I think this has been happening the last few decades, and the “coding for all” people had a chance to introduce stronger ideas. but failed to do so (and may not have understood the stronger ideas, or even that they exist).

  • 2. Austin Cory Bart  |  August 20, 2018 at 2:06 pm

    To the point of authenticity, I still hold out hope for the mutual language translation like in BlockPy (Python), Droplet (JS), and Stride (Java). Block languages that translate bidirectionally to professional text languages can let us move past the worries of whether they’re learning a real skill.

    • 3. Mark Guzdial  |  August 21, 2018 at 9:20 am

      We already know that there is transfer between block-based languages and text languages — see David Weintrop’s dissertation work for one. I know of no work that shows that block-based languages that are 1:1 with textual languages improve transfer.

      I’m choosing to ignore the crack about “real skill.”

      • 4. Austin Cory Bart  |  August 21, 2018 at 9:45 am

        My comment has nothing to do with the reality of what skills they learn and everything to do about student perception. Obviously scratch at al is real programming. But on countless occasions I’ve had students who don’t believe that. A not uncommon perception is that block based programming is a less authentic skill than text based programming. In the same vein, some students believe some text based languages are less useful to learn than others. My point is just that languages with strong connections to languages that students perceive as authentic allow us to sidestep the entire conversation. Yes, we can try and overcome that student misconception with more teaching, but then you’re adding another learning objective to the pile and that has its own drawbacks.

      • 5. Mike Zamansky  |  August 21, 2018 at 6:47 pm

        Does a 20 week look at 3 classes really mean “We already know that there is transfer between block-based languages and text languages?”

        • 6. Mark Guzdial  |  August 22, 2018 at 8:09 pm

          Sure — that’s a reasonable size for an experiment.

          In education (and most social science research), there are small-scale experiments where one can carefully control the treatment. Then there are large-scale “in the field” experiments (like the 5K CSP students in my recent blog post. You can’t know for sure what every student was taught (e.g., were they all taught blocks and text equally?), but it’s a convincing size. It’s hard to balance both.

          And by “know,” I mean what I usually do (href=”https://computinged.wordpress.com/2018/06/15/are-you-talking-to-me-interaction-between-teachers-and-researchers-around-evidence-truth-and-decision-making/”>as I explained here). This is the best evidence we have. We should get more, and we may change our minds, but from what I see in the literature, yes, we believe that there is transfer.

  • […] High school students learning programming do better with block-based languages, and the impact is gr… […]

  • 8. cycomachead  |  August 22, 2018 at 5:25 am

    I would love to see data on whether perceptions of “toys” affect both learning outcomes and continued study of CS.
    The concern over that view is valid. As someone who spends / spent my Berkeley career dealing with Snap!, I saw plenty of students at all levels of the spectrum of ‘toy’ness in how they viewed Snap!.

    For students who had the least knowledge, they seemed to care the least about whether Snap! was a “real” language. It was only when they learned of the other programming languages that their perceptions of Snap! changed. We dealt with our fair share to stereotypical hot-shot programmers, but I also suspect that us as TAs our own language had tons of impact too. Mostly subtle things, and not intentional, phrases like “real languages” or “later you’ll use Python” create the perception of lesser-ness among teaching languages.

    I hope that as time goes on, more people will see that there are tons of new tools that amount different forms of visual programming, even if they aren’t Turing Complete languages. Look at stuff from If This Then That, to Apple’s Siri Shortcuts / Workflows tools, or the many business apps that have GUIs for building data pipelines or even nearly-complete mobile apps.

  • 9. Omowale Casselle  |  September 5, 2018 at 4:51 pm

    We’ve found that while the block-based languages are better for teaching the fundamentals of programming, both the parents and students have a negative perception of block-based either because of a very introductory study during ‘Hour of Code’ in school or simply because the interface is visually appealing (not like a typical text editor).

    However, after we explain why it’s better to understand algorithm design and logic without struggling through syntax errors and additional debugging/troubleshooting efforts for a specific language (Python, Java); we are able to get students to achieve positive outcomes.

    Scratch while it might appear simple, that is only a result of the excellent design from the MIT team. One can actually build quite complex projects if they are willing to suspend judgment about what constitutes the proper modality for learning to program.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 5,317 other followers


Recent Posts

Blog Stats

  • 1,554,047 hits
August 2018
« Jun   Sep »

CS Teaching Tips

%d bloggers like this: