Media Computation today: Runestone, Snap!, Python 3, and a Teaspoon Language

I don’t get to teach Media Computation¹ since I moved to the University of Michigan, so I haven’t done as much development on the curriculum and infrastructure as I might like if I were teaching it today. I did get a new version of JES (Jython Environment for Students) released in March 2020 (blog post here), but have rarely even started JES since then.

But using Jython for Media Computation is so 2002. Where is Media Computation going today?

I’ve written a couple of blog posts about where Media Computation is showing up outside of JES and undergraduate CS. Jens Moenig has been doing amazing things with doing Media Computation in Snap! — see this blog post from last year on his Snap!Con keynote talk. SAP is now offering a course From Media Computation to Data Science using Snap! (see link here). Barbara Ericson’s work with Runestone ebooks (see an example blog post here) includes image manipulation in Python inside the browser at an AP CS Principles level (see example here). The amazing CS Awesome ebook that Beryl Hoffman and Jen Rosato have been doing with Barb for AP CS A includes in-browser coding of Java for the Picture Lab (see example here).

I was contacted this last January by Russ Tuck and Jonathan Senning. They’re at Gordon College where they teach Media Computation, but they wanted to do it in Python 3 instead of Jython. You can find it here. It works SO well! I miss having the image and sound explorers, but my basic demos with both images and sounds work exactly as-is, with no code changes. Bravo to the Gordon College team!

Most of my research these days is grounded in Task-Specific Programming languages, which I’ve blogged about here (here’s a thread of examples here and here’s an announcement of funding for the work in social studies). We now refer to the project as Teaspoon Computing or Teaspoon Languages — task-specific programming => TSP => Teaspoon. We’re adding a teaspoon of computing into other subjects. Tammy Shreiner and I have contributed a chapter on Teaspoon computing to a new book by Aman Yadav and Ulf Dalvad Berthelsen (see announcement of the book here).

We have a new Teaspoon language, Pixel Equations, that uses Media Computation to support an Engineering course in a Detroit Public School. Here, students choose a picture as input, then (1) enter the boolean equations for what pixels to select and (2) enter equations for new red, green, and blue values for those pixels. The conditionals and pixel loops are now implicit.

In several of our tools, we’re now exploring bilingual or multilingual interfaces, inspired by Sara Vogel’s work on translanguaging (see paper here) and Manuel Pérez-Quiñones’s recent work on providing interfaces for bilingual users (see his TED talk here and his ACM Interactions paper here). You can see in the screenshot below that colors can be referenced in either English or Spanish names. We’re now running participatory design sessions with teachers using Pixel Equations.

I’m planning a series of blog posts on all our Teaspoon languages work, but it’ll take a while until I get there.

---

1. For new readers, Media Computation is a way of introducing computing by focusing on data abstractions used in digital media. Students write programs to manipulate pixels of a picture (to create photo filters), samples of a sound (e.g., to reverse sounds), characters of a text, and frames of a video (for video special effects). More at http://mediacomputation.org ↩

JES 5.02 Now Released, and Media Computation 4th Edition Slides Available

JES 5.02 is now released at https://github.com/gatech-csl/jes/releases/tag/5.020.  I have links to all the main downloads at http://mediacomputation.org .

This is a maintenance release.  Thanks to Nina Koch’s student, Henry, we have fixes for colorizing and some other problems.  Henry has also written code to allow for capture of keystrokes and mouse movement in a picture window, so that you could build some simple games.  I’ll save that for JES 5.10.

I don’t do JES releases that often, so in-between, I forget how painful cross-platform development is. JES is all written in Java and Jython, which means we “write once, test everywhere.”  I developed mostly on Mac OS X, but makensis (for Windows Installers) doesn’t work on Mac OS X.  So I ran Ubuntu in VirtualBox to build the Windows installer and test the Linux version.  But I still had to test on Windows (e.g., to make sure that the Sound Explorer was fixed).  It’s all details —  ls here vs. dir/w there, can’t install Oracle Java from Ubuntu installer, remembering sudo apt-get, Lenovo toolkit updater interrupting installations…

From Github:

 Fixes in JES 5.02 (with many thanks to HenryStevens and sportsracer48 for fixes):

• Fixes a problem if you quit during raw_input that you can’t do another raw_input
• Makes the Sound explorer fully functional again
• Fixes colorizing
• Makes autosave work again
• Fixes a threading error if you hit return too rapidly in the Command Area
• Unicode characters in input file get flagged. (Jython can’t execute a line with unicode characters on it.)

In addition,

You can find all the fourth edition Powerpoint slides (including a 68Mb zip of all of them) at http://coweb.cc.gatech.edu/mediaComp-teach/60. I’ve put some of the Peer Instruction question slides into the chapter PPT slide decks, but you can find more at the instructors-only website (see the Media Computation website for more on the teacher website).

Most of the changes are in the early chapters. Chapter 3 on text and language manipulation is all new. The latter chapter PPT slide decks have a few new slides in each deck, including:

• Creating state-preserving versions of picture manipulation functions in Chapter 16 on Functional Programming
• Subclassing Picture and Sound to move functions into methods in Chapter 17 on Object-oriented Programming
• Recursive turtle patterns, which are possible with an improved Turtle class in JES 5 in Chapter 17