Live coders challenge CS to think about expression again

Bret Victor’s great time traveling video emphasized that the 1960’s and 1970’s computer scientists were concerned with expression. How do you talk to a computer, and how should it help you express yourself? As I have complained previously, everything but C and C-like languages have disappeared from our undergraduate curriculum. Bret Victor has explored why we talked about expression in those earlier years. I have a different question: How do we get computer scientists to think about expression again?

Live coders think about and talk about expression, as evidenced from the conversations at Dagstuhl. They build their own languages and their own systems. They talk about the abstractions that they’re using (both musical and computational, like temporal recursion), how their languages support various sound generation techniques (e.g., unit generators, synthesized instruments, sampled sounds) and musical styles. If you look at the live coders on the Dagstuhl Seminar participant list, most of them are in music programs, not computer science. Why are the musicians more willing to explore expressive notations than the computer scientists?

Lisp is alive and well in live coding. I now have a half-dozen of these systems running on my laptop. Overtone is a wonderful system based in Clojure. (See here more on Overtone, and particularly powerful combined with quil for Processing visuals combined with music.) Andrew Sorensen’s Impromptu was in Scheme, as is his new environment Extempore.

Extempore is amazing. Take a look at this video of an installation called “Physics Playroom,” all controlled in Extempore. It’s a huge touch sensitive display that lets groups of students play with physics in real-time, e.g., exploring gravity systems on different planets. Andrew said that he could build 90% of this in Impromptu, but the low-level bits would have to be coded in C. He wasn’t happy with changing his expressive tools, so he created Extempore whose lowest level parts would be compiled (via LLVM) directly to machine code. Andrew went to this effort because he care a lot about the expressiveness of his tools. (At the opposite end from the Physics Playroom, see this video of Extempore running on ARM boards.)

Not everything is S-Expressions. Thor Magnusson’s Ixi Lang (more on the Ixi Lang project) is remarkable. I love how he explores the use of text programming as both a notation and a feedback mechanism. When he manipulates sequences of notes or percussion patterns, whatever line he defined the sequence on changes as well (seen in red and green below, as agents/lines that have been manipulated by other operations).

Tidal from Alex Maclean is a domain-specific language built on top of Haskell, and his new Texture system creates more of a diagramming notation. Dave Griffiths has built his live coding environment, Fluxus, in Racket which is used in Program by Design and Bootstrap CS education projects. Dave did all his live coding at Dagstuhl using his Scheme Bricks, which is a Scratch-like block language that represents Scheme forms. (See here for Dave’s blog post on the Dagstuhl seminar.)

How many of our undergraduates have ever seen or used notations like these? How many have considered the design challenges of creating a programming notation for a given domain? Consider especially the constraints of live coding (e.g., expressiveness, conciseness, and usability at 2 am in a dance club). David Ogbourn raised the fascinating question at Dagstuhl of designing programming languages for ad hoc groups, in a collaborative design process. Some evidence suggests that there may be nine times as many end-user programmers in various domains as professional software developers.  Do we teach CS students how to design programming notations to meet the needs and constraints of various domains and communities?

I wonder how many other domains are exploring their own notations, their own programming languages, without much contribution or involvement from computer scientists.  I hope that the live coders and others designing domain-specific languages challenge the academic computer scientists to think again about expression. I really can’t believe that the peak of human expression in a computing medium was reached in 1973 with C, and everything else (Java, C++, C#) is just variations on the motif.  We in computer science should be leading in exploring the design of expressive programming languages for different domains.