Doing with Images Makes Symbols, and from Action to Abstraction

Here’s my morning Twilight Zone moment for you.  I’m here at the SILC Center doing an NSF Site Review.  (I learned last night that I’m allowed to say that–it’s considered public knowledge.)  I recommend this piece from Center Director Nora Newcombe as a readable introduction to their work.  One of their theoretical framings here is called “From Action to Abstraction“:

Learning from action to abstraction: In contrast to traditional views of the mind as an abstract information processor, recent theories of embodied cognition suggest that our representations of objects and events are often grounded in the sensorimotor systems we use to perceive and act on the world (Wilson, 2002). This linking of thought and action is readily observed when STEM practitioners talk about objects in their area of expertise. For example, organic chemists gesture heavily when discussing molecular structure and engineers rely on sketches in conceptual design. This leads us to believe that involving the action systems in learning may help to deepen students’ knowledge of abstract concepts by tying them to sensorimotor brain systems that are good at capturing spatial/action relationships. We are interested in understanding how performing different actions (ranging from feeling forces when learning about angular momentum in physics, to actually manipulating models of physical molecules in chemistry to learn about their spatial makeup, to sketching spatial relations in geosciences) might bolster spatial learning by engaging sensorimotor systems that might not otherwise be brought to bear on the concepts at hand. Moreover, we are interested in when this action information might harm performance by tying students’ representations too closely to the physical world and how tools such as gesture and sketching might serve as a bridge between concrete physical relations and more abstract knowledge. Finally, we are interested in how different forms of action can provide a window into learners’ minds by revealing information that they may not be able to articulate verbally.

Then I read this email from Brian Harvey of Berkeley (author of the excellent Computer Science Logo Style books) on the SIGCSE Members list talking about Alan Kay’s video on similar themes from several years back:

By the way, in our first course for CS majors I show them the 30? 40? year
old Alan Kay “Doing with Images Makes Symbols” lecture (google it) [That’s a live link — I did the Googling for you], still
the most inspiring thing I’ve seen about user interface design, and one
thing I like about it is that, although it shows some details, it isn’t
/about/ details, but about how ideas about human psychology inform UI design.

Kinda weird to get these similar ideas, from two different directions, in one morning, eh?