What do I mean by Computing Education Research? The Computer Science Perspective

 

Last week, I talked about how I explain what I do to social scientists. This time, let me explain what I do to computer scientists. I haven’t given this talk yet, and have only tried the ideas out on a few people. So consider this an experiment, and I’d appreciate your feedback.

Let’s simplify the problem of computing education research (maybe a case of a spherical cow). Let’s imagine that instead of classes of Real Humans, we are teaching programming to Human-like Turing Machines (HTMs). I’m not arguing that Turing machines are sufficient to represent human beings. I’m asking you to believe that (a) we might be able to create Turing Machines that could simulate humans, like those we have in our classes, (b) RH’s would only have additional capabilities beyond what HTM’s have, and (c) HTM’s and RH’s would similar mechanisms for cognition and learning. (Carl Hewitt has a great CACM blog post arguing that message passing is more powerful than TM’s or first order logic, so maybe these should be HMP, Human Message Passers. I don’t think I need more than TM’s for this post.)

This isn’t a radical simplification. Cognitive science started out using computation as a model for understanding cognition (see history here). Information processing theory in psychology starts from a belief that humans process information like a computer (see Wikipedia article and Ed Psychology reference). Newell and Simon won the ACM Turing award and in their Turing Award lecture introduced the physical symbol system hypothesis, “A physical symbol system has the necessary and sufficient means for general intelligent action.” If we have a program on a Turing machine that gives it the ability to process the world in symbols, our theory suggests that it would be capable of intelligence, even human-like intelligence. I’m applying this lens to how we think about humans learning to program.

This simplification buys me two claims:

  • The Geek Gene is off the table. The Geek Gene is the belief that some people can’t learn to program (see blog post for more). Any Turing machine can simulate any other Turing machine. Our HTM’s are capable of tracing a program. If any HTM can also write code, then all HTM’s can write code. Everyone has the same computational capability. (If HTM’s can all code, then RH’s can all code, because HTM’s have a subset of RH cognitive capabilities.)
  • Learning of our students can be analyzed and understood as information processing. The behavior of Turing machines is understandable with analysis. HTM’s are sophisticated Turing machines. The core mechanism of HTM’s can be analyzed and understood. If we think about our students as HTM’s, we might reason about their learning about computing.

Here are some of the research questions that I find interesting, within this framing.

How do HTM’s learn to program?

All HTM’s must learn, and learn at a level where their initial programming (the bootstrap code written on their tape when they come into our world) becomes indistinguishable from learned capabilities. HTM’s must have built-in programming to eat and to sleep. They learn to walk and run and decipher symbols like “A,” such that it’s hard to tell what was pre-programmed and what was learned. HTM’s can extend their programming.

There are lots of models that describe how HTM’s could learn, such as SOAR and ACT-R. But none so far has learned to program. The closest are the models used to build the cognitive tutors for programming, but those couldn’t debug and couldn’t design programs. They could work from a definition of a program to assemble a program, but that’s not what most of us would call coding. How would they do it?

How would HTM’s think about code? How would it be represented in memory (whether that memory is a tape, RAM, or human brains)? There is growing research interest in how people construct mental models of notional machines. Even experts don’t really know the formal semantics of a language. So instead, they have a common, “notional” way of thinking about the language. How does that notional machine get represented, and how does it get developed?

How do we teach HTM’s to learn to program?

You shouldn’t be able to just reprogram HTM’s or extend their programs by some manipulation of the HTM’s. That would be dangerous. The HTM might be damaged, or learn something that led them into danger. Instead, extending HTM’s programming can only be done by conscious effort by the HTM. That’s a core principle of Piaget’s Theory of Cognitive Development — children (RH’s and HTM’s) learn by consciously constructing a model of the world.

So, we can’t just tell an HTM how to program. Instead, we have to give them experiences and situations where they learn to program when trying to make sense of their world. We could just make them program a lot, on increasingly harder programs. Not only is that de-motivating (maybe not an issue for HTM’s, but certainly is for RH’s), but it’s inefficient. Turns out that we can use worked examples with subgoal labeling and techniques like Parson’s problems and peer instruction to dramatically improve learning in less time.

What native capabilities of HTM’s are used when they learn to code?

We know that learning to read involves re-using more primitive mechanisms to see patterns (see article here). When HTM’s learn to program, what parts of the native programming are being re-used for programming?

Programming in RH’s may involve re-use of our built-in ability to reason about space and language. My colleague Wes Weimer (website) is doing FMRI studies showing that programmers tend to use the parts of their brain associated with language and spatial reasoning. In our work, we have been studying the role of spatial reasoning and gesture in learning to program (see summaries of our ICER 2018 papers). We don’t know why spatial reasoning might be playing a role in learning to program. Maybe it’s not spatial reasoning, but some aspect of spatial reasoning or maybe it’s even some other native ability that is related to spatial reasoning.

How does code work as an external representation of HTM’s, and where does it help?

We can safely assume that HTM’s, like RH’s, would enhance their cognition through the use of external representations. Cognition and memory are limited. Even an infinite tape has limitations in terms of time to access. Human cognitive systems are limited in terms of how much can be attended to at once. RH’s use external representations (writing notes, making diagrams, sketches) to enhance their cognition. We’re assuming that HTM’s have a subset of RH abilities, so external representations would help HTM’s, too.

My students and I talk about a wonderful paper by David Kirsh, Thinking with External Representations (see link here). It’s a compelling view of how external representations give us abilities to think that we don’t have with just our brain alone.

How can program code be a useful external representation for HTM’s? When does it help, e.g., with what cognitive tasks is code a useful external representation? For example, a natural one is modeling and simulation — we can model more complex situations with program code than we can keep in our head, and we can simulate that model for a much larger range of time and possible values. Are there cognitive tasks where code by itself, as a notation like written language or mathematics, can enhance cognition? Here I’m thinking about the ability of code to represent causal relationships (e.g., as in Bruce Sherin’s work) or algebraic forms (e.g., as in Bootstrap) — see here for discussion of both.  I’m intrigued by the idea of the affordances of reading code even before writing it.

What makes programming worth learning for HTM’s?

Why should an HTM learn programming? Let’s assume that an HTM’s basic programming is going to be about staying alive, e.g., Maslow’s hierarchy of needs. When would an HTM want to learn programming?

The most obvious reason to learn programming is because you can get paid to do it. It’s about meeting physiological needs and safety. But, if you can meet those needs doing something that’s easier or more pleasant or has fewer barriers, you’ll likely do that.

Sometimes, you’ll want to learn programming because it makes easier something you want to do anyway. Brian Dorn’s graphic designers wanted to learn programming (see here) because they used Photoshop or GIMP and wanted a way to do that easier and faster. Maybe that’s about safety and physiological needs, but maybe it was about esteem or even self-actualization (if HTM’s care about those things).

Where my simplification breaks down: Real humans learn in situated and social contexts

Our learning theory about RH’s say that they are unlikely to start a new subject unless there’s social pressure to do so (see Pat Alexander’s Model of Domain Learning). Would HTM’s feel social pressure? Maybe.

As I described in the previous blog post, much of my work is framed around sociocultural models of learning, like Lave and Wenger’s situated learning. I use Communities of Practice to understand a lot of the situations that I explore. We can only go so far in thinking about programming as just being inside of individual minds (HTM or RH). Much of the interesting stuff comes when we realize that (a) our cognition interacts with the environments and situations around us, and (b) our motivation, affect, and cognition are influenced by our social world.

Setting aside whether it’s social science or computer science, I am still driven by a paper I read in 1982, which was five years after it was written: “Personal Dynamic Media” by Alan Kay and Adele Goldberg (see copy here).  I want people to be to use coding like they use other literacies, to create a literature, and in a casual, informal and still insightful way.  Mitchel Resnick often talks about people using Scratch to write a card to their mother or grandmother — that’s the kind of thing I want to see.  I want people to be able to make small computational models that answer questions, in the same way that people do “back of the envelope” calculations today. I also want great literature — we need Shakespeares and daVinci’s who convey great thoughts with computing (an argument that Andrea diSessa made recently at the PPIG conference which Felienne Hermans blogged about here.) That’s the vision that drives me, whether I’m using cognitive science or situated learning.

 

November 12, 2018 at 8:00 am 7 comments

When do we know that a programming course is not working for non-CS majors?

There’s a good discussion going on in Facebook that I wanted to make more public and raise as a question here.  The crush of undergraduates in CS today is making it difficult to offer tailored introductory CS courses to different majors.  The problem is particularly acute when designing instruction for future CS teachers.  If you put the CS teachers in the same course as the CS majors, it’s cheaper and easier — you just teach one big course, rather than multiple smaller courses. But is it as effective?

Some of my colleagues suggest that we can design undergraduate introductory computing courses that are effective for both non-CS and CS majors.  Here’s my question: How do you know when it’s not working?  At what point would you admit that the one course option isn’t working? It’s an interesting empirical question.

Here are some possible measures:

  • Learning is a tricky measure.  For any discipline, the majors in that discipline are more motivated to learn more than the outside-the-discipline majors. You can’t expect the non-CS majors to learn more than the CS majors.  Then again, the non-CS majors probably come in knowing less.  If you do pre- and post-tests on CS knowledge, do non-CS majors have as large of a gain as the CS majors?  I don’t know, but in any case, it’s not a great measure for deciding if a class is succeeding for the non-CS majors.
  • Taking more CS courses may be an effective measure, but only if you have more than one course that’s useful to non-CS majors.  If the rest of the classes are about software development, then non-CS majors will probably not want to go on, even if the intro course was effective and well-designed.
  • Retention is a reasonable measure.  If more of the non-CS majors are dropping out from the course than the CS majors, you may not be meeting their needs.
  • My favorite measure is relevance I argued in my blog post on Monday that programming is a practice that is relevant to many communities. Do the non-CS majors see the relevance of computing for them and their community after the introductory course?  If not, I don’t think it’s meeting their needs.
  • Another tricky measure is use. Should non-CS majors be able (after their first course) to build some program that they find useful?  Certainly, if you achieve that goal, you have also achieved relevance.  How do you judge useful?  CS faculty may not be good judges of what a non-CS major would find useful, and CS faculty are most likely going to assess in terms of code quality (e.g., modularization, appropriate variable and function/module names, commenting, code style, etc.), which I consider pretty unimportant for as a measure for the non-CS students’ experience in the first course.

What do you think?  How would you know if your intro course was meeting non-CS students’ needs?

November 9, 2018 at 7:00 am 12 comments

What do I mean by Computing Education Research? The Social Science Perspective

As a new guy at the University of Michigan, I spend a lot of my time explaining who I am and what I mean by computing education research. In this and the next blog posts, I am sharing two of those explanations.

The first one was a six minute lightning talk to the University of Michigan School of Information. My audience was well-versed in social sciences, so I explained who I was in those terms.

I ground a lot of my work in Lave and Wenger’s (1991) theory about Communities of Practice. Let me create a symbol here to represent a community of practice. My symbol has a light boundary where novices engage in legitimate peripheral participation, there is a darker area where full-fledged members of the community work, and the core where the experts live who represent the values and practices of the community.

When people think about computing education research, they mostly think about this community of practice: software development. How do we help students join the community of practice of software developers? That’s an interesting question, and I have done some work there (e.g., on how students come to understand multiple class object-oriented systems, like Model-View-Controller), but that’s not my focus.

Let’s set that community of practice aside (lower left), and consider another that I’m much more interested in: end-users who program. I’ve taught thousands of liberal arts, architecture/design, and business/management majors to program using Media Computation, a contextualized approach to computing that focuses on their interests and needs. I worked with Brian Dorn as he studied graphics designers who learn to program. End-user programmers are far more numerous than professional software developers. They use programming for different purposes, so we would expect them to use different practices, languages, libraries, and tools. That’s why we developed JES, a Python programming environment for our future end-users who are programming with media. I’m very interested in understanding and supporting the practices of end-user programmers.

Let’s keep that community of practice in consideration, and next consider a different one: High school teachers who teach computer science. They’re not about software development, either. They’re going to do something different (different practices) and have different values. High school teachers need to learn efficiently because they don’t have a lot of time to learn, and they want to learn effective methods for their students. Here’s where we work on ebooks, and subgoal labeling, and Barb’s Parsons problems. I’m interested in how we make computing education efficient and effective, and in understanding the underlying cognitive mechanisms at work. Why do some things work better for learning programming than others?

Here’s another community of practice I care about: scientists and engineers who use programming as a new way to do science. Again, different practices and values than software developers. How do we best support them?  What tools do they need for their practices?

I would like students to have the same advantages as scientists and engineers, to be able to use code as a powerful and executable notation. Lately, I’ve been particularly focused on pre-calculus and economics. I know it’s stretching Lave and Wenger’s notion to think about classrooms as kinds of community of practice, so maybe the real community of practice is economists and professionals who use Computing. My specific interest is the edge of the community of practice, constructing legitimate peripheral participation for students who might use computing to aid in their entrance into the field. When does programming help with learning something else, and why does it help, and how can we make it more effective (e.g., use the best parts of programming that have the greatest leverage on supporting learning)?

So let’s consider these communities of practice.

This is a really weird picture, from Lave and Wenger’s perspective. I’m saying that programming is a practice in all of these communities, but it’s different in each one. We actually do know practices like this: Reading and writing, and the use of mathematics.

I suggest that programming is a literacy. (I’m not the first, of course, and I don’t make the argument nearly as well as my colleague, Yasmin Kafai.) It’s a way of expressing thought, communicating with others, and testing and exploring new ideas.

And that’s what makes computing education a social justice issue. If we have really invented a new literacy, we need to make it available to everyone.

November 5, 2018 at 9:00 am 9 comments

Fixing Mathematical Notation with Computing, and “Proving” It with Education

I was looking for a paper that I needed to review last night, and came upon these paragraphs in the paper I brought up by mistake.

Computers_and_Mathematical_Notation_-_Iverson_on_J

This is bold language:

It might be argued that mathematical notation (MN) is adequate as it is, and could not benefit from the infusion of ideas from programming languages. However, MN suffers an important defect: it is not executable on a computer, and cannot be used for rapid and accurate exploration of mathematical notions.

The paper I found in my archive “Computers and Mathematical Notation” doesn’t seem to be published anywhere.The author is Kenneth E. Iverson, the inventor of APL. This paper echoes some of the thoughts in Iverson’s 1980 Turing Award Lecture, “Notation as a Tool of Thought.”

The unpbulished paper is notable because he wrote it in J, his successor language to APL.  He realized that his languages would be more accessible if they used the ASCII character set. J (which you can find at http://jsoftware.com/) is essentially APL, but mapped to a normal keyboard.

The attempt to “fix” mathematical notation (“suggestions for improvement,” to be exact) is bold and interesting.  What makes his argument particularly relevant for this blog is how he made the argument. How do you “prove” that you have improved on traditional mathematics notation?

Iverson decided that education was the way to do it.  He wrote mathematics textbooks, using J.  He wanted to show that basic mathematics is more explorable using his notation.

I find this network of papers and textbooks fascinating.  I love the goal of inventing a programming notation, not to develop software, but to improve the expression and exploration of mathematics. (In that sense, J is like Mathematica.) I am intrigued by the challenge of how to show that you succeeded, and to use education as a way to demonstrate that success. I’m amazed at these multiple textbooks that Iverson wrote and released for free, to encourage exploration of mathematical ideas with J.


This week, I was informed that I will be receiving the 2019 SIGCSE Award for Outstanding Contribution to CS Education. The award will be presented at the 2019 SIGCSE Technical Symposium to be held in Minneapolis, MN  from Feb 27 – March 2, 2019. I am honored and thrilled.  SIGCSE has been my academic home since my first ACM publication at SIGCSE’94. The list of awardees is stunning, including my advisor, Elliot Soloway, Alan Kay, Hal Abelson, Jan Cuny, Alan Perlis, Judith Gal-Ezer, Sally Fincher, Grace Murray Hopper, Wirth, Knuth, and Dijkstra (among many others — the award started in 1981). It’s an impressive club I’m joining.

That announcement didn’t feel like enough for a blog post in itself, so I’m just tacking it on down here.  I’ll probably write more about it when I figure out what I’m going to say in my talk.

 

November 2, 2018 at 7:00 am 4 comments

Novum Organum: The original “How To Not Be Wrong”

When I visited with Alan Kay and Bonnie MacBird in June, one of the ideas that he got me thinking about was Sir Francis Bacon’s Novum Organum (1620, wikipedia link), for ‘new instrument of science.’ Bacon understood human tendencies for bias long before behavioral economics. His book was the prototype for the modern popular book “How to Not Be Wrong: The Power of Mathematical Thinking” which advocates for mathematics as an approach to addressing human biases and limitations.

Bacon aimed to construct a foundation for real science, a body of knowledge that we could trust despite the fact that our minds are weak and that we are easily swayed. He lists four “idols” — the biases which keep us from thinking objectively and scientifically. Wikipedia has a short description for each. A couple that I found particularly striking:

  • Idols of the tribe: The things we get wrong because we like to see things at human scale and in regular structures. I read these as including the ideas we like because everyone else likes them, like picking a programming language because it’s popular and not because it suits the task.
  • Idols of the cave: The things we get wrong because of our unique education and background. Bias due to privilege (and assuming that everyone else has the same privilege) seem to fall in here.
  • Idols of the market: I just kept thinking “computational thinking” here. Idols of the market include words “which spring from fallacious theories” and “that are the result of imprecise abstraction.”  Unsupported theories of transfer and terms which we can’t actually define and test are part of Bacon’s warnings about “the market.”

I haven’t read the whole document — it’s available on Project Gutenberg, but it’s tough going.  I have found that Bacon talks about issues not in the Wikipedia article that are are significant today. For example, Bacon decries making decisions based on too “few experiments” which is explicitly a concern addressed in the efforts to replicate prior results (e.g., article here).

I keep thinking about what Bacon would say about computing education research. CER has some deep research questions it’s pondering (which I plan to address in some future blog posts). How do we make sure that we’re doing Science and not just following our Baconian idols?

October 29, 2018 at 7:00 am 1 comment

What would convince faculty in other disciplines that programming is useful?

Recently I came across an article from the journal Issues in Information Systems, “Faculty perspectives on the information technology and analytics requirements of business students.” The authors surveyed 204 business faculty from 20 different universities.  They found that “[N]early a third of respondents (32.6%) felt that computer programming skills should not be required at all. Interestingly, the same number (32.6%) also believe that Calculus should not be required of business students.”  Below is the table with the results.  About a third of faculty actually thought that all business students should take a three credit hour course in programming, but a third also felt that it shouldn’t be required at all. Details in the table below:

business-faculty

I’ve been working with the Georgia Department of Education on a new kind of pre-calculus course that uses computing to demonstrate the pre-calc concepts in a variety of contexts, e.g., scalar multiplication of a vector by reducing red in all the pixels in a picture, matrix multiplication by doing transforms of objects in 3-D space, periodicity of functions (like trigonometric functions) to generate sounds, etc. We did a careful mapping of each pre-calculus learning objective to relevant computing demonstrations, with multiple possible computing contexts for each pre-calculus learning objective. The course was rejected by the mathematics oversight board today. They didn’t buy it at all.  Among the responses: “The description of the course states that it is ‘designed to prepare students for calculus and other college level mathematics courses,’ which they believe it does not” and “Members feel that computer science is not mathematics and should not be replacing a mathematics course.”

I’m struck by these two stories.  For me, programming is this useful new notation that can enhance learning in many disciplines.  I’m swayed by the results with Bootstrap and with the CT-STEM effort at Northwestern. I hadn’t realized the extent to which the teachers in the non-CS disciplines were not buying the story.

  • Business faculty are clearly dubious about the benefits of programming for business students.  I wonder if they’ve done the studies about how many business school graduates use programming (from SQL queries and spreadsheet macros, to data analysis and even modeling and simulation) in their daily jobs.
  • Mathematics faculty are clearly dubious that (a) programming to apply mathematics topics leads to more mathematics learning and (b) computer science is even related to mathematics.

These create an interesting set of research questions to me. Why are faculty in non-CS disciplines dubious about the advantages of programming for their students?  What do they think programming is?  Maybe they’re right — maybe “programming” as we are currently defining it isn’t worth the credit hours for their students. How could we re-define programming (and programming languages and tools) to make it more useful?

October 26, 2018 at 7:00 am 33 comments

How Google is supporting computing education research

Google in August made five awards to support computing education research (see announcement here).  They are not huge awards, and they’re all fairly short time frames. But what I’m impressed with is how Google is doing their investment in computing education research.

Google started their process by asking Paulo Blikstein to poll the field and write a report summarizing the state of computing education research.  I blogged about that report here — I liked it.  I thought he covered a lot of ground in a small space, and he pointed out important open research questions.

Google wanted to hear from researchers directly. So they held a workshop with a bunch of researchers (some involved in Paulo’s report, some outsiders) to talk to them about what were the pressing research issues we saw and what we’d recommend Google should do about them.  (I was there, and mentioned the workshop in passing in this blog post.) So, first they educated themselves (and the community) with Paulo’s report, then they brought in more voices to respond to the report and point out other issues.

Now they’ve made their awards. The process is interesting because they engaged the community, at multiple levels. They didn’t just hire the people away from their campuses to come to Google. They made external awards, so that the faculty keep teaching (which we desperately need with the exploding enrollments). I hope other companies make note of the process and consider it as a model.

 

October 22, 2018 at 7:00 am Leave a comment

Older Posts


Enter your email address to follow this blog and receive notifications of new posts by email.

Join 4,325 other followers

Feeds

Recent Posts

Blog Stats

  • 1,575,301 hits
November 2018
M T W T F S S
« Oct    
 1234
567891011
12131415161718
19202122232425
2627282930  

CS Teaching Tips