Learning to code is really learning to code something: One doesn’t just “learn programming” nor “learn tracing”

I asked a group of social studies educators what programming language(s) they might want to use in their classes. One of the interesting themes in the responses was “the same as what’s in math and science classes.” One teacher said that she didn’t want a “weird hierarchy” where there’s one programming language in STEM and another in “history and English” for fear they’d be seen as “dumbed down.” Another said that maybe teaching JavaScript in history class “would make history cool.”

There’s a belief in this theme that I think is wrong. Learning to program in science class probably won’t transfer without a bunch of work to programming in mathematics class, and programming STEM classes will probably be a very different thing than programming in the humanities classes. Even expert programmers learn to program in a domain, and have a hard time transferring that knowledge of programming between domains. Expertise is expertise in a domain.

My advisor, Elliot Soloway, was involved in some of the early studies that supported this claim. The first paper was “The role of domain experience in software design” by Beth Adelson and Elliot Soloway from 1985. I quote from the abstract:

A designer’s expertise rests on the knowledge and skills which develop with experience in a domain. As a result, when a designer is designing an object in an unfamiliar domain he will not have the same knowledge and skills available to him as when he is designing an object in a familiar domain.

In this study, they took expert software designers in various fields, and have them design systems in other fields. They also asked novice designers to do some of the same tasks. For example, maybe we have a software designer who has been building banking software, and another who has been designing real-time control systems. Now, let’s ask both designers to design an elevator control system.

What they found was that the designers in the new domain struggled. They stopped planning (e.g., making notes). When they were in the familiar domain, they would often visualize the working system (“simulation” in the paper). Novices didn’t. The experts didn’t when they were faced with a new domain. Experts in an unfamiliar domain looked much like novices. Now, experts in an unfamiliar domain were better than the novices at noticing constraints on the design, so something transferred.

The second paper is even more striking. “Empirical Studies of Programming Knowledge” (1984) by Elliot Soloway and Kate Ehrlich. From the abstract:

We suggest that expert programmers have and use two types of programming knowledge: (1) programming plans, which are generic program fragments that represent stereotypic action sequences in programming, and (2) rules of programming discourse, which capture the conventions in programming and govern the composition of the plans into programs.

When we teach programming, we tend to focus on the syntax and semantics of the language. We don’t explicitly teach plans — chunks of code that do something useful. But we expect students to figure them out. We rarely teach discourse rules. The domain-specific knowledge lies in both plans and discourse rules.

To test the claim about the importance of these discourse rules, they produce sets of two programs: Alpha and Beta. Alpha is a perfectly fine program. Beta breaks the rules. For example, if you see a variable initialized n := 0;, you would find it weird to later see read(n); (to input a new value for n). It’s not wrong. The code might work just fine — in fact, it does work just fine in the experimental construction of Beta. But the program breaks the rules of discourse. They write:

Notice that both Alpha version and the Beta version are runnable programs that in almost all cases compute the same values. Moreover, to an untrained eye their differences may even not be apparent; they always only differ by a very few textual elements.

Here’s an example of one Alpha and Beta — these both work. In this case, they do not do the same thing:

Beta isn’t wrong. It successfully computes minimum. However, it uses the variable max which is confusing. It breaks our discourse rule. The program does work.

Different domains use different standards and different styles of programming. Engineers using MATLAB rarely use FOR or WHILE loops, for example. Graphic designers writing JavaScript code use far more exception handling than we ever expected.

Soloway and Ehrlich showed these programs to “experts” (undergraduate juniors to graduate students) and novices (students in their first programming course). When asked questions about Alpha (e.g., “What goes in this missing line in the code?” Or “Do you remember that code that I showed you?”), experts do far better than novices. When asked questions about Beta, experts do essentially the same as novices (no statistically significant differences).

I find it particularly notable that the expert drop is steeper.  Experts rely heavily on cues like variable names, even more than novices. CS expertise is really expertise in the discourse rules.

If expert programmers “knew programming,” they should be able to just trace the code (“be a computer”) and answer the questions correctly. Instead, they struggle to understand what’s going on. They’re pretty much like a student in their first semester of programming. The experts know Alpha well because it’s just like all the other programs they’ve ever seen — they can pattern match, rather than reason about the code itself. The experts struggle with Beta. It’s kind of like the difference between humans and Econs. Econs can reason through code rationally. Humans rely on expectations.

These results also suggest that the question of “Does tracing come before writing?” is moot.  Tracing what?  The program matters.  Some programs are harder to trace than others — for everyone, and particularly, with expertise.  There is no generic “tracing skill.”

Conclusion: People don’t just learn “coding.” Programmers in general know plans and discourse rules. Break the rules and you just have the programming language — and even experts aren’t really good at just applying the syntax and semantics rules. No better than a novice. If you have enough expertise in different domains, then you can work in different domains. When you start programming in a new domain, you’re not that much different than a new programmer.

The social studies teachers I’m working with have a sense that students can just
know JavaScript.” I don’t think that’s true. I think if I taught students to write JavaScript code to use Google’s Charts service for making data visualizations, it wouldn’t be much easier to teach them Web programming with React, to write scripts for Adobe Photoshop, nor to build simulations in Lively Web. It’s all JavaScript, and the syntax and semantics are the same in each — but in terms of what people really know and use (i.e., plans and discourse rules), it’s completely different.

May 20, 2019 at 7:00 am 7 comments

Seeking Data: What’s happening at your school as you cap CS major enrollment?

I’m just back from the 2019 NCWIT Summit (see link here), which was amazing — as always. I talked to people at schools who have instituted caps on undergraduate CS enrollment, and I’m hearing stories that I didn’t expect.  I’d love to hear your experience at your school.  Are you seeing these things?

  • One story is that students are taking and re-taking (“2-3 times”) the early classes, to get high enough grades to get past the grade cap.  Thus, the GPA grade cap has actually increased enrollment pressure on earlier classes.
  • Because of these course repeats, students are (presumably) taking longer to graduate. I didn’t talk to anyone with data on that — maybe it’s too soon, since caps are within the last 3-5 years at most institutions?
  • I was also hearing about incredible pressure that students are feeling because of the grade caps.  We expected to see impacts on enrollment for under-represented groups, but these reports say that everyone has increased stress because of the grade caps. The caps are leading to damage to department climate and even a spike in mental health issues. (I heard some pretty horrible stories.)

These are all just anecdote. I’m not sure how to cast a wider net for more information, but this blog might be a place to start.  Could you share your reports on how enrollment caps are impacting your course enrollment at the lower levels, on time to graduation, and on departmental climate (or other issues)? Thanks!

 

May 17, 2019 at 7:00 am 9 comments

Open Question around Mathematics in Undergraduate Computer Science

I’m always happy to see a new computing education blog, and I’m particularly excited by posts that identify open (research, and otherwise) questions.

At SIGCSE 2019 this past February, we organized a birds of a feather session (a one-hour discussion group) on modernizing mathematics in computer science. We expected a modest number of attendees but were surprised and delighted to host a completely filled room of discrete mathematics, algorithms, and theory of computation educators—60 attendees in total—interested in evolving how we, as a discipline, situate mathematical foundations in our curriculum!

What was even more surprising to us was how the discussion evolved over the hour. Our original intention was to focus on how we might re-shape the foundational portions of the computer curriculum in light of how computing has evolved over the last decade:

The undergraduate computer science curriculum is ever-changing but has seen particular turmoil recently. Topics such as machine learning, data science, and concurrency and parallelism have grown in importance over the last few years. As the content of our curriculum changes, so too does the mathematical foundations on which it rests. Do our current theoretical courses adequately support these foundations or must we consider new pedagogy that is more relevant to our students’ needs? In this BoF, we will discuss what a modern mathematics curriculum for computer scientists should cover and how we should go about accomplishing this in our classrooms. (https://dl.acm.org/citation.cfm?id=3293748)

At this point, we shifted our focus from trying to answer the original “concept” question to identifying the myriad of problems that educators wrestled with along these three dimensions. We outline the problems that people raised below:

From https://cs-foundations-ed.github.io/sigcse/2019/03/29/bof-report.html

May 13, 2019 at 7:00 am Leave a comment

Comparing performance in learning computer science between countries

Imagine that you are a high school chemistry teacher, and you’re convinced that you have developed a terrific way to teach the basic introductory to chemistry course. Your students do terrific on all your assessments and go on to success in chemistry in college. You decide that you want to test yourself — are your students really as good as you think they are?

You reach out to some friends in other schools and ask them to give your final exam to their students. You are careful about picking the other schools so that they’re really comparable along dimensions like student wealth, size of school, and student demographics. Your friends are willing, but they just have a few of their students take the test. You don’t know really how they pick. Maybe it’s the best students. Maybe it’s the students who need remedial help. Maybe it’s a punishment for students in detention. Of course, all of your students take the final exam.

In the end, you have lots of YOUR students who took YOUR exam, and you have a handful of other students. Your friends (who likely don’t teach like you) give you a few tests from their students. Is it at all surprising that your students will likely out-score the friends’ students?

That’s how I read this paper from Proceedings of the National Academy of Sciences of the US: “Computer science skills across China, India, Russia, and the United States.” The authors are quite careful about picking schools to compare, along dimensions of how “elite” the schools are. I’m quite willing to believe that there is a range of schools with different results along an “elite” spectrum.

They over-sample from the United States, compared to the population of these countries:

Altogether, 678 seniors from China (119 from elite programs), 364 seniors from India (71 from elite programs), and 551 seniors from Russia (116 from elite programs) took the examination…We also obtained assessment data on 6,847 seniors from a representative sample of CS programs in the United States (607 from elite programs).

The test they use is the “Major Field Test” from ETS. I don’t know that it’s a bad test. I do suspect that it’s US-centric. It’s like the final exam from our Chemistry teacher in my example. Compare that to the TIMSS assessments that go to great lengths to make sure that the data are contextualized and that the assessments are fair for everyone.

Maybe the results are true. Maybe US computer science students are far better than comparable CS students in Russia, China, and India. I’m just not convinced by this study.

May 6, 2019 at 7:00 am 1 comment

What’s NOT Computational Thinking? Curly braces, x = x + 1, and else.

In the previous blog post, I suggested that Computational Thinking is the friction necessary to make your problem solvable by a computer. It should be minimized unless it’s generative.  It’s a very different framing for computational thinking.  Rather than “what’s everything that we use in computing that might be useful for kids,” it’s closer to, “the day is full and students are already in so many subjects — what do they have to know about computing in order to use it to further their learning and their lives?”

What is NOT Computational Thinking

I have been talking to my students about what’s on the list of things that we typically teach but don’t fit into this model of computational thinking. Here’s what I’ve thought of so-far:

Here are criteria for what should NOT be part of teaching computational thinking:

  • These are hard for students — why go to that extra effort unless it’s worthwhile?
  • We have invented ways of framing problems for a computer that do not use these things, so they’re not necessary.
  • They are not generative. Knowing any of these things does not give you new leverage on thinking about problems within a domain.

If Computational Thinking is something we should teach to everyone, these are items that are not worth teaching to everyone.

Computational thinking includes programming, for me. It is generative.  It allows students to explore causal models that are tested with automation.  It’s the most powerful idea in computational thinking.

What is Computational Thinking for OTHER subjects

Then there are the ideas that are on most lists of computational thinking, like decomposition and abstraction. I absolutely believe that all programmers have those skills. They are absolutely generative. I believe that programming is a terrific place to try out and play with those ideas.

In the Rich et al. paper about learning trajectories that I reference so often, they talk about students learning “Different sets of instructions can produce the same outcome.” That’s a critical idea if you want students to learn that different decompositions can still result in the same outcome.

But does abstraction and decomposition belong in a Computational Thinking class?  They feel more like mathematics, science, and engineering to me.  Yes, use computing there, but don’t break them out into a separate class.  A mathematics teacher may be better prepared to teach decomposition and abstraction than are computer science teachers. It’s better to teach these ideas in a context with a teacher who has the PCK to teach them.

What’s more, it’s clear that you don’t need abstraction and decomposition to program computers as a way to learn.  Task-specific programming languages are usable for learning something else without developing new abilities to abstract or decompose. Our social studies teachers did in our participatory design study in March — they learned things about life expectancy in different parts of the world, using programming that they did themselves, in 10-20 minutes.

What is Computer Science that EVERYONE should know

There’s another list we could make that is ideas in computer science that everyone should know because it helps them to understand the computation in their lives.  Yes, there’s a lot in the school day — but this is worth it for the same reason that Physics or Biology is worth it. This is a different matter than what helps them solve problems (which is the guts of the computational thinking definitions we have seen earlier).  On my list, I’d include:

  • Bits, the atom of information processing.
  • Processes, what programs allow us to define.
  • Programming, as a way to define processes.

Other suggestions?

  • What’s on your list for what’s NOT necessary in Computational Thinking, and
  • What is in Computer Science that everyone needs but is not Computational Thinking?

May 3, 2019 at 7:00 am 36 comments

A new definition of Computational Thinking: It’s the Friction that we want to Minimize unless it’s Generative,

David Benedetto wrote a blog post about computational thinking for CSTA that gave me new insight into Computational Thinking (thanks to Shuchi Grover whose tweets drew me to it):

http://advocate.csteachers.org/2019/02/27/situated-computational-thinking/

David says:

I think this definition of CT is as good a starting point as any:

Computational Thinking is the thought processes involved in formulating problems and their solutions so that the solutions are represented in a form that can be effectively carried out by an information-processing agent (Cuny, Snyder, Wing, 2010).

He evolves this position until, like Shuchi, he comes up with two definitions of CT:

What are the implications of this? I think there are two clear options for how we define CT:

(A) Restrict what we mean by CT. This is perfectly reasonable and probably necessary for most practical purposes. However, this has the inevitable consequence of fragmenting our understanding of CT. There will be different CTs in different disciplines / fields. We will do this, but we should try to understand the restrictions that we are imposing, and the consequences of imposing them.

(B) Break our concept of CT wide open. I think the scientific community (at least, those who are studying the construct of CT and how it plays out in real cultural contexts) should do this, so that we can explore how CT is understood and practiced in a variety of contexts and for a wide range of purposes.

As a researcher, I’m more in favor of the former — let’s define Computational Thinking precisely.  David’s concern is really about the social context around CT. People want to call lots of things Computational Thinking. Can we come up with a definition for CT that bridges these? That represents the discipline-specific uses of CT, and is well enough defined that we can actually measure something about it?

There are many other “thinkings” that lay claim to providing students with critical skills. Admiral Grace Hopper would likely support “mathematical thinking” more than “computational thinking,” as this interesting essay from Yale points out. Skills like “decomposition” or “abstraction” are included in many definitions of computational thinking (eg this blog post), and it’s true that you need those in computing.  But those skills first belonged to mathematics, engineering, and science, and I’d argue that the teachers in those subjects might be in a better position to teach them and to measure them. Computation can play an important role in learning decomposition and abstraction, but those skills don’t belong uniquely to computation, or to a class on computational thinking. So, what is unique about computation?

The tension between HCI and Computational Thinking

On the computer science side of my life, my research community is human-computer interaction.  I’ve published in CHI, DIS, CSCW, VL/HCC, and UIST. The Cuny, Snyder, and Wing definition is hard for me to reconcile with being an HCI researcher.  The point of HCI research is to minimize the amount that a user has to learn in order to “formulate problems and their solutions so that the solutions are represented in a form that can be effectively carried out by an information-processing agent.”  HCI is trying to make it easier for the user to think with a computer whatever they want to think about. Computational Thinking is about what you need to think with a computer.

Over the last few weeks in this blog, I’ve been exploring the notion of task-specific programming languages. I was amazed at how much social studies teachers could do with Vega-Lite in a participatory design session we ran in early March. Sarah Chasin’s work with Helena and Rousillon is absolutely stunning for how much people could achieve with no training. Hariharan Subramonyam sent me this fascinating essay on end-user programming and about how to minimize the effort it takes end users to start programming: https://www.inkandswitch.com/end-user-programming.html. As I talked about in my SIGCSE 2019 keynote, Bootstrap:Algebra and most uses of Scratch actually rely on a small number of computational ideas. There is expressive and learning power in even a small amount of computation.

Michael Mateas wrote an essay back in 2009 that has been influential in my thinking. I blogged about it here: “There will always be friction.” Michael looked at the Alan Perlis talk of 1961 (that I talk and write about often), and particularly, at the exchange with Peter Elias. Elias argued that students shouldn’t have to learn to program — the computer should learn to understand us. Both Perlis and Mateas disagree. The computer can never understand us completely. We have to smooth the communication because the computer cannot. There will always be a challenge to human-computer interaction. There will always be friction, and it’s the human’s job to manage that friction..

A New Definition for Computational Thinking

So, here’s my new take on Computational Thinking: It’s the friction. Let’s take the original Cuny, Snyder, and Wing definition — computational thinking is about framing problems so that computers can solve them. The work around task-specific programming languages is showing us that we can make that amount that the user has to learn in order to use programming for their problem very small.

To meet Alan Kay’s point about generativity, there are some things in computing that we want to teach because they give us new leverage on thinking. We want to teach things that are useful, but not those that are necessary just because we have bad user interfaces.

A minimal definition of Computational Thinking: The stuff that we have to learn in order to communicate our tasks with a computer. It should be small, and the part that we learn should be generative, useful for new problems and new thinking. Everything else should be eliminated by good user interfaces.

You don’t have to master abstraction and decomposition just to use a programming language to help you learn. Our social studies teachers modified Vega-Lite programs, made mistakes (every single one of them) and recovered from them, and tried new things that they figured out on their own — all in 10-20 minutes. They already have problem solving skills. They didn’t need any “computational problem solving skills.” They certainly didn’t learn any special computational abilities to abstract and decompose in 10 minutes. They already know enough to use programming to learn. If we can eliminate the need for a particular skill in order to use computing to learn something else, we should.

This meshes with David Weintrop and Uri Wilesnky’s definition — it’s the computational practices of actual scientists and engineers who use computing. Their definition is particularly strong because it’s empirically grounded. They asked computational scientists and engineers what they actually do. Weintrop and Wilesnky’s participants want to do their work, not programming for its own sake. So they use a minimal subset of computing that buys them something for their thinking and in their tasks.

I like this definition because it’s aspirational.  Today, there’s a lot of stuff that you have to learn to use a computer to solve problems. Philip Guo gave a talk here at Michigan recently (similar to one he gave at U-W) and described how data scientists have to become system administrators to manage all the various packages and databases to do their job.  That’s a problem. That’s not computational thinking. That’s stuff to get rid of. How small can we make computational thinking?

 

April 29, 2019 at 7:00 am 10 comments

Congratulations to Chris Stephenson, Outstanding Contribution to ACM Awardee 2018

A well-deserved honor!  Chris has done so much for computing education, from CSTA to her great work at Google. Congratulations!

stephenson_5836051

CITATION

For advancing K-12 CS education by founding and developing the Computer Science Teachers Association and incorporating its mission and membership into the ACM professional community.

Dr. Chris Stephenson has expanded the definition of the professional computing community to include K-12 computer science (CS) teachers and education partners, supporting ACM’s mission to increase the diversity and perspectives of the community. Chris’s work in architecting, founding and nurturing the Computer Science Teachers Association (CSTA) in partnership with ACM into a professional international organization with more than 22,000 members dedicated to supporting teachers and pursuing excellence in K-12 CS education is widely recognized. Her scholarly research contributions, including work on computational thinking in K-12 education, the inaugural CSTA K-12 Computer Science Standards and the Running on Empty and Bugs in the System reports, have led to projects, initiatives and policy changes that have deeply and positively impacted K-12 education and educators globally.

 

From this web page.

 

April 26, 2019 at 7:00 am Leave a comment

Older Posts


Enter your email address to follow this blog and receive notifications of new posts by email.

Join 6,221 other followers

Feeds

Recent Posts

Blog Stats

  • 1,646,880 hits
May 2019
M T W T F S S
« Apr    
 12345
6789101112
13141516171819
20212223242526
2728293031  

CS Teaching Tips