Proving and Improving Teaching Programming Languages

SIGPLAN Education Board has produced a report “Why undergraduates should learn the principles of programming languages”  which was presented at the ACM Education Council meeting.  It makes four claims for why students should study programming languages:

• Students learn widely-applicable design and implementation techniques.
• Many students will need to create new domain specific languages or virtual machines, so it’s useful for them to study what’s known about languages.
• By learning programming languages, students learn new computational models and speed learning of new languages.  “The best preparation for quickly learning and effectively using new languages is understanding the fundamentals underlying all programming languages and to have some prior experience with a variety of computational models.”
• Students learn how to choose the right programming language for a task.

The problem is that we have empirical support for none of these claims.  People are amazingly bad at transferring knowledge.  People tend to learn about a specific situation and not recognize when the same idea applies in a new situation — or worse, they transfer negatively, mistaking the similarity and using older knowledge in an incorrect way.

One of the few treatments of transfer of programming knowledge is The Transfer of Cognitive Skill by Mark Singley and John Anderson.  Transfer between programming languages, even between skills in the same language, is surprisingly small.  For example, there is evidence that students don’t even transfer (“vertically” as they describe it) between knowledge of how to write programs and how to debug those programs.

This doesn’t mean that the SIGPLAN folks are wrong or that those claims are wrong.  It’s simply that they haven’t been shown yet.

• We need studies showing students learning design and implementation techniques from programming languages, then applying them in new contexts.
• We need to show that students can usefully draw on older languages when designing new languages.
• We need to show that knowing one set of languages improves learning of a later set.  (Ben Shneiderman argued in the late 70’s that learning a second language can be even harder than learning a first language.)
• We need to show that we can teach students rubrics or guides by which they can choose new languages effectively.

My guess is: We can do all these things.  The real trick is how we teach such that these things happen.  There are these great examples in How People Learn showing that highlighting foundational knowledge, so that students recognize it and can use it in new contexts, can improve performance.  It is possible to teach for transfer.  No, transfer doesn’t occur automatically.  That doesn’t mean it can’t happen.

The SIGPLAN Education Board is planning to produce curricula to support the goals they’ve outlined.  I hope that they also create learning guides, recommendations on how to teach programming languages, and studies showing that these guides and recommendations work.  I believe that we can prove that learning programming languages can be very useful, but it may involve improving on current practice, which may not be informed by what learning scientists know about teaching for transfer.

Technology plus policy for scale

I’m at the University of California at Berkeley for an ACM Education Council meeting this week.  Yesterday, we heard a slew of reports: On what the SIGs (from SIGCHI to SIGGRAPH to SIGPLAN) are doing in education, on the latest in the common core initiative, to what’s going on at CSTA.  Mehran Sahami gave an overview of Stanford’s new CS Curriculum, and Andy van Dam presented his report from CRA-E (which he’ll do again at Snowbird.)  (Both Mehran and Andy’s talks emphasized the role of context in motivating computing and in supporting learning about connections between computing and contexts that we want students to learn.)

The highlight of the day for me was a panel that Dan Garcia organized on the challenges and future of computing education, considered across the education pipeline.  The speakers were:

• Michelle Friend Hutton, middle school CS teacher and president of CSTA.
• Josh Paley, a high school CS teacher in Palo Alto (high end school).
• Eugene Lemon, a high school CS teacher from Oakland, CA (where four of their students were killed this year, including one of his AP CS students who was about to become the first student from their school to ever go on to a four year college).
• Tom Murphy, a community college professor (who teaches C++ and Scheme, and whose goal is for his students to not have to re-take anything when they get to Berkeley).
• David Patterson, a famous Berkeley professor and past president of ACM.

Dave went last, and expressed pessimism that the problems of K-12 CS education could ever be solved.  That was quite a gauntlet to throw down, so the Q&A session afterward was long (was scheduled for 30 minutes, and went on for over an hour) and active.  Roscoe Giles of Boston University encouraged us to think not only about solutions, but about solutions that scale.  Teaching CS in K-12 is a huge problem.  Eric Roberts of Stanford (with Dave Patterson agreeing) suggested that technology is really our only possible solution to the problem — we have to be able to use the technology we teach about, to teach about technology better.

I wanted to throw in a follow-on comment.  I strongly agree with Eric and Dave that technology is key, but I think that education policy is a critical component.  The CS10K project is about having 10,000 high school CS teachers ready to teach AP in 10K schools by 2015.  We have 2,000 high school CS AP teachers today.  We can’t possibly increase five-fold the number of teachers without distance education — we can’t ramp up face-to-face programs fast enough.

But what happens in 2020?  Lijun Ni’s research (based on studies of other STEM fields) suggests that we’ll have maybe 5K teachers left of that original 10K.  STEM teachers tend to drop out at a higher rate than other K-12 teachers, around 50% within five years.  What influences teachers staying?  Having a sense of belonging which is influenced by certification (e.g., teachers who are certified in science call themselves “science teachers” and tend to seek out professional development and community) and support systems.  Unless there is certification, and high school CS curricula (e.g., more than AP classes defined and being taught), and a community of CS teachers, we can expect to lose more than half those teachers in the first five years.

So technology is necessary to get the scale Roscoe is calling for, but so is policy to keep those teachers at scale.

Why Can’t Johnny Develop Secure Software?

The line of reasoning here is interesting.  The people interviewed in this piece argue that software developers will never learn to develop secure software — it’s at odds with their goals as developers (to write code fast, to meet customer needs).  But they also argue that it doesn’t work to bring in an outside security expert, because she won’t be able to pay attention to everything in the code to find every possible security breach.  Their answer: automated testing tools.  It feels like an Agile answer to me — we’ve got a development problem with no obvious solution, so we’ll test and iterate.

“The talent coming out of schools right now doesn’t have the security knowledge it needs,” says Paul Kurtz, executive director at SAFECode, a nonprofit organization backed by major software vendors and focused on secure software development practices. “There needs to be a lot more work in our educational institutions to teach them how to develop secure code.”

But nearly all experts agree that no matter how strong the training effort, the average developer will never be very security-savvy. “They’re always going to be more focused on code quality and trying to meet their deadlines,” Sima says. “If I’m a developer, as soon as I’ve been assigned a project, I’m already behind. If there’s a faster way to do something, they’re going to take it, because for them speed is more important than security.”

via Why Can’t Johnny Develop Secure Software? – secure software development/Security – DarkReading.