Is Liveness a critical factor in learning Computer Science? Context, motivation, and feedback for learning programming

November 30, 2018 at 7:00 am 17 comments

My CACM Blog post for November is on the topic of Direct Instruction, why it’s better than Discovery Learning, and how we should teach programming “directly.”

I wonder about the limitations of Direct Instruction.  I don’t think everything can be learned with direct instruction, even with deliberate practice.

At SIGCSE 2016, John Sweller made a provocative claim (that I haven’t yet found in his published papers).  He said that humans must be able to learn higher-order thinking skills.  We’d be dead if we didn’t. However, we cannot teach them.  Students have to figure them out from experience. Is programming a similar kind of task?

I have been studying Spanish on a streak of over 600 days in DuoLingo now. DuoLingo is the best direct instruction I’ve ever had.  Everything I do is deliberate practice — it’s really good at figuring out what I’m not good at, and giving me more problems on that.  I am nowhere near fluent.  I know some words. I can read some. I’m getting better at hearing. I am not fluent.  Maybe learning natural and programming languages both require more than direct instruction.

What leads to fluency, in natural languages or programming languages?  I suspect that part of it is context and motivation.  You have to be in a position to want to say something (in a natural or programming language) in order to learn it.

But I also think it’s about feedback.  I don’t really learn Spanish well because I’m rarely in a position to use it. If I did, I’d get a response to what I said. Can anyone learn to program without trying to write some code and getting feedback on whether it works? The issue of feedback came up several times in the recent discussion about the relationship between teaching programming and teaching composition.

Steven Tanimoto talks about the value of “liveness” in a programming environment (see paper here), which is about the ease of writing code and getting different kinds of feedback on the code. Maybe Liveness encapsulates the kinds of things we need for successful CS learning. Of course, even “liveness” doesn’t give the kind of feedback that a human reader can, but it does shorten the feedback timing loop.



Entry filed under: Uncategorized. Tags: , , , .

African-Americans don’t want to play baseball, like women don’t want to code: Both claims are false The systemic factors that limit Black participation in the Tech sector

17 Comments Add your own

  • 1. Tom Morley  |  November 30, 2018 at 8:10 am

    Consider Jazz improvisation. Another fascinating case. There’s all sorts of rules and exercises and direct experiences. What works? And is there a difference between “sounds ok” and “that’s really good”, and if so what? And is it teachable?

  • 2. Bonnie  |  November 30, 2018 at 8:22 am

    Feedback should be about a lot more than just “does the program work”. You can get that information from an autograder. Feedback to students should be similar to the feedback given during code reviews at companies, and should focus on code quality. Obviously the definition of code quality will vary depending on the level of the student, but would typically include things like:
    (for beginners)
    is the program readable (hopefully without needing comments)?
    is the program designed in a straightforward way?
    (for intermediate students)
    is the program easy to modify?
    Is the program testable?
    is the program free of security risks?
    does the program use appropriate data structures?

    and so on.
    I was a member of the working group at ITiCSE2017 that looked at differences in perception of code quality between students, professors, and professionals. We still have a mound of data waiting to be analyzed, but we saw some real differences even in the initial analysis. Students and faculty tend to look at small scale measures like variable names, use of functions or methods, and comments. Professionals tend to look at larger scale measures, and also think about time-based measures such as maintainability and modifiability. While there are automated tools to measure some aspects of code quality, the reality is that humans are needed to do a lot of it. And that is why so many companies use code reviews by humans. I think this needs to be brought into computer science education.

    I know you tend to focus on teaching programming to non CS majors, and that you and others argue that teaching quality factors is not important for this group. But I will maintain that anybody who is going to be writing code that will be used by others, or who will be overseeing people who are writing code to be used by others, needs to have some acquaintance with the concepts of code quality, especially testing and readability.

    Back to the natural languages – I agree that you can’t learn a language through just directed practice. I took several years of French in middle and high school, but I didn’t actually learn any French until I moved to France, and had to go to school there. The central goal of a language is to communicate ideas, which means you need a listener or a reader. There is something else, though, that is important. When you first move to a foreign country and really have to communicate, you tend to forget about “quality” and just focus on getting the idea out. I definitely did a lot of that in France. But French people are not shy about correcting grammar, use of idioms, and so on. So over time, the quality of my French (both spoken and written since I was in school) really improved a lot. You can’t really be fluent in a foreign language unless you pay attention to the quality of your language use. I would argue the same is true for programming languages

    • 3. Mark Guzdial  |  November 30, 2018 at 10:38 am

      You raise some great points, Bonnie. Many years ago, two of my students did a Christmas-break study where they downloaded all the code from a Photoshop scripting community repository, and reviewed the code quality there (see Most of it was just horrible — literally, just recordings of the screen (move mouse here, click here, do these keystrokes) which was easy to generate in Photoshop, but was not readable, modifiable, and barely testable. Some of it was a slight step over that, e.g., some of the recordings were parsed into parameterized functions with reasonable names. Some of the code was very good. I wanted to tell a story about end-user programmers starting with recordings, then playing with functions, and eventually learning good form. Allison and Brian pushed back — we didn’t have any data than anyone followed that trajectory. It might be just as reasonable to expect that most people who saw the illegible code were turned off, went away, and never learned to program. They’re right, of course. We don’t know.

      What I will continue to push back on is requiring good code as students are first starting to learn. We don’t ask anyone to learn to read and write in grammatically correct sentences as they start — as your French story describes. If we did, we’d turn people off of reading and writing as much as the illegible code might.

      I see your points, but think we need a middle ground.

      On the issue of feedback: Completely agreed. I’m quoted in Inside Higher Ed today on exactly that point.

      • 4. Bonnie  |  November 30, 2018 at 12:30 pm

        I really like that article. In general, I hate it when course rely on autograders exclusively for grading. I consider it to be educational malpractice. When I was a grad student, I TA’ed for one of those huge CS1 courses, and we were still able to give feedback to the students. They did it simply by having lots of TAs for the course, and a tightly written rubric for every project. Yes, it costs money to hire TAs and train them to do this, but universities keep claiming that computer science is very important to them. And we don’t do ourselves or our students any favors by letting administrators think we can teach effectively without feedback.

        • 5. gasstationwithoutpumps  |  November 30, 2018 at 1:26 pm

          Training the TAs is difficult, because most of them have never read a program. We teach programming as if it were writing, but without first (or ever) teaching reading. Most CS majors and most professional programmers have never seen examples of excellent programming—many have never even seen examples of good programming.

          That said, I’m in 100% agreement that sole reliance on auto-graders is educational malpractice.

        • 6. cycomachead  |  December 2, 2018 at 1:31 am

          (I was at Berkeley for a while and have many friends who are TAs there.)

          There are lots of TAs for a course like CS61B and they get quite a bit of opportunity for training. (And yes, the department does spend tons of money…)

          I 100% agree that autograder’s shouldn’t be the only feedback. I think that when done well, autograding + humans do a better job than either alone. For BJC we’ve experimented with using autograder’s and only exposing them to TAs, just to aid in their grading and feedback.

          I think IHE’s article hits on the real big problems but only briefly:
          This was a problem with too many new and under tested projects. Autograder or not, giving “beta” versions of assignments invariably leads to challenges.
          And as the quoted student mentions — for most students, it’s not the grade but the stress (and related confusion, etc) that’s the real problem. This is doubly impacted by a culture which is already pushing too many students to their limits.

          There’s lots more minor things about this case that can be said…because there’s two levels of technical problems and the ones like the system crashing can be helped by new infrastructure which UCB already has access to.

      • 7. cycomachead  |  December 2, 2018 at 1:36 am

        I can totally see where you’re coming from with Photoshop recording. I think one thing that makes such analysis more complex, especially with highly specific programs like a Photoshop script, is that the purpose and expectations of the author matter.

        Valuing code quality makes sense when you need a program to live for a long time and be adapted, especially if by others. But as someone who’s done a tiny bit of Photoshop scripting — I don’t really care about the quality there. If I need to make a big enough change, it’s usually easier and faster to re-record than edit a script, but for small changes, edits are fine and I won’t bother cleaning up the code.

        When starting out, there’s many reasons to want to learn to program and many different goals.

    • 8. gasstationwithoutpumps  |  November 30, 2018 at 1:35 pm

      “is the program readable (hopefully without needing comments)?” seems to me to miss the point of comments. Comments are for explaining the *point* of programs, parts of programs, and data structures—they are for expressing those things that are not directly expressible in the programming language: perhaps emergent properties.

      So wanting a program to be readable without comments is equivalent to saying that the program should be so trivial that a programmer can hold the whole thing in their head and know what all parts of it do and what the point of it all is. Even a first course in programming should be requiring students to produce more complicated programs than that.

      We might be better off requiring students to produce just the comments for a piece of code than just the code without comments, at least in terms of getting them to think about what they are doing, rather than just randomly modifying code until it passes an autograder.

      • 9. Bonnie  |  November 30, 2018 at 4:36 pm

        I agree that comments should explain, but comments are notorious for going out of date, so many software professionals prefer to minimize comments. API style comments, for example Javadoc, are something else – those comments, aimed at users of an API, are expected and important. But the little inline comments that say // loop termination, or //holds XYZ value
        no, just write the code so you don’t need those.

        The nirvana we want to aim for is self documenting code. Books like Clean Code, which are used by practitioners, talk about this.

        • 10. gasstationwithoutpumps  |  December 3, 2018 at 12:02 pm

          I agree that the low importance given to comments by programmers (and their teachers) means that comments are not properly maintained. I have yet to see any non-trivial examples of “self-documenting code” that so many people hold up as an ideal. Good variable names are certainly important, but they are not sufficient.

          I agree that most in-line comments are useless, because students have only seen comments used to teach a programming language, and so the comments they use “explain” what should already be obvious. That is why I teach students to use block comments (or Python docstrings) to establish the intent of a block of code before the code itself.

          • 11. orcmid  |  December 3, 2018 at 5:30 pm

            I’m with @gas on this. Anything to establish what the code is for, not what it is, and then how what it is connects to what it is for. Without that, maintenance without doing damage to the purpose and invariants that apply is extremely risky. Such comments should not become obsolete unless there is some sort of serious re-engineering going on.

  • 12. alanone1  |  November 30, 2018 at 1:07 pm

    Two ponderings:

    1. Let’s suppose that we were worrying about teaching the spectra of health care including basic care, first aid, all the way to various forms of medicine, and to biological and medical research.

    In my imagination, not only are there many different kinds of students involved here, but part of the responsibilities of the educators are to carefully try to exclude those who are not up to the demands and requirements of the eventual “processes out in the world” — and “not up to” includes “not up to morally” as well as “not up to technically”. Part of the responsibilities of government are to carefully try to certify and continuously vet those who attempt to provide services of any and all kinds for public health.

    Analogies like these peter out quickly, so I won’t push this one further. A basic idea here is that there are a few important things that all citizens need to understand, but there is no larger imperative to have all citizens go much farther in ad hoc understanding — some form of vetted professionalism is needed, both for care and for education/training of care givers.

    2. A revelation in the Parc days came a year after the first Ethernet was working perfectly. Bob Sproull did some testing and discovered a bug in the code — he fixed it and sped it up by a factor of 2. No one noticed the bug before, and no one noticed the fix. During this process, one Alto was discovered to have almost no packet volume — the owner wasn’t aware of it. Examination revealed that the “stinger” into the coax had bent and missed the core, so the signal to noise ratio was very close to zero. Nonetheless every once in a while a packet got squeezed out and in and the files the owner was trying to print got printed.

    At the same time, some of us were working on both Parc’s own “internetworking system” (PUP) and were on the official Internet group, and some of us were working on a continuously working programming language — Smalltalk — that could both fix itself while running, and could also bootstrap the next version of itself while running.

    We realized that it could be possible via more careful systems design — and would be necessary giving the immense scaling that was happening via Moore’s Law — to “make systems which didn’t crash”. Bugs, etc., would slow things down but couldn’t be allowed to affect the overall integrity of the system.

    This was easier to think about vis a vis networking as opposed to making a programming system, but there were already some very good ideas about this at various levels within a programming system, and we could all see that interactive systems for end-users just couldn’t be allowed to fail no matter what they or the system tried to do, and that extending this not just to end-user programming, but to more “professional programming” would be a great leap.

    And, in fact, this can be done.

    Now, I can see why a trade school might want to teach poorly designed systems because they are in business use. But why should a university? The latter are supposed to be about knowledge and epistemological frameworks, not primarily vocational preparation.

    Another way to look at it is via (1.) if you have to deal with something that has quite a bit of unhealthiness built into it via past poor design, it means that you need really special training to have any chance at safety (this is rather like what we actually had to learn ca. 55 years ago when most code was still in machine language form and programmers had to learn to safely control many more degrees of freedom than were in HLLs, and especially the start of VHLLs.

    • 13. orcmid  |  November 30, 2018 at 3:03 pm

      Thanks Alan. I think the trade-school attribution is unnecessary. The observation about degrees of freedom at the machine language level is spot on.

      This is memorable: ” A basic idea here is that there are a few important things that all citizens need to understand, but there is no larger imperative to have all citizens go much farther in ad hoc understanding.” Amen.

  • 14. orcmid  |  November 30, 2018 at 2:55 pm

    When I first learned of this in DLisp I thought it was interesting. When I hear of it as something to do with production software (ignoring the fascinating use in performing arts and music), I find that worrisome.

    I also wonder if it has attention at the wrong level, such as using a debugger to learn a program, rather than in the context of what a program is for and is it suitable to purpose. And I worry about obsessiveness around live coding. (I think liveness is an inappropriate choice of language.)

    Setting all that aside, I found Tanamoto’s 2013 Workshop paper of interest. And the recent paper “Coding, Reading and Writing: Integrated Instruction in Written Language” might be even more valuable in the context of this blog.

    Since Tanamoto is nearby, I look forward to hearing from him on his latest thinking.

  • […] 30, 2018 This post was originally published on this […]

  • 16. alanone1  |  December 3, 2018 at 1:51 pm

    Backing up a bit, let’s talk about a train that has left the station, and whether it can return — namely, what universities are primarily supposed to be doing, and what education is supposed to mean at a university. Have universities completely capitulated into being trade-schools and have they traded in-depth education for “training” for jobs?

    I think this is especially tricky at a place such as Georgia Tech, where the implication in the title, and its engineering associations, implies “practicality” and “real-world-applications”.

    I’m a holdout for “Computer Science” being much more than about programming, and “Software Engineering” also being a lot more than about programming — and programming as being much more than learning to program in vendor’s language X.

    I’ve also recently switched over to using medicine as a metaphor. For example, to what extent do we want undergraduate degrees in biology out practicing medicine? Even with some extra training? (Could make some sense in a completely dire emergency, but I think we want practitioners in something vital to have much more knowledge and experience under their belts before they do much work on people.)

    Since we are discussing programming here in the context of “education” to be taught by people who are dedicating their life to be educators (not just “trainers”) we should look at some of the added implications and things that are needed.

    I’m grateful to have learned to program in the early 60s when it was completely clear that “programming languages were terrible and inadequate to the needs and demands”. These languages were “machine languages” with a little “assembler paint” on top of them. Because the machines were different (often very different back then) it was not worthwhile to fasten on any one of them — one had to make something in one’s mind to “think about programming with” and then map that to the machine at hand. The internal stuff was more abstract and programming was a combination of designing in it and then hand compiling to the particular machine.

    This led to two main styles (which were not independent): (a) a very careful way to write code, and (b) transportable higher level abstractions.

    Method (a) was quite fragile in many ways, in no small part because extreme optimizations were often called for on those tiny slow machines, and these usually violated different kinds of module boundaries in various ways. This made code very hard to read per se.

    Method (b) wound up being more powerful and tractable.

    Because many of the assemblers had macro facilities, most programmers made macros for their main abstractions. On a new machine, one would take several weeks to implement and carefully test one’s collection of macros, before starting to program.

    In programming, there is “what?”, “why?”, and “how?”. Most features in most programming languages can only express “hows” (and this dims even for the original programmers). To understand code, one has to form a picture of both “what?” and “why?” — that is: intentions, goals, heuristics, etc. (Harry Huskey’s version of Algol 58 — Neliac — allowed arbitrary length labels, and the style was to write a comment as a label for each section of code.)

    I’m not aware of a programming language around today that is at all good at both expressing intentions and also being machine interpretable, so I think forcing commentary on main sections (as Kevin has mentioned in one of his comments here) is mandatory.

    A middle ground is what macros led into in the 60s — extensible languages. Macro languages have a real flaw in that the structure of the macro is lost in macro expansion, and debugging gets worse. A real extensible language retains meaning and the extensions are fully fledged elements recognized by the debugger.

    Much discipline has to be taught with an extension facility (it is all too easy to come up with extensions that (a) don’t pay their way, and (b) turn the language into a new Babel).

    However, if we take “Computer Science” and “Software Engineering” and “Systems Design” etc. seriously, we can see that a language capable of graceful extensions can make all the difference when used as part of a larger design discipline.

    It can even bridge a good part of the gap between the “hows” and the “whats” and “whys”.

    The line I’m pursuing here is quite at odds with teaching the fad language of the day and how to defensively program in it.

    What the field needs is more and more people who have learned how to design systems and program them in general, and who can use abstraction facilities for making much more separable and understandable systems.

    This sounds like a proper job for university education, and it is something that needs to be re-affirmed even while so many are in the midst of a mad rush for financial gains from vocational training.

    • 17. orcmid  |  December 3, 2018 at 5:55 pm

      “I’m a holdout for ‘Computer Science’ being much more than about programming, and ‘Software Engineerin.” also being a lot more than about programming — and programming as being much more than learning to program in vendor’s language X …

      “In programming, there is “what?”, “why?”, and “how?”. Most features in most programming languages can only express “hows” (and this dims even for the original programmers). To understand code, one has to form a picture of both “what?” and “why?” — that is: intentions, goals, heuristics, etc. [Considering that underneath, Neliac was as close to machine language as C, I loved the arbitrary-length labels.] …

      “[If] we take “Computer Science” and “Software Engineering” and “Systems Design” etc. seriously, we can see that a language capable of graceful extensions can make all the difference when used as part of a larger design discipline.”

      My experience is similar and I am completely aligned with you on these points. I am a bit nervous about extensibility when it hides how, and there are the debugging, verification, and maintenance issues @Alan touches on. I am not so wary if there is some treatment and that “larger design discipline.”

      Coming back to the idea of CS4all and CS Education for all, we are not talking about much CS so maybe what we are talking about some kind of precursor fluency, not unlike the three Rs.

      Sometime I think early exposure to programming is maybe just one rung up Wittgenstein’s ladder, to be thrown away in order to achieve the next level. Maybe that’s how some of us with lengthy experiences and introspection can have a view back down the ladder from a different perspective. I’m not clear how to work that into some rational levels of curricula.

      We’ve already witnessed what a poor idea is for someone to have a smattering of programming exposure and think that is enough for the big picture. How many non-programming managers expect “just a small matter of programming” and how District Court Judges can mislead themselves (even though the judgment might be correct)?

      At the same time, I’m not at all clear how there can be meaningful introductory exposure to Computer Science and/or Software Engineering without programming of some useful, re-usable form.

      I worry that Liveness might not be a virtue in this case.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 9,038 other followers


Recent Posts

Blog Stats

  • 2,014,284 hits
November 2018

CS Teaching Tips

%d bloggers like this: