Archive for March, 2018

How to Study for a CS Exam

I just recently finished Make It Stick (Amazon link) which takes modern learning science and develops recommendations to learners on how to study in order to learn most effectively. As I read it, I was wondering, “What are the implications for how students should study for a computer science exam or quiz?” I did a Web search for “computer science study skills,” and the results were disappointing. Most of the recommendations are wrong (e.g., “take good notes” and “highlight passages in the book”) and were not specific to learning computer science.

So, I wrote up some recommendations for my students. I have not tried to generalize these yet (but would be fun and useful to do so), and I haven’t gone back through Making it Stick to mine it for all the CS recommendations that one might invent from their lessons.

Caveat: these are “evidence-based” as in the learning sciences behind them is pretty solid, but they are not “evidence-based” in the sense of actually testing these practices with learning CS. We know that some learning sciences or educational psychology results from other fields don’t map cleanly to CS (see Briana’s work). I don’t know of research evaluating study practices for learners studying computer science — but it would be great to have some!

I’d be interested to hear how you help your students study computer science!  Please do share your ideas and recommendations in the comments.

How to study for a programming quiz

Here are my suggestions for how to study for Monday’s quiz (with some advice that is useful for all your other CS quizzes).

1. Read the Book. Once

If you haven’t read the relevant chapters, do read them. If you have already, I don’t suggest re-reading them. There are better ways to study than re-reading.

2. Do Your Programming Homework

The Collage Project (and all your other homework) is good preparation for Monday’s Quiz. Finish that up this weekend, and you’ll have reviewed many of the ideas needed for Monday’s Quiz.

3. Make sure you know what you didn’t know

Go over your homework assignments and past quizzes, and make sure that you now know everything you got wrong in the past.

4. Quiz Yourself

The best way to study is always by quizzing yourself. You’ll learn far more from quizzing yourself than from re-reading or reviewing your notes.

Use the problems at the back of the chapter. Get with others in the class and challenge each other by inventing variations on programming problems you have seen before:

  • Rather than create a negative of the whole picture, can you negate just the top half of a picture?
  • Can you move the bottom half of a picture onto the top?
  • Can you mirror a square picture diagonally?
  • Can you put a 100-pixel wide border of grayscale around a picture?

5. Elaborate on each Chapter

At the beginning of each chapter is a list of Learning Objectives — media learning goals and CS learning goals. Write a sentence or two demonstrating that you have learned those objectives. Explain what the goal means, and show that you have achieved those goals. Elaboration helps you to connect ideas so that it’s easier to remember those ideas later.

At the end of each chapter is the Programming Summary that lists the new programming ideas (new functions, new control structures) in that chapter. Make sure that you can use each of those. Just write a program, any program that uses each function at least once.

6. Try things in more than one way

In this class, you know more that one way of doing things.

  • You can get and set colors with setRed, getRed, setBlue, getBlue, setGreen, getGreen. OR getColor, setColor, makeColor. Make sure that you can use these interchangeably. Write increaseRed and decreaseRed using setColor. Write negation and grayscale with both sets.
  • You know several ways of accessing individual pixels.
    • You can use for pixel in getPixels(picture).
    • You can get all the pixels in a list with all = getPixels(picture) then generate the indices for that list with for index in range(0,len(all)): and access each pixel with pixel = all[index].
    • You can generate x and y coordinates with for loops, then get individual pixels with getPixel(picture,x,y).
    • Make sure that you can use all three well. Write chromaKey with all three forms. Create a green border around a picture using all three forms.

Take any program that you have written before, or that we have done in class, and rewrite it another way. This is both a form of elaboration and a form of quizzing.

7. When confused, ask and talk CS

Ask on Piazza. Ask your friends. Come to office hours. When studying, all collaboration is strongly encouraged.

Just talking through things (both what you understand and what you do not understand) with anyone will help you learn.  Explain your programs to someone else.  Summarizing your programs in English is a great way to make sure you understand the programs and to elaborate on your knowledge.

March 30, 2018 at 7:00 am 5 comments

New programming languages are important to develop as we improve our knowledge of how students learn computing

I was at a workshop at Google a couple weeks ago where someone asked me, “Do you still think that there’s a place for developing new programming languages in computing education?” I said, “ABSOLUTELY!”.

We know little about how people learn programming, and developing new programming languages is important for improving usability, learnability, and productivity of programmers (professional, novice, end-user, casual, or conversational). The interplay between design of programming languages and research into how people learn programming languages is a hot and important research topic. (See, for example, the recent Dagstuhl seminar on empirical data for programming language design.)

My Blog@CACM post for this month (see link here) is based on the cover story for the March Communications of the ACM (CACM), on “A Programmable Programming Language.” The (interesting and recommended) article is on building problem-specific programming languages. My post was about the educational questions raised by these languages. Would they be easier or harder to learn if they’re problem-specific? Will novices be willing to put in the effort to learn a programming language that is specific to a problem? Do problem-specific languages make it harder or easier to find (or train) programmers to work on old software (built in these problem-specific languages)? If a programmer learns a problem-specific programming language created at Company X, then leaves for Company Y and creates a similar problem-specific programming language, was intellectual property stolen?

Barbara Ericson’s defense was March 12 (as mentioned here). It was very successful — not only did she pass, but all of her committee signed off on the same day. She’s Dr. Ericson!

Alan Kay was on her committee and asked some insightful questions about her work with Parsons problems. In a Parson problem, students are ordering lines of code into a correct solution. Barb did her research using Python, and she’s also done work with Parsons problems in Java. These are pretty similar languages in terms of notional machines.

What’s the influence of the programming language on student success with Parsons problems? What if the underlying notional machine was simpler to understand? Would students find it easier to sequence a program? In general, we explore non-imperative programming paradigms so rarely in computing education research. We change modality (e.g., Scratch), but not the underlying computational model. The work with Racket is a rare example. Alan mentioned HyperCard in his comments, which was explicitly designed to be easy to learn. Would HyperCard programs be easier for students to order correctly?

I hope that we continue to invent new programming languages and explore the educational implications of them. There’s a big space of possible designs, and we have only started evaluating them empirically.

March 26, 2018 at 7:00 am Leave a comment

When more information leads to worse performance: Beware throwing in “something fun and totally optional”

Eliane Wiese gave a talk here this last week. She told a story that I found fascinating. It connects to a story I just read about from Kahneman and Tversky. The theme has important implications for the design of software for CS education.

Story One: In Eliane’s dissertation work she explored how to give grounded feedback that would lead students to learn from mistakes. Here (in summary form) is the result of one of her studies.

In some questions, students were shown graphical representations of fractions. In other questions, they were shown some combination of graphical representations and symbolic fractions. In a fourth kind of questions, they’re just shown symbolic fractions. The vertical axis is performance.

The part that I find amazing is the results for condition two and three for fraction addition. Getting more information led to worse performance. Symbolic fractions are so confusing that their appearance depresses performance, even when the graphical information is still there. The students don’t just ignore the fractions. The mere presence of the fractions makes the problem harder for students.

(Original paper available here. Her follow-up/replication study can be found here. Thanks to Eliane for reviewing this post and sending me these links!)

Story Two: I just finished reading The Undoing Project (Amazon link) by Michael Lewis, the story of Daniel Kahneman and Amos Tversky’s amazing collaboration and friendship. One of their experiments is particularly relevant to Eliane’s finding.

You tell people that they’re going to pick a person at random from a pool of 100 people, 70 of whom are engineers and 30 of whom are lawyers. What is the probability that you’re going to get an engineer? Participants in the studies correctly guess 70%. You can change it to lawyers, or change around the ratios, and people solve this problem correctly and easily.

Now you tell them that, from the same pool, they have selected “Dick.”

Dick is a 30 year old man. He is married with no children. A man of high ability and high motivation, he promises to be quite successful in his field. He is well liked by his colleagues.

Now, what is the probability that Dick is an engineer? Participants say that the probability is 50% — they can’t tell. Notice that the description of Dick offers no additional information to discern if he is an engineer or a lawyer. Yet, people can’t ignore the useless descriptive information. They can’t just rely on the numbers. Getting more information leads to worse performance. People seem to feel a need to use all available information, even if it’s not useful, even if leads to worst performance.

What’s the implication for CS Ed? Our programming languages and professional IDE’s are complex. How about public static void main(String[] args)? How about all the bells and whistles in Eclipse?

When I point these out to teachers, the most common response I get is, “It’s okay. Students just ignore that part.”

I’m not sure that they do, or that they even can. People try to make sense of the information in front of them. We are drawn to create narratives. It is difficult for us to ignore information and make decisions based on only the relevant information. This is particularly hard for novices who don’t understand the relevant information, let alone separate the relevant from the irrelevant.

Before we toss something into our classes, we should pause and consider these stories. Sure, your CS1 students could use a cool new library that lets them do something cool (whatever — robotics, data visualizations, social network analysis) but has a confusing API and almost no documentation. The new library will consume their time and effort to understand. Sure, you might decide to introduce something (maybe list comprehensions or lambda expressions) into your Python code, just as “something fun” and “totally optional.” But students will try to understand it, and might not learn the things you really want them to learn. Sure, you could throw in a quick algorithm animation or use some super cool new debugger, but if your students are already confused, you’ve now just given them yet another representation or interface to make sense of. Think about the fact that the additional/extra/irrelevant information may be distracting your students from what is important. And that might lead to worse performance.

March 23, 2018 at 7:00 am 15 comments

Constructivism vs. Constructivism vs. Constructionism

I wrote the below in 1997. I’m surprised that I still find references to it from time-to-time. That website may be going away soon, so I thought I’d put it here (only very slightly edited) in case others may find it useful.

I’d like to offer my take on the meaning of these words. I hear them used in so many ways that I often get confused what others mean by them.

Constructivism, the cognitive theory, was invented by Jean Piaget. His idea was that knowledge is constructed by the learner. There was a prevalent idea at the time (and perhaps today as well) that knowledge is transmitted, that the learner was copying ideas read or heard in lecture directly into his or her mind. Piaget theorized that that’s not true. Instead, learning is the compilation of complex knowledge structures. The learner must consciously make an effort to derive meaning, and through that effort, meaning is constructed through the knowledge structures. Piaget liked to emphasize learning through play, but the basic cognitive theory of constructivism certainly supports learning through lecture — as long as that basic construction of meaning takes place.

I don’t know who invented the notion of Constructivism, the educational philosophy, but it says that each students constructs their own, unique meaning for everything that is learned. This isn’t the same as what Piaget said. Piaget’s theory does not rule out the possibility that you and I may construct exactly the same meaning (i.e., exactly the same knowledge constructions) for some concept or domain. The philosophy of constructivism say that learners will construct their own unique meanings for concepts, so it is not at all reasonable to evaluate students as to how well they have all met some normative goal. (Radical constructivists go so far as to say that the whole concept of a curriculum makes no sense since we cannot teach anyone anything — students will always simply create their own meaning, regardless of what teachers do.) Philosophical constructivists emphasize having students take control of their own learning, and they de-emphasize lecture and other transmissive forms of instruction. This philosophical approach gets complicated by varying concepts of reality: If we all interpret things differently, is there any correct reality?

From my perspective, the assumption of constructivists is currently an untestable hypothesis. We know of no way to peer into someone’s mental constructions. Until we can, we do not know if you and I think about the concept of velocity differently or the same.

Constructionism is more of an educational method which is based on the constructivist learning theory. Constructionism, invented by Seymour Papert who was a student of Piaget’s, says that learning occurs “most felicitously” when constructing a public artifact “whether a sand castle on the beach or a theory of the universe.” (Quotes from his chapter “Situating Constructionism” in the book “Constructionism” edited by Papert and Idit Harel.) Seymour does lean toward the constructivist learning philosophy in his writings, where he talks about the difficulty of conveying a complex concept when the reader is going to construct their own meaning. In general, though, his claim is more about method. He believes that students will be more deeply involved in their learning if they are constructing something that others will see, critique, and perhaps use. Through that construction, students will face complex issues, and they will make the effort to problem-solve and learn because they are motivated by the construction.

The confusion that I and others have about these terms stems from (a) similar looking words and (b) meaning at different levels of the word construct. Piaget was talking about how mental constructions get formed, philosophical constructivists talk about how these constructions are unique (noun construction), and Papert is simply saying that constructing is a good way to get mental constructions built. Levels here are shifting from the physical (constructionism) to the mental (constructivism), from theory to philosophy to method, from science to approach to practice.

March 19, 2018 at 9:00 am 5 comments

How CS differs from other STEM Disciplines: Varying effects of subgoal labeled expository text in programming, chemistry, and statistics

My colleagues Lauren Margulieux and Richard Catrambone (with Laura M. Schaeffer) have a new journal article out that I find fascinating. Lauren, you might recall, was a student of Richard’s who applied subgoal labeling to programming (see the post about her original ICER paper) and worked with Briana Morrison on several experiments that applied subgoal labeling to textual programming and Parson’s problems (see posts on Lauren’s defense and Briana’s).

In this new paper (see link here), they contrast subgoal labels across three different domains: Chemistry, statistics, and computer science (explicitly, programming).  I’ve been writing lately about how learning programming differs from learning other STEM disciplines (see this post here, for example). So, I was intrigued to see this paper.

The paper contrasts subgoal labeled expository text (e.g., saying explicitly as a heading Compute Average Frequency) and subgoal labeled worked examples (e.g., saying Compute Average Frequency then showing the equation and the values and the computed result).  I’ll jump to the punchline with the table that summarizes the result:

Programming has high complexity.  Students learned best when they had both subgoal labeled text and subgoal labeled worked examples. Either one alone didn’t cut it. In Statistics, subgoal labeled examples are pretty important, but the subgoal labeled text doesn’t help much.  In Chemistry, both the text and the worked examples improve performance, and there’s a benefit to having both.  That’s an argument that Chemistry is more complex than Statistics, but less complex than Programming.

The result is fascinating, for two reasons.  First, it gives us a way to empirically order the complexity of learning in these disciplines. Second, it gives us more reason for using subgoal labels in programming instruction — students just won’t learn as well without it.


March 16, 2018 at 7:00 am 4 comments

Announcing Barbara Ericson’s Defense on Effectiveness and Efficiency of Parsons Problems and Dynamically Adaptive Parsons Problems: Next stop, University of Michigan

Today, Barbara Ericson defends her dissertation. I usually do a blog post talking about the defending student’s work as I’ve blogged about it in the past, but that’s really hard with Barb.  I’ve written over 90 blog posts referencing Barb in the last 9 years.  That happens when we have been married for 32 years and collaborators on CS education work for some 15 years.

Barb did her dissertation on adaptive Parsons problems, but she could have done it on Project Rise Up or some deeper analysis of her years of AP CS analyses. She chose well. Her results are fantastic, and summarized below. (Yes, she does have six committee members, including two external members.)

Starting September 1, Barbara and I will be faculty at the University of Michigan. Barb will be an assistant professor in the University of Michigan School of Information (UMSI). I will be a professor in the Computer Science and Engineering (CSE) Division of the Electrical Engineering and Computer Science Department, jointly with their new Engineering Education Research program. Moving from Georgia Tech and Atlanta will be hard — all three of our children will still be here as we leave. We are excited about the opportunities and new colleagues that we will have in Ann Arbor.

Title: Evaluating the Effectiveness and Efficiency of Parsons Problems and Dynamically Adaptive Parsons Problems as a Type of Low Cognitive Load Practice Problem

Barbara J. Ericson

Human-Centered Computing

School of Interactive Computing

College of Computing

Georgia Institute of Technology

Date: Monday, March 12, 2018

Time: 12pm – 3pm

Location: TSRB 222


Dr. Jim Foley (Advisor, School of Interactive Computing, Georgia Institute of Technology)

Dr. Amy Bruckman (School of Interactive Computing, Georgia Institute of Technology)

Dr. Ashok K. Goel (School of Interactive Computing, Georgia Institute of Technology)

Dr. Richard Catrambone (School of Psychology, Georgia Institute of Technology)

Dr. Alan Kay (Computer Science Department, University of California, Los Angeles)

Dr. Mitchel Resnick (Media Laboratory, Massachusetts Institute of Technology)


Learning to program can be difficult and time consuming.  Learners can spend hours trying to figure out why their program doesn’t compile or run correctly. Many countries, including the United States, want to train thousands of secondary teachers to teach programming.  However, busy in-service teachers do not have hours to waste on compiler errors or debugging.  They need a more efficient way to learn.

One way to reduce learning time is to use a completion task.  Parsons problems are a type of code completion problem in which the learner must place blocks of correct, but mixed up, code in the correct order. Parsons problems can also have distractor blocks, which are not needed in a correct solution.  Distractor blocks include common syntax errors like a missing colon on a for loop or semantic errors like the wrong condition on a loop.

In this dissertation, I conducted three studies to compare the efficiency and effectiveness of solving Parsons problems, fixing code, and writing code. (Editor’s note: I blogged on her first study here.) I also tested two forms of adaptation. For the second study, I added intra-problem adaptation, which dynamically makes the current problem easier.  For the last study, I added inter-problem adaptation which makes the next problem easier or harder depending on the learner’s performance.  The studies provided evidence that students can complete Parsons problems significantly faster than fixing or writing code while achieving the same learning gains from pretest to posttest.  The studies also provided evidence that adaptation helped more learners successfully solve Parsons problems.

These studies were the first to empirically test the efficiency and effectiveness of solving Parsons problems versus fixing and writing code.  They were also the first to explore the impact of both intra-problem and inter-problem adaptive Parsons problems.  Finding a more efficient and just as effective form of practice could reduce the frustration that many novices feel when learning programming and help prepare thousands of secondary teachers to teach introductory computing courses.

March 12, 2018 at 7:00 am 7 comments

Exploring the question of teaching recursion or iterative control structures first

Someone raised the question on the SIGCSE Members list: Which should we teach first, iteration or recursion?

I offered this response:

The research evidence suggests that one should teach iterative control structures before recursion, IF you’re going to teach both.  If you are only going to teach one, recursion is easier for students.  If you teach recursion first, the evidence (Kessler & Anderson, 1986; Wiedenbeck, 1989) suggests that it becomes harder to learn the iterative control structures.

The push back I got was, “Surely, we have better data than 30 year old studies?!?”  Here was my reply:

I agree that it would be great to do these studies again.  Given that we have an experiment and a successful replication, it could be an MS or advanced undergrad project to replicate one of those earlier experiments.

For myself, I don’t expect much difference.  As you say, student brains have stayed the same.  While the languages have changed, the basic iterative control structures (for, while, repeat) haven’t changed much in modern languages from what they were in C and even Pascal.  Curriculum may be a factor, and that would be interesting to explore.

Two directions that I think would be great to explore in this space:

(1) The Role of Block-Based Languages: As you say, the previous research found that iterative control structures are syntactically complicated for novices.  But multiple studies have found that block-based iterative structures are much easier for novices than text-based versions.  What if we went recursion->block iteration->text iteration?  Would that scaffold the transition to the more complicated text-based iterative control structures?

(2) The Role of High-Level Functions: I don’t know of any studies exploring high-level functions (like the ones that Kathi Fisler used to beat the Rainfall Problem, or even map/reduce/filter) in the development of understanding of recursion and iterative control structures.  High-level functions have a fixed form, like for/repeat/while, but it’s a simpler, functional form.  Could we teach high-level functions first, to lead into recursion or iterative control structures?  Or maybe even teach recursion or iterative control structures as two different ways of implementing the high-level functions?

In general, there are too many questions to explore and too few people asking these questions with empirical data. We might rely on our teaching experience to inform our answers to these questions, but as Neil Brown showed us (see CACM Blog post this month that talks about this result), higher-education CS teachers are actually way off when it comes to estimating what students find hard.

SIGCSE-Members, please consider asking some of these questions on your campus with your students. There are well-formed questions here that could be answered in a laboratory study that could be encapsulated in a single semester.  The students will get the opportunity to do empirical research with humans, which is a useful skill in many parts of computing.

March 9, 2018 at 7:00 am 3 comments

Older Posts

Recent Posts

March 2018
« Feb   Apr »


Blog Stats

  • 1,498,647 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 5,248 other followers

CS Teaching Tips