WYSIATI: CS Teachers need to ask “What am I not seeing?”

I’m currently reading Nobel laureate Daniel Kahneman’s book, “Thinking Fast, Thinking Slow” (see here for the NYTimes book review).  It’s certainly one of the best books I’ve ever read on behavioral economics, and maybe just the best book I’ve ever read about psychology in general.

One of the central ideas of the book is our tendency to believe “WYSIATI”—What You See Is All There Is.  Kahneman’s research suggests that we have two mental systems: System 1 does immediate, intuitive responses to the world around us.  System 2 does thoughtful, analytical responses.  System 1 aims to generate confidence.  It constructs a story about the world given what information that exists.  And that confidence leads us astray. It keeps System 2 from asking, “What am I missing?”  As Kahneman says in the interview linked below, “Well, the main point that I make is that confidence is a feeling, it is not a judgment.”

It’s easy to believe that University CS education in the United States is in terrific shape.  Our students get jobs — multiple job offers each.  Our graduates and their employers seem to be happy.  What’s so wrong with what’s going on? I see computation as a literacy. I wonder, “Why is our illiteracy rate so high? Why do so few people learn about computing? Why do so many flunk out, drop out, or find it so traumatic that they never want to have anything to do with computing again?  Why are the computing literate primarily white or Asian, male, and financially well-off compared to most?”

Many teachers (like the comment thread after this post) argue for the state of computing education based on what they see in their classes.  We introduce tools or practices and determine whether they “work” or are “easy” based on little evidence, often just discussion with the top students (as Davide Fossati and I found). If we’re going to make computing education work for everyone, we have to ask, “What aren’t we seeing?”  We’re going to feel confident about what we do see — that’s what System 1 does for us.  How do we see the people who aren’t succeeding with our methods?  How do we see the students who won’t even walk in the door because of how or what we teach? That’s why it’s important to use empirical evidence when making educational choices. What we see is not all there is.

But, System 1 can sometimes lead us astray when it’s unchecked by System 2. For example, you write about a concept called “WYSIATI”—What You See Is All There Is. What does that mean, and how does it relate to System 1 and System 2?

System 1 is a storyteller. It tells the best stories that it can from the information available, even when the information is sparse or unreliable. And that makes stories that are based on very different qualities of evidence equally compelling. Our measure of how “good” a story is—how confident we are in its accuracy—is not an evaluation of the reliability of the evidence and its quality, it’s a measure of the coherence of the story.

People are designed to tell the best story possible. So WYSIATI means that we use the information we have as if it is the only information. We don’t spend much time saying, “Well, there is much we don’t know.” We make do with what we do know. And that concept is very central to the functioning of our mind.

via A machine for jumping to conclusions.