Call for AP CS Principles Readers

Guest blog post from Barbara Ericson, copying a message from Paul Tyman:

Please consider signing up to an an AP CS Level A or CS Principles reader. We will need lots of new readers for the CSP exam. I did the pilot reading last year and it was interesting to see what the students submitted for their paper about a computing innovation and their code for the create task. The readings are really a great professional development opportunity for you. There is always an invited speaker and demos in the evenings. You will meet lots of great people who care about computer science education, both in high school and higher education teaching. We have a social space in the evenings which is quite busy with lots of card games, board games, and music. There are also groups who walk, do yoga, run, etc. They pay for your travel, hotel, meals, and pay you a stipend as well.

Barb Ericson
Georgia Tech

From: Paul Tymann <>

Sent: Saturday, October 1, 2016 9:54 AM
Subject: CSP Readers Needed!!


Current estimates indicate that we will need more than 200 readers to score the AP CS Principles exam that will be administered in May 2017. I need your help recruiting new readers. Could you reach out to a couple of your colleagues and encourage them the apply to be readers? As former readers you are in an unique position to explain the reading process and the benefits of participating.

Potential readers can find out more information about becoming an AP CSP reader, and more importantly can sign up to become a reader, by pointing a browser to:

Please contact me if you have any questions. I hope to, no will, see you in Kansas City!!


October 5, 2016 at 7:54 am 1 comment

How to Write a Guzdial Chart: Defining a Proposal in One Table

In my School, we use a technique for representing an entire research proposal in a single table. I started asking students to build these logic models when I got to Georgia Tech in the 1990’s. In Georgia Tech’s Human-Centered Computing PhD program, they have become pretty common. People talk about building “Guzdial Charts.” I thought that was cute — a local cultural practice that got my name on it.

Then someone pointed out to me that our HCC graduates have been carrying the practice with them. Amy Voida (now at U. Colorado-Boulder) has been requiring them in her research classes (see syllabus here and here). Sarita Yardi (U. Michigan) has written up a guide for her students on how to summarize a proposal in a single table. Guzdial Charts are a kind of “thing” now, at least in some human-centered computing schools.

Here, I explain what a Guzdial Chart is, where it came from, and why it should really be a Blumenfeld Chart [*].

Phyllis Teaches Elliot Logic Models

In 1990, I was in Elliot Soloway’s office at the University of Michigan as he was trying to explain an NSF proposal he was planning with School of Education professor, Phyllis Blumenfeld. (When I mention Phyllis’s name to CS folks, they usually ask “who?” When I mention her name to Education folks, they almost always know her — maybe for her work in defining project-based learning or maybe her work in instructional planning or maybe her work in engagement. She’s retired now, but is still a Big Name in Education.) Phyllis kept asking questions. “How many students in that study?” and “How are you going to measure that?” She finally got exasperated.

She went to the whiteboard and said, “Draw me a table like this.” Each row of the table is one study in the overall project.

  • Leftmost column: What are you trying to achieve? What’s the research question?
  • Next column: What data are you going to collect? What measures are you going to use (e.g., survey, log file, GPS location)?
  • Next column: How much data are you going to collect? How many participants? How often are you going to use these measures with these participants (e.g., pre/post? Midterm? After a week delay?)?
  • Next column: How are you going to analyze these data?
  • Rightmost column: What do you expect to find? What’s your hypothesis for what’s going to happen?

This is a kind of a logic model, and you can find guides on how to build logic models. Logic models are used by program evaluators to describe how resources and activities will lead to desired impacts. This is a variation that Phyllis made us use in all of our proposals at UMich. (Maybe she invented it?) This version focused on the research being proposed. Each study reads on a row from left-to-right,

  • from why you were doing it,
  • to what you were doing,
  • to what you expected to find.

When I got to Georgia Tech, I made one for every proposal I wrote. I made my students do them for their proposals, too. Somewhere along the way, lots of people started doing them. I think Beth Mynatt first called them “Guzdial Charts,” and despite my story about Phyllis Blumenfeld’s invention, the name stuck. People at Georgia Tech don’t know Phyllis, but they did know Guzdial.

Variations on a Guzdial Chart Theme

The critical part of a Guzdial Chart is that each study is one row, and includes purpose, methods, and expected outcome. There are lots of variations. Here’s an example of one that Jason Freeman (in our School of Music) wrote up for a proposal he was doing on EarSketch. He doesn’t list hypotheses, but it still describes purpose and methods, one row per study.

In Sarita’s variation, she has the students put the Expected Publication in the rightmost column. I like that — very concrete. If you’re in a discipline with some clearly defined publication targets, with a clear distinction between them (e.g. , the HCI community where Designing Interactive Systems (DIS) is often about process, and User Interface Software and Technology (UIST) is about UI technologies), then the publication targets are concrete and definable.

My former student, Mike Hewner, did one of the most qualitative dissertations of any of my students. He used a Guzdial Chart, but modified it for his study. Still one row per study, still including research question, hypothesis, analysis, and sampling.

I still use Guzdial Charts, and so do my students. For example, we used one to work through the story for a paper. Here’s one that we started on a whiteboard outside of my office, and we left it there for several weeks, filling in the cells as they made sense to us.


A Guzdial Chart is a handy way of summarizing a research project and making sure that it makes sense (or to use when making sense), row-by-row, left-to-right.



[*] Because Ulysses now makes it super-easy to post to blogs, and I do most of my writing in Ulysses, I accidentally posted this post to Medium — my first ever Medium post.  I wanted this to appear in my WordPress blog, also, so I decided to two blog posts: The Medium one on Blumenfeld Charts, and this one on Guzdial Charts.

October 3, 2016 at 7:05 am 2 comments

No, Really – Programming is Hard and CS Flipped Classrooms are Complicated: ITICSE 2016 Award-Winning Papers

I only recently started digging into the papers from the ITICSE 2016 conference (see Table of Contents link at ACM Digital Library here).  There were two papers that caught my attention.

First, the best paper award went to one of my former PhD students, Brian Dorn: An Empirical Analysis of Video Viewing Behaviors in Flipped CS1 Courses, by  Suzanne L. Dazo, Nicholas R. Stepanek, Robert Fulkerson, and Brian Dorn.  Brian has this cool piece of technology where students can view videos, annotate them, be challenged to answer questions from specific places, and have discussions.  They used this for teaching a flipped CS1 class, where students were required to watch videos before class and then engage in more active learning opportunities in class.  The real trick, as you might imagine and that the paper goes into detail on, is getting students to watch the video.  I liked both the techniques for prodding students to watch videos and the fascinating results showing the relationship between watching the videos and learning.

ITICSE 2016 recognized two “commended” papers this year.  I haven’t found the listing of which papers they were, but I did learn that one of them is Learning to Program is Easy by Andrew Luxton-Reilly.  I enjoyed reading the paper and recommend it — even though I disagree with his conclusions, captured in the paper title.  He does a good job of exploring the evidence that programming is hard (and even uses this blog as a foil, since I’ve claimed several times that programming is hard), and overall, is a terrific synthesis of a bunch of computing education papers (40 references is a lot for a six page ITICSE paper).

His argument that programming is easy has two parts:

  • First, children do it. As he says in the abstract, “But learning to program is easy — so easy that children can do it.”  That’s a false comparison — what children do in programming is not the same definition of “programming” that is in most of the literature that Andrew cites.  The evidence that programming is hard is coming mostly from higher-ed CS classes.  What is going on in introductory University CS classes and what children do is dramatically different.  We saw that in the WIPSCE 2014 Fields and Kafai paper, and those results were recently replicated in a recent ICER 2016 paper.  These are two different activities.
  • Second, what higher-education CS teachers expect at the end of the first course is too much.  He presents significant evidence that what CS teachers expect is achieved by students, but at the end of the second course.  The paper from Morrison, Decker, and Margulieux supports the argument that students think and work very differently and much more successfully by the end of the second CS course than in the first course.

I see Andrew’s argument as evidence that programming is hard.  The problem is that Andrew doesn’t define the target.  What level of ability counts as “programming”?  I believe that level of ability described by the McCracken Working Group, by the FCS1/SCS1 exams, and by most teachers as the outcomes from CS1 (these are all cited by Andrew’s paper) is the definition of the lowest level of “programming ability.”  That it takes two courses to reach that level of ability is what I would call hard.

I’ve been reading a terrific book, Proust and the Squid: The Story and Science of the Reading Brain by Maryanne Wolf.  It’s the story of how humans invented reading, how we teach reading, and how reading changes our brains (physically and in terms of cognitive function).  Oral language is easy.  We are literally wired for that.  Reading is hard.  We are not wired for that, and much of the invention of reading is about inventing how to teach reading.  Unless you can teach reading to a significant part of your population, you don’t develop a literate culture, and your written language doesn’t succeed.

Much of the invention of written language is about making it easier to learn and teach because learning to read is so hard.  Have you ever thought about why our Latin alphabet is ordered?  Why do we talk about the “ABC”‘s and sing a song about them?  We don’t actually need them to be ordered to read.  Ordering the alphabet makes it easier to memorize, and learning to read is a lot about memorization, about drill-and-practice to make the translation of symbols to sounds to words to concepts effortless (or at least, all System 1 in Kahneman terms).  This makes it easier, but the task of learning to read and write is still a cognitively complex task that takes a significant amount of time to master. It’s hard.

Programming is hard like written language is hard.  It’s not possible to program unless you know how to read.  Programming is particularly hard because the concepts that we’re mapping to are unfamiliar, are not part of our daily experience.  We only see it as easy because we have expert blind-spot.  We have already learned these concepts and made those mappings.  We have constructed understandings of iteration and conditional execution and variable storage.  It is difficult for experts to understand how hard it is to develop those concepts.  The evidence of children programming suggests that most children who program don’t have those concepts.

I remain unconvinced by Andrew’s argument, but I recommend the paper for a great summary of literature and an interesting read.


September 30, 2016 at 7:22 am 3 comments

Making learning effective, efficient, and engaging: An Interview With an Educational Realist and Grumpy Old Man, Paul Kirschner

I am a fan of Paul Kirschner‘s work.  This interview is great with useful insights about education — deep and pragmatic thinking.

I want to fundamentally understand how people can learn in effective, efficient, and enjoyable ways, and how you can teach and design learning materials to achieve this objective. If a learner doesn’t enjoy the learning experience, even if it’s effective and/or efficient, they won’t do it. The same is true for teaching: that is it must also be effective, efficient, and enjoyable for the teacher because if a teacher doesn’t enjoy the teaching process, even if it’s effective and/or efficient, they won’t do it.

Source: GUEST POST: An Interview With an Educational Realist and Grumpy Old Man — The Learning Scientists

September 28, 2016 at 7:59 am Leave a comment

Assessing Learning In Introductory Computer Science: Dagstuhl Seminar Report now Available

I have written about this Dagstuhl Seminar (see earlier post). The formal report is now available.

This seminar discussed educational outcomes for first-year (university-level) computer science. We explored which outcomes were widely shared across both countries and individual universities, best practices for assessing outcomes, and research projects that would significantly advance assessment of learning in computer science. We considered both technical and professional outcomes (some narrow and some broad) as well as how to create assessments that focused on individual learners. Several concrete research projects took shape during the seminar and are being pursued by some participants.

Source: DROPS – Assessing Learning In Introductory Computer Science (Dagstuhl Seminar 16072)

September 26, 2016 at 7:26 am Leave a comment

Should Computing be in its own College or School?

Probably my favorite session from the CRA Snowbird conference this last summer (see agenda with links to all talks) was a session on creating Colleges or Schools of Computer Science.  Should we?  Why?

The most compelling two talks in the session were from Randy Bryant and Rich LeBlanc, because they were so similar in structure.  They both argued that you don’t make the argument for a high-level College or School of Computing because you’re big and important.  You make it because you have a driving definition of computing that makes it unique.

  • Randy told the story of how CMU’s School of Computer Science was driven by the original definition of computer science from Newell and Simon, and how that definition was broader than most people’s definition of CS today. I recently blogged on that definition.
  • Rich told the story of how Georgia Tech’s College of Computing was driven by the ACM report The Future of Computing (led by Peter Denning) which showed how Computing crossed science, mathematics, and engineering.  Of course, Rich’s story was particularly powerful for me because I lived that definition — that was the vision that drove the College of Computing when I first got here in 1993.  Rich told the story of how that definition convinced faculty and administrators at Georgia Tech that Computing couldn’t be contained within the Colleges of Engineering or Science.  It needed to be its own entity. (I may also be biased because Rich quoted me from this blog🙂

Many of the people in the audience wanted to know, “How can I turn my Department into a School or College?”  One audience member said, “My CS department is the biggest one in the College of Engineering. How do I break apart into my own College.”  All the panelists told him, “You can’t.”  No Dean will allow its biggest department to leave — that would be crazy.  Some participants (from U. Michigan and U. Washington, in particular) pointed out why they don’t have a College or School of Computing — they have successful multi-department collaborations that make it unnecessary.  A new College or School is expensive.  Don’t do it unless you have to.


September 23, 2016 at 7:10 am 3 comments

Every University Student should Learn to Program: Guzdial Arguing for CS for All in Higher Education

A colleague recently approached me and said, “It would be useful if Universities got involved in this CS for All effort.  All Universities should offer courses aimed at everyone on campus. There should be a systematic effort to get everyone to take those classes.”

I agree, and have been making this argument for several years now.  I spent a few minutes gathering the papers, blog posts, and book where I’ve made that argument over the last decade and a bit.

In 2002, Elliot Soloway and I argued in CACM that we needed a new way to engage students in intro programming: Teaching the Nintendo Generation to Program.

In 2003, I published the first paper on Media Computation: A media computation course for non-majors.

In 2004, Andrea Forte led the team studying the Media Computation class at GT:Computers for communication, not calculation: Media as a motivation and context for learning and  A CS1 course designed to address interests of women.

In 2005, Andrea Forte and I presented empirical evidence about the courses we’d designed for specific audiences: Motivation and nonmajors in computer science: identifying discrete audiences for introductory courses. I published a paper in CACM about how the courses came to be at Georgia Tech: Teaching computing to everyone.

In 2008, I offered the historical argument for teaching everyone to program: Paving the Way for Computational Thinking.

We’ve published several papers about our design process: Imagineering inauthentic legitimate peripheral participation: an instructional design approach for motivating computing education and Design process for a non-majors computing course.

My 2013 ICER paper was a review of a decade’s worth of research on Media Computation: Exploring hypotheses about media computation

My keynote at VL/HCC 2015 was on how computing for all is a requirement for modern society: Requirements for a computing-literate society

My 2015 book is, to a great extent: an exploration of how to achieve CS for All: Learner-Centered Design of Computing Education: Research on Computing for Everyone.

In blog posts, it’s been a frequent topic of conversation:

I don’t know how to convince University CS departments to do just about anything, but here are my contributions to the dialogs that I hope are happening at Colleges and Universities worldwide about how to prepare students to engage in computational literacy.

September 19, 2016 at 7:15 am 15 comments

Older Posts Newer Posts

Recent Posts

October 2016
« Sep    


Blog Stats

  • 1,277,757 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 4,576 other followers

CS Teaching Tips