Posts tagged ‘CSLearning4U’

“What does Guzdial do anyway?”

I gave the last GVU Brown Bag seminar of the academic year.  Video is available at the link below.

Speaker: Mark Guzdial

Title:  What We Know About Teaching Computer Science (“What does Guzdial do, Anyway?”)


We have known for over 30 years that learning to program is surprisingly hard.  A series of international studies have shown remarkably little success in teaching programming. In my group, we have been developing approaches to improve learning about computing, by improving retention through relevance and by teaching in problem domain context.  Our classes and studies have utilized computer-supported collaborative learning, so we explore learning on-line as well as in-classroom. We have learned how anchored collaboration can lead to longer on-topic discussions, but how perceptions of course culture can dramatically inhibit discussion.  We have shown that well-designed on-line activities can lead to better learning at reduced cost (including time costs for the student and instructor). We are currently developing an ebook for learning computer science by high school teachers where we are trying to integrate these lessons for a new audience.

via GVU Brown Bag Seminar: Mark Guzdial | GVU Center at Georgia Tech.

May 15, 2013 at 1:33 am 6 comments

What is the current state of high school computer science professional development? The results of the UChicago Landscape Study

I am at the meeting in Portland of all the awardees from the NSF programs in Broadening Participation in Computing (BPC-A, like ECEP), Computing Education in the 21st Century (CE21, like our CSLearning4U project), and all the funded projects related to CS10K, sponsored by NCWIT.

You may recall that I invited people to participate in the Landscape Study on the capacity of our computing community’s professional development efforts.  The results of that survey are being presented here at this meeting, and a summary is available at the URL below.

I find the results a little depressing.  The folks at UChicago who do the study compare us to professional development in Science or Mathematics, and we don’t much look like that.  We have such a long way to go.

What is the current state of high school computer science professional development?


that are available for high school computer science (CS) teachers. The primary data collection for this strand took place through a survey administered to providers of high school computer science teacher professional development (PD).

via Computer Science – Landscape Study – The Center for Elementary Mathematics and Science Education.

January 14, 2013 at 8:00 am 2 comments

How students use an electronic book

Our technical report on evaluation of Luther College students’ use of the first generation Runestone Interactive Python ebook is finally available: This was a paper that we wrote for ICER 2012 but was rejected. The reviewers’ general argument is that we’re just describing one class using an ebook: No comparisons to other uses of books or ebooks, no particular hypothesis being tested. That is a fair criticism, but the problem is that we don’t know of a comparable study. We don’t know of anyone who has studied how CS students use their normal textbooks and IDE’s, for example, so that we can contrast it with the ebook use.

Here’s an example of one of our findings, which we found surprising. You might recall that the Runestone Interactive ebook has three special kinds of features: Embedded videos, “ActiveCode” segments (where students can actually program in Python from within the pages of the book), and “CodeLens” interactive visualizations of code, which can be run forward or backward. Below is a histogram of the number of ActiveCode events on each day of the first five weeks of class. The red bar is the day of the midterm. Blue bars are days when students had class (and most use on that day was in class), and gray bars are use out-of-class.

So here’s the first surprise: Use in-class swamps use out-of-class. The Luther college students are using the book in-class, and are working on programming activities (directed by the teacher) in-class. Those of us at Georgia Tech expected students to be programming far more out-of-class, maybe two or three times as much as in-class. Not so here. Is that unusual behavior only found at Luther? We don’t know yet.

Here’s the second surprise: Notice that the use on the day before the midterm is not one of the larger spikes. If you had an ebook to help you learn CS, with lots of examples that you could poke with, wouldn’t you study by using those? We are not seeing much of that, and students reported studying by “reading” it — few of them mentioned exploring code.

We at Georgia Tech built quizzes for each of the first five chapters of the book, completely separate from Brad Miller and David Ranum, the teachers at Luther. We also got the students’ midterm scores. So of those three features, which one most closely correlated with better performance outcomes on the quiz and midterms? The visualization tool. Here’s a scatterplot of the midterm score and use of the CodeLens by student.


Now, what did the students think was most valuable for their learning? Lectures, by quite a bit. We asked them to rank the various learning affordances in the class, and to score each in value from 1 to 5 (lower scores are better).


There are surprises here, too. Videos are not a big win here. Students do value being able to run code in the ebook (and told us about how much they liked that in our surveys), but don’t value it as much for their learning as lecture. Students don’t rank CodeLens very highly at all, but it was the feature that had the greatest measurable effect.

We have a whole bunch more data now: From several classes, and from a group of high school teachers that we can compare to the undergraduates. Christine Alvarado has ended her sabbatical at Georgia Tech, but is still working with us on this analysis. Brad Miller is still graciously allowing us to pester him with questions, and is giving us log data in identity-scrubbed form so that we can dig into it without compromising student identities. I hope that we can produce another paper for a peer-review forum, this time, with comparisons across multiple data sets so that we can start to figure out what is normal or typical use of a CS ebook.



November 8, 2012 at 7:34 am 12 comments

How do we teach students to use e-books differently than books?

Our paper on our ebook evaluation did not get accepted to ICER2012, but we’re going to turn it into a tech report and make it more generally available (rather than just a link here) soon (and then I’ll give it its own blog post).  But our bottomline isn’t too different from the one described below: Students don’t use ebooks differently than regular books, and that’s a problem.  How do they learn to use the affordances of the new medium?

I had dinner with Sebastian Thrun Wednesday night (which deserves get its own blog post soon!), and he suggested that the problem was calling them “books” at all — it suggests the wrong kinds of interaction, it connects to an incorrect model.  Maybe he’s right.  He suggested that “video games” was better, but I think that name has its own baggage with our in-service teachers. What could we call these new media, such that students would interact with them differently than traditional paper-based books?

The report is based on a survey conducted this spring of students and faculty at five universities where e-textbook projects were coordinated by Internet2, the high-speed networking group. Students praised the e-books for helping them save money but didn’t like reading on electronic devices. Many of them complained that the e-book platform was hard to navigate. In addition, most professors who responded said that they didn’t use the e-books’ collaborative features, which include the ability to share notes or create links within the text.

via Students Find E-Textbooks ‘Clumsy’ and Don’t Use Their Interactive Features – Wired Campus – The Chronicle of Higher Education.

August 24, 2012 at 9:49 am 15 comments

Instructional Design Principles Improve Learning about Computing: Making Measurable Progress

I have been eager to write this blog for months, but wanted to wait until both of the papers had been reviewed and accepted for publication.  Now “Subgoals Improve Performance in Computer Programming Construction Tasks” by Lauren Margulieux, Richard Catrambone, and Mark Guzdial has been accepted to the educational psychology conference EARLI SIG 6 & 7, and “Subgoal-Labeled Instructional Material Improves Performance and Transfer in Mobile Application Development” by the same authors have been accepted into ICER 2012.

Richard Catrambone has developed a subgoal model of learning.  The idea is to express instructions with explicit subgoals (“Here’s what you’re trying to achieve in the next three steps”) and that doing so helps students to develop a mental model of the process.  He has shown that using subgoals in instruction can help with learning and improve transfer in domains like statistics.  Will it work with CS?  That’s what his student Lauren set out to find out.

She took a video that Barb had created to help teachers learn how to build apps with App Inventor.  She then defined a set of subgoals that she felt captured the mental model of the process.  She then ran 40 undergraduates through a process of receiving subgoal-based instruction, or not:

In the first session, participants completed a demographic questionnaire, and then they had 40 minutes to study the first app‘s instructional material. Next, participants had 15 minutes to complete the first assessment task. In the second session, participants had 10 minutes to complete the second assessment task, which measured their retention. Then participants had 25 minutes to study the second app‘s instructional material followed by 25 minutes to complete the third assessment.

An example assessment task:

Write the steps you would take to make the screen change colors depending on the orientation of the phone; specifically, the screen turns blue when the pitch is greater than 2 (hint: you’ll need to make an orientation sensor and use blocks from “Screen 1” in My Blocks).

Here’s an example screenshot from one of Barb’s original videos, which is what the non-subgoal group would see:

This group would get text-based instruction that looked like this:

  1. Click on “My Blocks” to see the blocks for components you created.
  2. Click on “clap” and drag out a when clap.Touched block
  3. Click on “clapSound” and drag out call clapSound.Play and connect it after when clap.Touched

The subgoal group would get a video that looks like this:

That’s it — a callout would appear for a few second to remind them of what subgoal they were on.  Their text instructions looked a bit different:

Handle Events from My Blocks

  1. Click on “My Blocks” to see the blocks for components you created.
  2. Click on “clap” and drag out a when clap.Touched block

Set Output from My Blocks

  1. Click on “clapSound” and drag out call clapSound.Play and connect it after when clap.Touched

You’ll notice other educational psychology themes in here.  We give them instructional material with a complete worked example.  By calling out the mental model of the process explicitly, we reduce cognitive load associated with figuring out a mental model for themselves.  (When you tell students to develop something, but don’t tell them how, you are making it harder for them.)

Here’s a quote from one of the ICER 2012 reviewers (who recommended rejecting the paper):

“From Figure 1, it seems that the “treatment” is close to trivial: writing headings every few lines. This is like saying that if you divide up a program into sections with a comment preceding each section or each section implemented as a method, then it is easier to recall the structure.”

Yes. Exactly. That’s the point. But this “trivial” treatment really made a difference!

  • The subgoal group attempted and completed successfully more parts (subgoals) of the assessment tasks and faster — all three of those (more subgoals attempted, more completed successfully, and time) were all statistically significant.
  • The subgoal group completed successfully more tasks on a retention task (which wasn’t the exact same task — they had to transfer knowledge) one week later, again statistically significantly.

But did the students really learn the mental model communicated by the subgoal labels, or did the chunking things into subgoals just make it easier to read and parse?  Lauren ran a second experiment with 12 undergraduates, where she asked students to “talk-aloud” while they did the task.  The groups were too small with the second experiment to show the same learning benefits, but all the trends were in the same direction.  The subgoal group were still out-performing the non-subgoal groups, but what’s more they talked in subgoalsI find it amazing that she got these results from just one hour sessions.  In one hour, Lauren’s video taught undergraduate students how to get something done in App Inventor, and they could remember and do something new with that knowledge a week later — better than a comparable group of Georgia Tech undergraduates seeing the SAME videos (with only callout differences) doing the SAME tasks.  That is efficient learning.

Here’s a version of a challenge that I have made previously: Show me pedagogical techniques in computing education that have statistically significant impacts on performance, speed, and retention, and lead to developing a mental model of (even part of) a software development process.  What’s in our toolkit?  Where is our measurable progress? The CMU Cognitive Tutors count, but they were 20-30 years ago and (unfortunately) are not part of our CS education toolkit today. Alice and Scratch are tools — they are what to teach, not how to teach.  Most of our strong results (like Pair Programming, Caspersen’s STREAMS, and Media Computation) are about changing practice in whole courses, mostly for undergraduates, over several weeks.  Designing instruction around subgoals in order to communicate a mental model is a small, “trivial” tweak, that anyone can use no matter what they are teaching, with significant wins in terms of quality and efficiency.  Instructional design principles could be used to make undergraduate courses better, but they’re even more critical when teaching adults, when teaching working professionals, when teaching high school teachers who have very little time.  We need to re-think how we teach computing to cater to these new audiences.  Lauren is showing us how to do that.

One of the Ed Psych reviewers wrote, “Does not break new ground theoretically, but provides additional evidence for existing theory using new tasks.”  Yes. Exactly.  This is no new invention from an instructional design perspective.  It is simply mapping things that Richard has been doing for years into a computer science domain, into “new tasks.”  And it was successful.

Lauren is working with us this summer, and we will be trying it with high school teachers.  Will it work the same as with GT undergraduates?  I’m excited by these results — we’re already showing that the CSLearning4U approach of simply picking the low-hanging fruit from educational psychology can have a big impact on computing education quality and efficiency.

(NSF CE21 funds CSLearning4U.  Lauren’s work was supported by a Georgia Tech GVU/IPaT research grant. All the claims and opinions here are mine, not necessarily those of any of the funders.)

June 5, 2012 at 7:30 am 22 comments

Interactive eBook from Runestone Interactive: A Python eBook with IDE and visualization built-in

Brad Miller and David Ranum have opened up their eBook for general use at their new site.  This is the book whose use we have been studying for the last year as part of our CSLearning4U effort.  It’s a great alternative to the Udacity/Coursera model of distance education, to make a book more like a course, rather than capture the course in video.  Our paper on this analysis just got rejected, so I’m not sure when and where we can tell the story of what happened, but I’m hoping that we can talk about it soon*.

Its fun to see my sabbatical project getting loose the wild.  It is always a bit scary to work on something creative and new and then let other people play with it and respond to it.  Such is the case with the new eBook I worked on during my sabbatical.  Unlike other eBooks that you may be aware of,  this book — in the words of Emeril — “kicks it up a notch”.  Using some cool open source javascript code that I’ve had to modify and bend a bit for my own use this book allows the reader to try their hand at Python right in the book.  Examples are fully runnable in two different ways.  Each section has an accompanying video.  My co-author, David Ranum and I are using this book in class this Fall and its fun to see how the students interact with the book.  We’ve had none of the usual Fall frustration at getting Python installed on students machines.  You can have a look at the book here.

via Reputable Journal, How to Think Like a Computer Scientist – Interactive Edition.

*I try to be careful when talking about new, not-yet-published work here, because it annoys reviewers when they can easily discern the authorship of a “blind review” paper.  In CS Ed, the identity of *any* work can be easily determined within five minutes of Googling/Binging — there are just too few people in the field.  Still, reviewers downgrade our scores because I “broke faith” by talking about the work in my blog. Sigh.

On a more positive note, we got three papers accepted to ICER 2012, so I do plan to talk about that work here soon.

June 4, 2012 at 8:12 am 14 comments

Helping Everyone Create with Computing: Video of C5 Talk

A YouTube video of my talk (with Alan’s introduction) at C5 is now available.

February 15, 2012 at 10:33 am 2 comments

Older Posts Newer Posts

Recent Posts

July 2018
« Jun    


Blog Stats

  • 1,527,368 hits

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 5,291 other followers

CS Teaching Tips