Student and Teacher CSP Ebooks are now Available

We now have TWO ebooks supporting CS Principles (see website here) now available — one for teachers and one for students.

Our teacher ebook summer study is now ended. (Announcement about launching the study is here.) We’re crunching the data now. We’ve already learned a lot about what teachers want in an ebook. We learned where our user interface wasn’t obvious, and where we needed to explain more. We learned that teachers expect end-of-chapter exercises. We have used what we have learned so far to produce the two new ebooks.

STUDENT CSP EBOOK: About a year ago, we received additional NSF funding (from the Improving Undergraduate STEM Education (IUSE) program) to develop a student version of our CSP ebook. We have been running participatory design studies and gathering usability surveys from students to get input on what a student ebook should look like. We have now released the first version of the student ebook.

The student CSP ebook is available at http://interactivepython.org/runestone/static/StudentCSP/index.html  It doesn’t require a login, but we recommend that teachers have their students login. Without a login, we store saved answers on the local computer, but if the student logs in, we save the answers by the student’s username.  The course name is StudentCSP.

We recommend that teachers create a custom version of the student ebook for your students.  This allows teachers to customize the ebook, assign homework, and view student’s progress, and even create additional assessments for students.

New Version TEACHER CSP EBOOK: We iterated on our teacher ebook at the same time that we were developing the student ebook. We hypothesize that the student CSP ebook may actually encourage teachers to complete the teacher ebook. We can imagine that teachers who use the student ebook might want to stay one step ahead of the students, e.g., “My students are starting Chapter 3 on Monday, so I better finish Chapter 3 this weekend.”

We have now created a second version of our teacher CSP ebook. This one is in lockstep with the student CSP ebook, includes all the end-of-chapter exercise answers and teacher notes (e.g., on how to teach particular concepts, common student difficulties, etc.). We are not making the second teacher ebook available openly (because it includes answers to the student problems).

Teachers, please contact us at cslearn4u@gmail.com with the name and location of your school, and we’ll send you the URL.

We recommend that teachers create their own course for their students.  See http://interactivepython.org/runestone/static/overview/instructor.html for why a teacher might want to build a custom course and how to do it.

• You must register on Runestone first at http://interactivepython.org/runestone/default/user/register. Enter StudentCSP as the course name. Be sure to record your username. We find that users often forget what they entered and assume it was their e-mail address — and it may not have been. You can also choose to sign in with your account on Google Plus, Facebook, Twitter, or several others.
• Then go to http://interactivepython.org/runestone/admin/index and select “Create your own Course”.
• Create a unique name for your course (use your school name and StudentCSP and year maybe), add a description, and your institution, and then select “CS Principles: Big Ideas in Programming by Mark Guzdial, Barbara Ericson, and Briana Morrison“.
• Leave the rest as defaults and click the “Submit” button.  This will build a custom version of the student ebook for your students and it will have a unique URL and course name.  You will be listed as the instructor and can look at the log files and view other information on the instructor page (you can get to this by clicking on the icon that looks like a head and shoulders and the top right of your screen when you are in the ebook).

ICER 2015 Preview: First CSLearning4U Ebook Paper

ICER 2015 (see website here) is August 9-13 in Omaha, Nebraska. The event starts for me and Barbara Ericson, Miranda Parker, and Briana Morrison on Saturday August 8. They’re all in the Doctoral Consortium, and I’m one of the co-chairs this year. (No, I’m not a discussant for any of my students.) The DC kickoff dinner is on Saturday, and the DC is on Sunday. My thanks to my co-chair Anthony Robins and to our discussants Tiffany Barnes, Steve Cooper, Beth Simon, Ben Shapiro, and Aman Yadav. A huge thanks to the SIGCSE Board who fund the DC each year.

We’ve got two papers in ICER this year, and I’ll preview each of them in separate blog posts. The papers are already available in the ACM digital library (see listing here), and I’ll put them on my Guzdial Papers page as soon as the Authorizer updates with them.

I’m very excited that the first CSLearning4U project paper is being presented by Barbara on Tuesday. (See our website here, the initial blog post when I announced the project here, and the announcement that the ebook is now available). Her paper, “Analysis of Interactive Features Designed to Enhance Learning in an Ebook,” presents the educational psychology principles on memory and learning that we’re building on, describes features of the ebooks that we’re building, and presents the first empirical description of how the Runestone ebooks that we’re studying (some that we built, some that others have built) are being used.

My favorite figure in the paper is this one:

This lists all the interactive practice elements of one chapter of a Runestone ebook along the horizontal axis (in the order in which they appear in the book left-to-right), and the number of users who used that element vertically. The drop-off from left-to-right is the classic non-completion rate that we see in MOOCs and other online education. Notice the light blue bars labelled “AC-E”? That’s editing code (in executable Active Code elements). Notice all the taller bars around those light blue bars? That’s everything else. What we see here is that fewer and fewer learners edit code, while we still see learners doing other kinds of learning practice, like Parsons Problems and multiple choice problems. Variety works to keep more users engaged for longer.

A big chunk of the paper is a detailed analysis of learners using Parsons Problems. Barbara did observational studies and log file analyses to gauge how difficult the Parsons problems were.  The teachers solved them in one or two tries, but they had more programming experience.  The undergraduate and high schools students had more difficulty — some took over 100 tries to solve a problem. Her analysis supports her argument that we need adaptive Parsons Problems, which is a challenge that she’s planning on tackling next.

Launching our Teacher Ebook for learning Python and CS Principles

Back in September 2011, I announced that we received NSF funding to try to “beat the book.” (See post here.) Could we create an electronic (Web-based) book that was better for CS teacher learning than reading a physical book? Took us three years, but I’m confident that the answer is now, “Yes.”

Our ebook is hosted by Brad Miller’s Runestone tools and site.  We use worked examples (as mentioned here) interleaved with practice, as Trafton and Reiser recommend.  We have coding in the book as well as Philip Guo’s visualizations.  There are audio tours to provide multi-modality code explanations (see modality effect), and Parson’s problems to provide low cognitive load practice (see mention here). We support book clubs that set their own schedule, in order to create social pressure to complete, but at a scale that makes sense for teachers.

2011 was a long time ago.  That original post didn’t even mention MOOCs.  We ran two studies in the Fall, one on learning with novices and one on usability (which involved several of you — thank you for responding to my call for participants!). I’m not going to say anything about those results here, pending review and publication. We have updated the book based on the results of those studies.  I don’t know if we beat the MOOC.  We’re running at about a 50% completion rate, but we’ll only really know when we go to scale.

I am pleased to announce the book is ready for release!

Please send this url to any teacher you think might want to learn about teaching CS (especially for the AP CS Principles — see learning objectives here) in Python: http://ebooks.cc.gatech.edu/TeachCSP-Python/  Thanks!

Our next steps are to develop a student ebook.  By Fall, we hope to have a teacher and a student CSP ebook, which may make for an additional incentive for teachers to complete.

SIGCSE 2015 Week! ECEP BOF and Ebooks and IRB and other CS Ed terms

This week is the SIGCSE 2015 Technical Symposium, the largest computing education conference in the US, perhaps in the world.  About 1300 people will be heading to Kansas City for four days of discussion, workshops, and talks.  See the conference page here and the program here.

Barbara Ericson and I will be presenting at several events:

• I’m speaking on a panel Thursday afternoon at 3:45 on human-subjects review of experiment protocols (by Institutional Review Boards (IRB)) and the challenges we’ve had in working in high schools and working on cross-institutional projects.
• Barb and I will be hosting with Rick Adrion a Birds of a Feather (BOF) session on state-level change at 6:30 Thursday.  This is part of our ECEP work.
• On Friday morning at 10 am, we’ll be showing our electronic book (ebook) for high school teachers interested in learning CS Principles.  The first public showing was at the NSF BPC Community meeting in January, but that was to a small audience.  We’ll be presenting at the NSF Showcase at 10 am on the exhibition floor.
• Barb is speaking on Friday afternoon in a panel at 3:30 on activities for K-12 CS outreach.
• On Friday night, Barb is running her famous “How to run a computing summer camp workshop.”

As usual, Georgia Tech is sending several of us (not just Barb and me).  One of my PhD students, Briana Morrison, is on a panel on Flipped Classrooms Thursday 1:45-3 pm in 3501G.  Another of my PhD students, Miranda Parker, is part of a BOF Preparing Undergraduates to Make the Most of Attending CS Conferences 6:30-7:20 on Thursday evening.  Our colleague, Betsy DiSalvo, is speaking Friday morning 10:45-12 on a panel Research, Resources and Communities: Informal Ed as a Partner in Computer Science Education in 2505A.

This is one of my shorter stays at a SIGCSE conference.  I’ll be coming in late Wednesday and leaving Friday afternoon.  I’ve been traveling way too much lately (NSF BPC Community meeting in January, talk at Penn in early February, Tapia conference in Boston two weeks ago, AP CS Principles review meeting in Chicago this last week).  I am fortunate to be teaching Media Computation this semester, and I hate to miss so many lectures.  More, it’s hard on our family when we’re both gone.  Barb will be at SIGCSE all week, from Tuesday night to Sunday morning, so be sure to stop by and say hello to her.

Help us figure out how to design ebooks to be usable

Like the post I made last week, we’ve been working on a bunch of experiment setups during the summer, and are now looking for participants.  This one is open to most readers of this blog. 

We have found that there is a lot of literature on how to design text to be readable on the screen.  But for interactive ebooks with embedded elements like coding areas, visualizations, and Parson’s problems, we know less about usability.  Steven Moore is an undergraduate researcher working with us, and he’s put together a collection of three different ebooks and a survey on preferences for each.  We’d love to get participants to try out his ebook samples and survey, please.

Hello,

We are a research group at Georgia Tech developing new approaches to teaching computer science at a distance.  In collaboration with researchers at Luther College, we have created a new kind of electronic book for learning Python. The book is entirely web-based and cross-platform, with special features, including programming within the book, program visualizations, videos, multiple-choice questions, and Parson’s problems (a special kind of programming problem).

We are currently seeking individuals with 6 months or more experience with programming in a textual language.  If you are willing to volunteer, you will need to complete a survey regarding the design and usability of three different interactive computer science e-books and specific components within those e-books.  Links to the e-books will be provided within the survey and the whole study can be completed via most web browsers.  The survey should take roughly forty-five minutes to complete.  We would like you to complete it by September 30^th, 2014.

The risks involved are no greater than those involved in daily activities.  You will receive a $15.00 gift card for completing the survey.  Study records will be kept confidential and your participation in this study is greatly valued.

If you are interested in participating, please contact Steven Moore at smoore46@gatech.edu.  If you have any questions, please feel free to contact me, Mark Guzdial at guzdial@cc.gatech.edu.

Seeking Participants for Modality Study: Please forward

This is part of Briana Morrison’s dissertation work.  She’s asking the question about the role of explaining programs in different modalities (e.g., visual vs. oral text) have on understanding.  If you know potential applicants (e.g., maybe advertise it to your whole class?), please forward this to them. We’d appreciate it!

Do you like to watch videos on the internet?

Want to help with a research study?

 

We need volunteers, age 18 and older, with no computer programming experience to help us determine the best way to explain code using videos.

 

No more than 2 hours of your time!

Completing a portion of the study allows you to enter a raffle for one of four

$50 Amazon Gift Cards

Completion of entire study allows you to enter a raffle for one

$100 Amazon Gift Card

 

Interested? Go to the following website:

 

https://www.surveymonkey.com/s/CLModality

 

Online education is dead; long live Mentored Simulated Experiences

Roger Schank (one of the founders of both cognitive science and learning science) declares MOOCs dead (including Georgia Tech’s OMS degree, explicitly), while recommending a shift to Mentored Simulation Experiences.  I find his description of MSE’s interesting — I think our ebook work is close to what he’s describing, since we focus on worked examples (as a kind of “mentoring”) and low cognitive-load practice (with lots of feedback).

So, while I am declaring online education dead, because every university is doing it and the market will soon be flooded with crap, I am not declaring the idea of a learning by doing mentored experience dead.

So, I  propose a new name, Mentored Simulated Experiences.

via Education Outrage: Online education and Online degrees are dead; now let’s move on to something real.

“What does Guzdial do anyway?”

I gave the last GVU Brown Bag seminar of the academic year.  Video is available at the link below.

Speaker: Mark Guzdial

Title:  What We Know About Teaching Computer Science (“What does Guzdial do, Anyway?”)

Abstract:

We have known for over 30 years that learning to program is surprisingly hard.  A series of international studies have shown remarkably little success in teaching programming. In my group, we have been developing approaches to improve learning about computing, by improving retention through relevance and by teaching in problem domain context.  Our classes and studies have utilized computer-supported collaborative learning, so we explore learning on-line as well as in-classroom. We have learned how anchored collaboration can lead to longer on-topic discussions, but how perceptions of course culture can dramatically inhibit discussion.  We have shown that well-designed on-line activities can lead to better learning at reduced cost (including time costs for the student and instructor). We are currently developing an ebook for learning computer science by high school teachers where we are trying to integrate these lessons for a new audience.

via GVU Brown Bag Seminar: Mark Guzdial | GVU Center at Georgia Tech.

What is the current state of high school computer science professional development? The results of the UChicago Landscape Study

I am at the meeting in Portland of all the awardees from the NSF programs in Broadening Participation in Computing (BPC-A, like ECEP), Computing Education in the 21st Century (CE21, like our CSLearning4U project), and all the funded projects related to CS10K, sponsored by NCWIT.

You may recall that I invited people to participate in the Landscape Study on the capacity of our computing community’s professional development efforts.  The results of that survey are being presented here at this meeting, and a summary is available at the URL below.

I find the results a little depressing.  The folks at UChicago who do the study compare us to professional development in Science or Mathematics, and we don’t much look like that.  We have such a long way to go.

What is the current state of high school computer science professional development?

THIS STRAND OF WORK FOCUSED ON DESCRIBING THE CURRENT PROFESSIONAL DEVELOPMENT OPPORTUNITIES

that are available for high school computer science (CS) teachers. The primary data collection for this strand took place through a survey administered to providers of high school computer science teacher professional development (PD).

via Computer Science – Landscape Study – The Center for Elementary Mathematics and Science Education.

How students use an electronic book

Our technical report on evaluation of Luther College students’ use of the first generation Runestone Interactive Python ebook is finally available: http://hdl.handle.net/1853/45044. This was a paper that we wrote for ICER 2012 but was rejected. The reviewers’ general argument is that we’re just describing one class using an ebook: No comparisons to other uses of books or ebooks, no particular hypothesis being tested. That is a fair criticism, but the problem is that we don’t know of a comparable study. We don’t know of anyone who has studied how CS students use their normal textbooks and IDE’s, for example, so that we can contrast it with the ebook use.

Here’s an example of one of our findings, which we found surprising. You might recall that the Runestone Interactive ebook has three special kinds of features: Embedded videos, “ActiveCode” segments (where students can actually program in Python from within the pages of the book), and “CodeLens” interactive visualizations of code, which can be run forward or backward. Below is a histogram of the number of ActiveCode events on each day of the first five weeks of class. The red bar is the day of the midterm. Blue bars are days when students had class (and most use on that day was in class), and gray bars are use out-of-class.

So here’s the first surprise: Use in-class swamps use out-of-class. The Luther college students are using the book in-class, and are working on programming activities (directed by the teacher) in-class. Those of us at Georgia Tech expected students to be programming far more out-of-class, maybe two or three times as much as in-class. Not so here. Is that unusual behavior only found at Luther? We don’t know yet.

Here’s the second surprise: Notice that the use on the day before the midterm is not one of the larger spikes. If you had an ebook to help you learn CS, with lots of examples that you could poke with, wouldn’t you study by using those? We are not seeing much of that, and students reported studying by “reading” it — few of them mentioned exploring code.

We at Georgia Tech built quizzes for each of the first five chapters of the book, completely separate from Brad Miller and David Ranum, the teachers at Luther. We also got the students’ midterm scores. So of those three features, which one most closely correlated with better performance outcomes on the quiz and midterms? The visualization tool. Here’s a scatterplot of the midterm score and use of the CodeLens by student.

 

Now, what did the students think was most valuable for their learning? Lectures, by quite a bit. We asked them to rank the various learning affordances in the class, and to score each in value from 1 to 5 (lower scores are better).

 

There are surprises here, too. Videos are not a big win here. Students do value being able to run code in the ebook (and told us about how much they liked that in our surveys), but don’t value it as much for their learning as lecture. Students don’t rank CodeLens very highly at all, but it was the feature that had the greatest measurable effect.

We have a whole bunch more data now: From several classes, and from a group of high school teachers that we can compare to the undergraduates. Christine Alvarado has ended her sabbatical at Georgia Tech, but is still working with us on this analysis. Brad Miller is still graciously allowing us to pester him with questions, and is giving us log data in identity-scrubbed form so that we can dig into it without compromising student identities. I hope that we can produce another paper for a peer-review forum, this time, with comparisons across multiple data sets so that we can start to figure out what is normal or typical use of a CS ebook.

 

 

How do we teach students to use e-books differently than books?

Our paper on our ebook evaluation did not get accepted to ICER2012, but we’re going to turn it into a tech report and make it more generally available (rather than just a link here) soon (and then I’ll give it its own blog post).  But our bottomline isn’t too different from the one described below: Students don’t use ebooks differently than regular books, and that’s a problem.  How do they learn to use the affordances of the new medium?

I had dinner with Sebastian Thrun Wednesday night (which deserves get its own blog post soon!), and he suggested that the problem was calling them “books” at all — it suggests the wrong kinds of interaction, it connects to an incorrect model.  Maybe he’s right.  He suggested that “video games” was better, but I think that name has its own baggage with our in-service teachers. What could we call these new media, such that students would interact with them differently than traditional paper-based books?

The report is based on a survey conducted this spring of students and faculty at five universities where e-textbook projects were coordinated by Internet2, the high-speed networking group. Students praised the e-books for helping them save money but didn’t like reading on electronic devices. Many of them complained that the e-book platform was hard to navigate. In addition, most professors who responded said that they didn’t use the e-books’ collaborative features, which include the ability to share notes or create links within the text.

via Students Find E-Textbooks ‘Clumsy’ and Don’t Use Their Interactive Features – Wired Campus – The Chronicle of Higher Education.

Instructional Design Principles Improve Learning about Computing: Making Measurable Progress

I have been eager to write this blog for months, but wanted to wait until both of the papers had been reviewed and accepted for publication.  Now “Subgoals Improve Performance in Computer Programming Construction Tasks” by Lauren Margulieux, Richard Catrambone, and Mark Guzdial has been accepted to the educational psychology conference EARLI SIG 6 & 7, and “Subgoal-Labeled Instructional Material Improves Performance and Transfer in Mobile Application Development” by the same authors have been accepted into ICER 2012.

Richard Catrambone has developed a subgoal model of learning.  The idea is to express instructions with explicit subgoals (“Here’s what you’re trying to achieve in the next three steps”) and that doing so helps students to develop a mental model of the process.  He has shown that using subgoals in instruction can help with learning and improve transfer in domains like statistics.  Will it work with CS?  That’s what his student Lauren set out to find out.

She took a video that Barb had created to help teachers learn how to build apps with App Inventor.  She then defined a set of subgoals that she felt captured the mental model of the process.  She then ran 40 undergraduates through a process of receiving subgoal-based instruction, or not:

In the first session, participants completed a demographic questionnaire, and then they had 40 minutes to study the first app‘s instructional material. Next, participants had 15 minutes to complete the first assessment task. In the second session, participants had 10 minutes to complete the second assessment task, which measured their retention. Then participants had 25 minutes to study the second app‘s instructional material followed by 25 minutes to complete the third assessment.

An example assessment task:

Write the steps you would take to make the screen change colors depending on the orientation of the phone; specifically, the screen turns blue when the pitch is greater than 2 (hint: you’ll need to make an orientation sensor and use blocks from “Screen 1” in My Blocks).

Here’s an example screenshot from one of Barb’s original videos, which is what the non-subgoal group would see:

This group would get text-based instruction that looked like this:

1. Click on “My Blocks” to see the blocks for components you created.
2. Click on “clap” and drag out a when clap.Touched block
3. Click on “clapSound” and drag out call clapSound.Play and connect it after when clap.Touched

The subgoal group would get a video that looks like this:

That’s it — a callout would appear for a few second to remind them of what subgoal they were on.  Their text instructions looked a bit different:

Handle Events from My Blocks

1. Click on “My Blocks” to see the blocks for components you created.
2. Click on “clap” and drag out a when clap.Touched block

Set Output from My Blocks

1. Click on “clapSound” and drag out call clapSound.Play and connect it after when clap.Touched

You’ll notice other educational psychology themes in here.  We give them instructional material with a complete worked example.  By calling out the mental model of the process explicitly, we reduce cognitive load associated with figuring out a mental model for themselves.  (When you tell students to develop something, but don’t tell them how, you are making it harder for them.)

Here’s a quote from one of the ICER 2012 reviewers (who recommended rejecting the paper):

“From Figure 1, it seems that the “treatment” is close to trivial: writing headings every few lines. This is like saying that if you divide up a program into sections with a comment preceding each section or each section implemented as a method, then it is easier to recall the structure.”

Yes. Exactly. That’s the point. But this “trivial” treatment really made a difference!

• The subgoal group attempted and completed successfully more parts (subgoals) of the assessment tasks and faster — all three of those (more subgoals attempted, more completed successfully, and time) were all statistically significant.
• The subgoal group completed successfully more tasks on a retention task (which wasn’t the exact same task — they had to transfer knowledge) one week later, again statistically significantly.

But did the students really learn the mental model communicated by the subgoal labels, or did the chunking things into subgoals just make it easier to read and parse?  Lauren ran a second experiment with 12 undergraduates, where she asked students to “talk-aloud” while they did the task.  The groups were too small with the second experiment to show the same learning benefits, but all the trends were in the same direction.  The subgoal group were still out-performing the non-subgoal groups, but what’s more they talked in subgoals! I find it amazing that she got these results from just one hour sessions.  In one hour, Lauren’s video taught undergraduate students how to get something done in App Inventor, and they could remember and do something new with that knowledge a week later — better than a comparable group of Georgia Tech undergraduates seeing the SAME videos (with only callout differences) doing the SAME tasks.  That is efficient learning.

Here’s a version of a challenge that I have made previously: Show me pedagogical techniques in computing education that have statistically significant impacts on performance, speed, and retention, and lead to developing a mental model of (even part of) a software development process.  What’s in our toolkit?  Where is our measurable progress? The CMU Cognitive Tutors count, but they were 20-30 years ago and (unfortunately) are not part of our CS education toolkit today. Alice and Scratch are tools — they are what to teach, not how to teach.  Most of our strong results (like Pair Programming, Caspersen’s STREAMS, and Media Computation) are about changing practice in whole courses, mostly for undergraduates, over several weeks.  Designing instruction around subgoals in order to communicate a mental model is a small, “trivial” tweak, that anyone can use no matter what they are teaching, with significant wins in terms of quality and efficiency.  Instructional design principles could be used to make undergraduate courses better, but they’re even more critical when teaching adults, when teaching working professionals, when teaching high school teachers who have very little time.  We need to re-think how we teach computing to cater to these new audiences.  Lauren is showing us how to do that.

One of the Ed Psych reviewers wrote, “Does not break new ground theoretically, but provides additional evidence for existing theory using new tasks.”  Yes. Exactly.  This is no new invention from an instructional design perspective.  It is simply mapping things that Richard has been doing for years into a computer science domain, into “new tasks.”  And it was successful.

Lauren is working with us this summer, and we will be trying it with high school teachers.  Will it work the same as with GT undergraduates?  I’m excited by these results — we’re already showing that the CSLearning4U approach of simply picking the low-hanging fruit from educational psychology can have a big impact on computing education quality and efficiency.

(NSF CE21 funds CSLearning4U.  Lauren’s work was supported by a Georgia Tech GVU/IPaT research grant. All the claims and opinions here are mine, not necessarily those of any of the funders.)

Interactive eBook from Runestone Interactive: A Python eBook with IDE and visualization built-in

Brad Miller and David Ranum have opened up their eBook for general use at their new http://interactivepython.org site.  This is the book whose use we have been studying for the last year as part of our CSLearning4U effort.  It’s a great alternative to the Udacity/Coursera model of distance education, to make a book more like a course, rather than capture the course in video.  Our paper on this analysis just got rejected, so I’m not sure when and where we can tell the story of what happened, but I’m hoping that we can talk about it soon*.

Its fun to see my sabbatical project getting loose the wild.  It is always a bit scary to work on something creative and new and then let other people play with it and respond to it.  Such is the case with the new eBook I worked on during my sabbatical.  Unlike other eBooks that you may be aware of,  this book — in the words of Emeril — “kicks it up a notch”.  Using some cool open source javascript code that I’ve had to modify and bend a bit for my own use this book allows the reader to try their hand at Python right in the book.  Examples are fully runnable in two different ways.  Each section has an accompanying video.  My co-author, David Ranum and I are using this book in class this Fall and its fun to see how the students interact with the book.  We’ve had none of the usual Fall frustration at getting Python installed on students machines.  You can have a look at the book here.

via Reputable Journal, How to Think Like a Computer Scientist – Interactive Edition.

*I try to be careful when talking about new, not-yet-published work here, because it annoys reviewers when they can easily discern the authorship of a “blind review” paper.  In CS Ed, the identity of *any* work can be easily determined within five minutes of Googling/Binging — there are just too few people in the field.  Still, reviewers downgrade our scores because I “broke faith” by talking about the work in my blog. Sigh.

On a more positive note, we got three papers accepted to ICER 2012, so I do plan to talk about that work here soon.

Helping Everyone Create with Computing: Video of C5 Talk

A YouTube video of my talk (with Alan’s introduction) at C5 is now available.

Using a Worksheet to Address a CS Education Problem

I just gave a midterm exam in my class on data structures using media computation.  The results are disappointing, and have prodded me into trying something new: I made a worksheet.

Two of the problems on the midterm were tracing problems.  One had a posterize method (reduces the possible number of colors in a picture), and I provided some RGB values for some sample pixels: What new RGB values will be in those pixels after this method executes?  The second problem had three graphics images, and three programs: Which program generated which image?  I had several people review the tracing problems — they weren’t trivial.

One of the problems was a code writing problem.  I gave them the Sound methods for reverse() and for increaseVolume().  They were to write a new method that reversed the first half the sound, then increased the volume in the second half the sound.  I literally expected them to copy the bodies of the two loops, and just tweak the ranges on the loops.

The average score on the two tracing problems (adding the percentages together and dividing by 200) was 96%.  15 of the 16 students in the class got over 90% on the two problems.  The average score on the one writing problem was 75%.  Five of the 16 students got less than 70% right.  These students are quite good at reading code.  Some of these students were unsuccessful writing code.

What happened?  The research literature on CS education has lots of examples of how reading and writing skills develop separately.  Pete Pirolli reported in the early 1980’s that, in several studies, Anderson’s lab at CMU found no correlation between reading and writing skills.  Raymond Lister and his colleagues have written several papers about students having one or the other, but not both sets of skills.  So, it’s not a surprising result, but it’s one that I need to address.

How?  The students are working hard on programming assignments.  Programming assignments are so heavy — they take so much time, they require so much effort.  I do require students to do a weekly out-of-class quiz, often oriented around videos.  This semester, two of the quiz assignments were to use Problets on for and while loops.  That may have helped with the reading skills, but doesn’t seemed to have helped much with writing skills.

I decided that I needed to try out one of the ideas in CSLearning4U, that is more examples and more lighterweight practice.  I made a worksheet for this week’s quiz. The worksheet has four completely worked out array manipulation programs (with outputs provided) and two more sound manipulation programs (for analogous actions as on the arrays), to make clear the connections.  There are five more programs that are nearly complete, including all outputs, but with some blanks. The blanks are mostly where range manipulations would take place.  The students have to complete the blanks.  I’m offering two points each for the 10 blanks, where the first 10 points go towards the quiz grade and the second possible 10 points are bonus points toward the midterm exam.

I made very clear that there was no partial credit.  The students get them right, or they don’t.  I explicitly said that the smartest thing to do is to type in the program, then fill in the blanks such that the output is right.  This is a lighterweight activity than programming from scratch.

This is unusual for me, and maybe for other higher-education CS teachers.  Worksheets are very common in high schools, and even in higher-education in other STEM disciplines.  I’ve seen physics labs and chemistry pre-labs in worksheet formats.  I can’t remember seeing them in CS, but I’m sure that somebody has done them.  What I’m doing here is something that K-12 teachers are taught to do: See a particular learning problem, then find or invent an activity to address that problem.  This is the first time I’ve created a worksheet as that activity.  In CS, we tend to assign some more programming to address a learning problem.  I’m looking for something with less cognitive load: More example, and more lightweight practice.

I’m still figuring out how I’ll evaluate if this worked.  My guess is that I’ll have a slightly more heavyweight version of this on next week’s quiz, e.g., I provide a partial program, with inputs and outputs specified, and I’ll ask them to fill in the rest of the program.  That will give me a sense if they’re developing the writing skills I hope to see.