Preview for WiPSCE 2015: Usability and usage of interactive features in an online ebook for CS teachers

Next week in London, at WiPSCE 2015, the 10th Workshop in Primary and Secondary Computing Education, Barbara Ericson is going to present the pilot study of our teacher CSP ebook that Steven Moore (undergrad researcher here at Georgia Tech) ran last year.  This is the pilot that came before our Spring trial (see post here) and which led to our student and teacher ebooks that we recently released (see post here).  The authors of the paper are Barbara, Steven, Brianna Morrison, and me.

Steven’s study had two parts to it.  The first was a usability survey comparing three different ebook platforms: Runestone Interactive, Zyante, and CS Circles.  You may recall the post here where I invited you to participate in that survey.  Runestone did well in the survey, just beating out Zyante, and both were far ahead of CS Circles.

The meat of the paper is the study of 10 teachers who qualified for our study (got less than 40% on a pretest) and read the 8 chapters we had ready for them.  Every two chapters, there was a post-test on those two chapters.  Some of the findings:

• 50% of the study participants finished all 8 chapters.  That’s pretty good, but isn’t directly comparable to MOOC studies because we did offer them a $50 gift card for completing.
• As we expected and designed for, teachers read the books in chunks of time when they could fit it in.
• Those who used the book (e.g., did the Parson’s problems, ran the code, etc.), gained confidence in teaching CS and performed well on the post-tests. This is a big deal!  The teachers are not just writing code.  They are using a variety of different kinds of learning activities (see our ICER ebook paper) — and successfully learning and gaining confidence.  Writing code is not the only way to learn CS.  This has been one of the more controversial hypotheses. Many CS teachers believe that apprenticeship is the only way to learn CS, but we believe that we can successfully use a range of pedagogical practices.

Barbara did an extensive log file analysis of the participants, and we learned a lot from that. We learned where our books were not usable, e.g., when participants skipped over interactive features, or when they used the features wrong (e.g., clicking “Check Me” on a Parson’s problem, without ever moving pieces around). We used these findings in designing the current ebooks.

This paper is exciting for us — it’s the first where we test our design claims.