Tools for Building Tutors, and Tutors for Computing Education

I took a workshop this morning on building intelligent tutoring systems.  That’s surprising if you knew me even 10 years ago, when I thought that intelligent tutoring systems were an interesting technology but a bad educational idea. I thought that tutors were the fancy worksheets that I thought deadened education and taught only the kinds of things that weren’t worth teaching.  Then I spent the last eight years trying to figure out how to teach computing to people who do want to learn about computing but don’t want to become professional software developers (i.e,, Media Computation).

• I’ve come to realize that there are students who need drill-and-practice kinds of activities to succeed, for whom discovery or inquiry learning is more effort than it’s worth. I recognize that in myself — I find economics fascinating and enjoy reading about it, but I’m not interested enough in economics to (for example) sit for hours with an economic simulator to figure out the principles for myself.
• I also now believe that even those students who do want to discover information for themselves still need a bunch of foundational knowledge on which to base their discoveries. A student who wants to figure out something about computing using Python, still has to learn enough Python to be able to use it as a tool. It’s not worth anybody’s time to learn Python syntax through trial-and-error discovery or inquiry learning.

I am now interested in tools like intelligent tutoring systems to help students learn foundational skills and concepts as efficiently as possible.

The workshop this morning was short, only three hours long. Still, we all built simple model-tracing tutors for a single mathematics problem, and I think most of us started building a tutor for something that we were interested in. I started building a tutor that would lead a student through writing the decreaseRed() function that we start with in both the Python and Java CS1 books.

The Cognitive Tutor Authoring Tools (CTAT) that the CMU folks have built are amazingly cool! They’ve built Java and Flash versions, but the Flash version is actually totally generic. Using a socket-based interface, the CTAT for Flash tool can observe behavior to construct a graph of potential student actions, which can labeled with hints, structure for success/failure paths, made ordered/unordered, and made generic with formulas. The tool can also be used for creating general rule-based tutors. CTAT really is a general tutoring engine that can be integrated into just about any kind of computational activity. I’m still wrapping my head about all the ways to use this tool.

My biggest “Aha!” (or maybe “Oh No!”) moment came from this table:

First, I’d never realized that 30 minutes of activity in the famous Geometry Tutor took two months to develop! The whole point of the CTAT effort is to reduce these costs. This table gave me new insight into what it’s going to take to meet President Obama’s goal of computational, individualized tutors. A typical semester course in college is about three contact hours and 10-15 hours of homework per week for 15 weeks. Let’s call it 13 hours of scripted learning activity a week, for a total 195 hours. The best ratio on that table is 48:1 — 48 hours of development for one hour of student activity. 9360 development hours (for those 195 hours at a 48:1 ratio), at 40 hours per week, is just over four person-years of effort to build a single college semester course. That’s not beyond reason, but it is certainly a sobering number. A full year high school course, at 45 minutes a week, five days a week, for 30 weeks is 112.5 student hours, which is (again using best case of 48:1) 5400 development hours. Two person-years of effort is a minimum to produce a single all-tutored high school course.

Here’s another great role for computer scientists: Build the tools to make these efforts more productive, and make the tools easier to use and easier to understand so that a wider range of people can engage in the effort.  CTAT is great, but still requires a hefty knowledge and time investment.  Can we make that easier and cheaper?

Talks and Trips: Learning Computing Concepts vs. Skills?

I’m writing from Chicago where I’m attending the International Conference of the Learning Sciences 2010. It’s pretty exciting for me to be back here. I helped co-chair the 1998 ICLS in Atlanta, but I haven’t been at this conference since 2002, when my focus shifted from general educational technology to specifically computing education. The theme this week is “Learning in the Disciplines.” I’m here at the invitation of Tom Moher to be part of a panel on Friday morning on computing education, with Yasmin Kafai, Ulrich Hoppe, and Sally Fincher. The questions for the panel are:

• What specific type of knowledge is characteristic of computer science? Is there a specific epistemology?
• Are there unique challenges or characteristics of learning in and teaching about computer science?
• What does learning about computing look like for different audiences: young children, high school, undergraduate, and beyond (e.g., professional scientists, or professionals from non-computing disciplines)? In the case of “non-computing professionals,” what do they learn, and how do they learn it (e.g.,what information ecologies do they draw upon, and how do they find useful information)?
• How do we support (broadly) learning about computer science?

In a couple weeks, I’m giving the keynote talk at the first AAI-10: The First Symposium on Educational Advances in Artificial Intelligence. I’m no AI person, but this conference has a strong computing education focus. I’m planning to use this as an opportunity to identifying challenges in computing education where I think AI researchers have a particularly strong lever for making things better. Not much travel for that one — I get to stay in Atlanta for a whole week!

In getting ready for my talk Friday, I’ve been trying to use themes from learning sciences to think about learning computing. For example, physics educators (BTW, Carl Weiman is here for the opening keynote tonight) have identified which physics concepts are particularly hard to understand. The challenge to learning those concepts is due in part to misconceptions that students have developed from years of trying to understand the physical world in their daily lives. I’ve realized that I don’t know about computing education research that’s looked at what’s hard about learning concepts in computing, rather than skills. We have lots of studies that have explored how students do (not?) learn how to program, such as in Mike McCracken’s, Ray Lister’s, and Allison Tew’s studies. But how about how well students learn concepts like:

• “All information in a computer is made up of bytes, so any single byte could be anything from the red channel of a pixel in a picture, to an instruction to the processor.” Or
• “All Internet traffic is made up of packets. So while it may seem like you have a continuous closed connection to your grandmother via Skype, you really don’t.”

Does anybody have any pointers to studies that have explored students learning conceptual (not skill-based) knowledge about computing?

I know that there is an argument says, “Computing is different from Physics because students have probably never seen low-level computer science before entering our classes, so they have few relevant preconceptions.” I believed that until I saw Mike Hewner’s data from his study of high school students in our Georgia Computes! mentoring program this last year. These are high school students who are being trained to be mentors in our younger student (e.g., middle school kids, Girl Scouts) workshops. They’re getting to see a lot of cool tools and learning a bunch about computer science. Mike found that they had persistent misconceptions about what computer science is, such as “Someone who is really great at Photoshop is a great computer scientist.” While that’s not a misconception about bytes or packets, that’s a misconception that influences what they think is relevant. The concept about bytes might seem relevant if students think that CS is all about great graphics design, but the packet concept interferes with their perception of Skype and doesn’t help with Photoshop — students might ignore or dismiss that, just as physics students say to themselves, “Yeah, in class and on exams, gravity pulls the projectile down, but I know that it’s really about air pressing down on the projectile.” So students’ misconceptions about what’s important about computing might be influencing what they pay attention to, even if they still know nothing about computer science.

Using technology to improve college completion rates

EduCause is heading up a new effort funded by the Gates Foundation to use technology improve college readiness and thus completion rates.  Below are their main bullets and a link to more information.  This links a couple of themes showing up in this blog lately: The importance of college completion rates, and how we in Computing should be in the forefront of figuring out how to use technology for learning.

• The high school graduation rate for all U.S. students is just over 70%. For African-Americans, Hispanics, and low-income students, the rate hovers at slightly over 50%.
• Of those who do graduate from high school, only half are prepared to succeed in college.
• For those who do enroll in postsecondary education, only about half will actually earn a degree or certification, with as few as one quarter of low-income students completing a degree.
• Today, it is virtually impossible to reach the middle class, and stay there, with only a high school diploma.
• Postsecondary education is increasingly critical to individual and family financial security, to a vibrant economy, and to an engaged and participatory society.

via Next Gen Learning Challenges | EDUCAUSE.