CS Majors are least likely to finish degree among Science majors

“Of the STEM fields, computer or information sciences had the lowest rate at 46.4 percent.”  Maybe lots of CS faculty agree with the commentor to the last blog post, that it’s better to flunk out CS students than let them graduate and suffer the pains of being a computing professional.

All significant education questions are economic

Seymour Papert once said, “All the interesting questions in education are social questions.”  I heard this when I was visiting his company, Logo Computer Systems Incorporated (LCSI) in Montreal in the early 90’s.  His point was that what was really interesting was now whether or not Logo “worked,” but why was it being taught or not taught, what did society value and how did that get reflected in schools, how were teachers trained and then valued by their society, and so on.  He found that the education questions that most interested him were actually about the societies in which the education occurred, and what that told him about the choices of that society.

I’ve been thinking about Seymour’s comment while thinking about my last blog post, about the research questions surrounding Media Computation.  This leads me to Guzdial’s Claim: All significant education questions are economic. (By naming it, we make it easier to refer to it and refute it.)  Those education questions that lead to wondering about society are interesting, but they don’t lead to change.  A significant question, as I’m defining it here, leads to change.  Change in education is all an issue of economics.

We know how to get education right.  If we have a good tutor for every child who wants to learn, learning will occur.  Bruner tells us that anything can be learned in an honest way. Thus, one tutor per child can lead to any kind of learning. However, we can’t afford that.  Everything else we try is an economic compromise over what we know will work.

Economics is not about money.   A nice definition of economics (cited in the Wikipedia article) is “the science which studies human behaviour as a relationship between ends and scarce means which have alternative uses.”  Human time and attention is a scarce resource that could be used for alternative pursuits.  “What do students choose to study?” is an economic question.  “Why do teachers teach that?” is an economic question.  They’re making choices.  If you want to change something significant in education, you have to know why people are making the economic choices they’re making, and figure out a way to make your alternative more attractive.

The questions that Media Computation was designed to answer were economic ones.  Losing 30-50% of the students in a CS1 is a huge waste of resources.  That’s what we tried to address.  It’s interesting is how the contextualized approach impacted future student choice.  Did we convince students that computing is interesting?  I don’t think so.  Instead, we convinced students that computing was relevant to the things that they already had value for, that it had utility for them.  Thus, we had students who were willing to take “More Media Computation” classes (as long as we kept explaining the utility), but not “More Computer Science” classes.

What I find myself struggling with here is how to describe design questions in education.  Is it significant or interesting to design a better way to teach something? If your better way is trying to address economic issues, I am claiming that it is significant.  Is it important, interesting, or worthwhile to figure out how to teach something simply better?  And what does that mean to do better?  Can you do better than the tutor with the good student?  We don’t currently have good measures of what “better learning” in computing education means.  When I look at my favorite textbooks (Structure and Interpretation of Computer Programs and Computer Science Logo Style), I have a hard time saying that they were “significant” — few teachers ever used them, few students know about them today, and the languages they use have fallen out of favor.  (We CS teachers are so driven by fads.)  SICP has always been expensive to implement because it often increased WDF rates. While SICP was used to teach many of the leaders in computer science today (since it was used at MIT, Berkeley, and Brown), it’s hard to measure the impact of the approach.  Was SICP significant for those leaders?

I have certainly seen design questions answered in insignificant ways.  I get frustrated with people who show me their “great new way of teaching X” that “my students and Girl Scout groups really love,” and when I ask them for their evidence, they have none.  “I talked to the students” or “I could tell”  or “Look at the cool thing they built” is the best that I get. Teachers are lousy probes.  A teacher is not good at telling how students are learning or engaging, just by talking with them.  Measurement should be uniform across students to get any kind of sense of what of significance is happening.

As you can tell, I’m struggling these days with what makes for worthwhile education research questions.  What is the value of education design, development, and research?  Is it possible to have long-term findings?  Economic conditions (e.g., what students and teachers value) change regularly.  Is it possible to come up with a curriculum that could be used successfully for more than 20 years?  Or is curriculum always churning, to keep up with changing values and choices? What would be a significant education research question whose answer would be useful 100 years from now?  Do we have any that we can point to?

Within computing education research, doing something of worthwhile significance becomes even more dubious.  If you doubt that all significant computing education questions are eocnomic, come up with a non-economic explanation for changing our CS1 language every 5-10 years!  It’s certainly not because we have evidence that students learn better with the new language.  Computing education research questions that have any significance, that can lead to change, are economic ones.

Media Computation is not “done”

I heard from a colleague last week that Media Computation is “done.”  “There are no more research questions to answer.  No one will be able to publish on it anymore.”  I’m at a Media Computation workshop in Kansas City this week where I can assure you, nobody thinks all the questions about Media Computation are answered.

What we do know is that adoption of Media Computation (at multiple schools, with both genders and a variety of ethnic groups, including under-represented minorities) has improved retention in the first course compared to the more common approaches to CS1.  That was our explicit goal in designing Media Computation, to address the 30-50% WDF rate.  While that’s a pretty often-repeated result now, I still wonder what that means.  Is Media Computation “teacher proof”?  Could a really lousy teacher do a really lousy implementation that does not improve retention?  Well, of course!  So, what then does that finding mean?  It’s a true statement empirically, and it’s telling us something about what engages and motivates students.  I’m not sure what it tells us about computing education and its practice yet.

What are the other open questions?

• Does Media Computation lead to the same learning? I don’t know.  Nor does anyone else have evidence of their CS1 approach leading to the “same learning.”  We have no way to measure “same learning” across languages and approaches. We are behind other domain-specific education research fields.
• Does Media Computation lead to more interest in computing as a major or minor? The evidence that I’ve seen suggests “no,” and further, there’s not much that we can do with any new approach at the undergraduate level to influence our enrollment numbers.  Sarita Yardi and Amy Bruckman’s ICER 2007 paper suggests to me that decisions about whether computing is a promising career are made by students in their pre-teens.  That meshes with what we’re seeing in our Girl Scout and high school studies.
• Does Media Computation lead to more interest in going on to more computer science classes? I don’t know.  We did have enough interest in our CS1 to build a CS2.  Bryn Mawr is finding that their IPRE approach in CS1 is leading to their largest CS2’s ever, which is using a context of robots and media.  I’m not sure how one would go about answering this question in a way that generalizes across schools — it’s an interesting question.
• What’s the real impact of Media Computation or any contextualized computing approach past the first semester? That’s a hard, interesting, and open question.  Mike Hewner answered this question in one way in his ICER 2008 paper where he studied Georgia Tech seniors and asked them (indirectly) about the impact that their contextualized first course had on the way that they used computing over the rest of their academic career.  Bottomline: Not much.  If you use computing in your other classes, than that first course was useful and spurred on your interest.  But if your major doesn’t use much computing, then that first course may have been enjoyable, but it was one course out of 40 that you took in your undergraduate years — why should that have made much difference?  On the other hand, Lana Yarosh’s paper suggests context is still useful in CS2, and we had a paper in JERIC last year showing the benefit of context in a computer organization class.  Context may still be useful, but maybe different contexts.

What is a contextualized computing education approach, anyway?  Is it just problem-based learning applied to computing? Project-based learning?  Is it “teaching with a theme” as Bill Gates says has been so successful with the high schools funded by the Gates Foundation?  What kind of contexts/themes lead to improved engagement, and which don’t?  Do contexts have to change with generations of kids?  Is that why our generation did fine without contexts (unless computing the next Fibonacci number is a “context”) but we’re seeing an impact of context on this generation?

No, I don’t think MediaComp is “done” yet.  It’s actually a bit worrying that teachers or researchers would think that it is.  That’s all the evidence you need?  You’ll accept something as being “known” with what we have so-far?  As computing education researchers, we should have a higher bar.  It’s one thing to say “It’s no longer interesting” or “It’s pretty clear what direction the results are pointing now.”  It’s another to say that “We know all that we need to know.”

The meaning of computer science: What our students are thinking about

If you haven’t seen these blog posts yet by “The Wicked Teacher of the West,” I recommend them.  The author is middle school computer science teacher whom I think is terrific–smart, cares about teaching, trying hard to learn something new.  The challenges she had in her workshop are probably like those of our best students when they are struggling in our computing classes.
– http://geek-knitter.blogspot.com/2009/07/lost-in-syntax-part-1-or-omg-im-going.html
– http://geek-knitter.blogspot.com/2009/07/lost-in-syntax-part-2-or-omg-im-going.html

I also know a bit about the workshop that she was in, and I also hold it in high regard.  It’s got great ideas in it, and there are definitely things in that workshop that Wicked Teacher might use in her classes.  Here’s the rub as I see it: those two things weren’t connected for her.  As she put it, “I felt like I didn’t ‘get it’ but other than saying that I felt like I had no context, I couldn’t articulate what I meant.”

Today, I’m currently working on the lists of “Big Ideas of Computer Science” for the new APCS “Computer Science: Principles” Commission (and connecting these to Peter Denning’s Great Principles of Computing), and for the last few days, I’ve been working on the lecture slides for the Second Edition of our Python Media Computation book.  The combination of these blogs and these activities have me wondering, “How do we explain the big picture to students, especially when we don’t agree on the big picture?”

Whenever I hear someone saying matter-of-factly, “Computer Science is really just engineering,” I know that they haven’t really thought about what computer science is, at least in terms that students are looking for.  I think the workshop leaders had a reason for telling the Wicked Teacher why they were asking her to do all that she did.  But not only didn’t they tell her, I don’t think it was the same one that  she was looking for.

The workshop leader saw the meaning of the code as being the correct execution.  What Wicked Teacher was looking for was why should she care.  The state education officer who critiqued me for not thinking hard enough about the match of standards to computing principles was right.  If I want teachers to care about computing, I need to show them why it’s important in terms that they consider important.

This connects to my Python slides and Great Principles. In one example, in the book, we show how to decreaseRed in a picture:

def decreaseRed(picture):
  for pixel in getPixels(picture):
    value = getRed(pixel)
    setRed(pixel, value * 0.5)

Then we parameterize that function further:

def changeRed(picture, factor):
  for pixel in getPixels(picture):
    value = getRed(pixel)
    setRed(pixel, value * factor)

Why do that? Why should we add the additional parameter to the function? I completely believe that this is an important part of introductory computing that we should teach. But what’s the story that we give students to give this meaning?

In our Python book, we give the traditional, engineering-based explanation: By adding the additional parameter, we make the code more reusable so we can later build even more complex things more easily.  But what if someone doesn’t care about building more complex things, or building anything later?  What if they care about other things that are just as valuable?

I realized that I really could tell a story about “bindings” here, about associating names with values.  Peter Denning’s Great Principles touch on some of these.  By delaying the association of a value for factor, I maintain flexibility and expressiveness.  That’s a path that leads me to thinking about a wonderful set of abstractions that are powerful and unique to computer science: scopes and namespaces, functions as first class data objects, lambda (as the value for a function binding), creating new kinds of control structures, aggregating data and procedures together to create objects, and the power of “messages” where the function to be invoked for a given “name” is decided later, by the receiver.  That might be engineering, but it’s closer to mathematics to me.

What many of my most serious students really care about is exploring the effects of this new function.  What visual effects can you get by manipulating red?  When do you want to?  What is the power and  limitation of a red/green/blue color model?  What can I say easily, and what is more complex?  They care about the power of representations, and choosing a particular model, and about empirical data resulting from experiments with these program.  Now, computer science looks like science.  That’s yet another, equally valid meaning for adding the factor parameter.

The Wicked Teacher was looking for a meaning in what she was learning, and I suspect that it’s a different meaning than what the workshop leaders were offering.  There are several efforts, like Peter Denning’s and those of the APCS Commission, to define what “Computing” means. The real challenge that we face (in these efforts, and as teachers) is to offer a variety of meanings. We want to encourage deep thought and engagement in the power of computing.

Media Computation update

Barb and I turned over the final edits to production on our data structures textbook last night.  (YAY!)  It’s promised in December.

The second edition of the Python book has just now hit the streets. This version is in full color, so it works more like the Java book and less like a grayscale book hinting at multimedia.

I finally got around to fixing up http://www.mediacomputation.org Please do visit and let me know if you can find what you’re looking for there.

I’m off to Kansas City for a Media Computation workshop July 29-31, then an APCS Advisory Board meeting August 1-2, and then five days of teacher workshops (on Media Computation and IPRE) here in Atlanta.  It’s going to be an intense 10 days, so I probably won’t be here much in that time.

Trust me! I’m a Computer Science Teacher!

U. Washington has a terrific Computer Science & Engineering Colloquium series available on iTunes.  I just finished a talk by Alan Borning on his UrbanSim project, which is (as he describes it) “Professional SimCity” which supports people making urban planning decisions.  He talks about the challenge of trust in that setting.  These are high-stakes decisions with complicated issues, trading off pollution with equitable land use with water needs with land values, and so on.  The project has moved from Java to Python, so that the models can be more easily inspectable and changed — he said modelers just didn’t like Java and hated declaring types in their models.  He said that their tools used to have a button to re-run the unit tests on the software, so that users could be assured that the basic software underlying the models was working as expected.  Nobody ever used it, so they removed the button.  People trusted the software, even though changes/mistakes there could have had much bigger impact on the end result than the models themselves.

Mike Hewner’s comment on my last blog post and Alan Borning’s talk have had me thinking about when students trust us, and when they don’t.  It’s not obvious to me when students trust us and when they don’t. I’ve two stories to share about trusting (or not) the computer science teacher.

Story 1: We trust you. Bill Leahy teaches CS2261 Media Device Architectures here at Georgia Tech.  This is our class on computer organization for students in our Computational Media degree program, which students find much more motivating than our traditional computer organization class.  Instead of working on a simulator of a pretend processor, CS2261 has students program a Nintendo Gameboy.  Same low-level focus, and even more complicated (since it’s a real computer), but more motivating, so it’s a real win.  Students tell us that they like programming for the Gameboy. Yet, they never do.

Bill shows them on the first day of class how he can write code for the Gameboy, compile it, download it to flash memory, and then boot it on the Gameboy.  Lo-and-behold, his game runs on the Gameboy.  To the best of Bill’s knowledge, no student has ever repeated that process!  None of them ask him how to get the flash card set-up, and nobody ever shows him their program running on their own Gameboy.  Amazing as it seems, it seems to be enough for the students to know that their code could run on the Gameboy.  During the course, they only use an emulator.  So, in a real sense, it’s exactly the same as the other course!  The authenticity of the course (that it really is about programming a Gameboy) is entirely based on trust and that one example on the first day.

Story 2: We don’t trust you. Soon after we started teaching Media Computation here, we started having requests for a second Media Computation course.  No, students didn’t want to take the next CS class — they explicitly wanted to do more in “Media Computation.”  So we built one, which focuses on how the wildebeests were animated stampeding in Disney’s The Lion King.  That was Disney’s first experience with computer-modeled characters in a crowd (herd) simulation. Explaining that scene requires us to cover all the basic data structures as well as continuous and discrete simulations.  The course has been well received with a 90% success rate — pretty good for a data structures course mostly serving non-CS majors.

Faculty in Industrial and Systems Engineering (ISyE) asked us if their students could take this course.  They wanted their ISyE students to learn some Java, and a focus on simulations is right in-line with how ISyE professionals use computing.  However, nobody told their students that.

At Georgia Tech, ISyE typically has the smallest Freshman class and the largest Senior class of the Engineering programs.  Students don’t know what it is coming in, and then they discover ISyE while here.  When they first discover it, they don’t necessarily know what it’s all about.  In particular, they don’t understand the relationship between ISyE and Computing is.

I have gobs of surveys with student comments like, “Boy, will I be glad when this class is over with, because then I’ll never use a computer again!” and “Why do I have to take this stuff?  Industrial engineers don’t ever program and hardly ever even use computers!”  Now, we do tell them otherwise.  They simply don’t believe us. Gregory Abowd, when he taught this course, got an ISyE faculty member to come in and tell the students, “Honest! I do use simulations! My research group does program!  You will, too!”  Do we have to do that every semester?  Maybe — the students certainly don’t trust us on this point.

The Economics of Computing Education

Economics is a fascinating field.  It’s psychology-of-masses, a form of psychological engineering, and the closest thing we have to Hari Selden’s psychohistory (from Asimov’s Foundation series).  It’s a study of how people make choices in order to maximize their benefit, their utility.  It is not only about money–money is just a way of measuring value, about some common sense of the potential of some consumable for providing utility.  I’ve been reading more economics this summer, and that’s got me thinking about what economic theory might have to say about computing education.

Students, especially in undergraduate education, are clearly economic decision makers.  They choose their classes.  That isn’t to say that they are our customers whose wants we must meet.  It means that we provide consumables (classes) under various rule sets, and the students seek to maximize their benefit.

What students want from higher education (that is, what utility the classes are meant to provide) these days isn’t in much doubt.  Most studies of higher education that I’ve read suggest that a big change occurred in the 1970’s, and that since then, over 90% of incoming students in higher education are attending higher education in order to get a better job and improve their socioeconomic class. There is some evidence that suggests that students, by the time they are in their fourth year, they value education for its own sake more.  Students in their first years, on the whole, make choices based on their job prospects.

We’ve talked in this blog about why a student should study computer science.  One argument is because of the value of computing as a field and the insights that it provides.  Smart students will probably recognize that learning computing for those reasons will result in greater utility over the long run.  How do we get students to see value, to receive benefit from what we know will help them more in the long run?  Is it possible to teach students new and better utility functions?  Can we help students to realize the greater utility of valuing knowledge, even from their first years in higher education?  That’s an interesting question that I have not seen any data on.

What if we simply say, “This is the way it is.  I’m teaching you this because it will be the best for you in the long run”?  Paul Romer’s work on rule sets has been describing how the rules in effect in a country or a company can encourage or discourage innovation, and encourage or discourage immigration and recruitment. He would point out that higher education is now a competitive market, and deciding to teach for what the students should value is creating a set of rules.  Students who don’t value those rules will go elsewhere.  Those students who say will probably succeed more, but the feedback loop that informs us in higher education that we’re doing the right thing doesn’t currently exist.  Instead, we simply have lower enrollments and less tuition–not the right feedback.

It’s that last part, about the feedback on teaching, that I have been specifically thinking about in economic terms.  Malcolm Gladwell wrote a fascinating New Yorker piece last December about the enormous value of having a good teacher.  What makes for a good teacher?  Maybe those who create effective rule sets, who create incentives for student success?  What provides utility for teachers?  How do we make sure that teachers receive utility for good teaching?

How do we recognize and reward success in teaching?  I listened to a podcast of a lecture by William Wulf who points out how badly we teach in engineering education.  In economic terms, that’s not surprising. I don’t know of research into what university teachers value in terms of teaching. What is the utility function for a higher-education teacher, a faculty member?  Job prospects and tenure are based on publication, not teaching, at least in research universities.  When we do evaluate teaching, how do we do it?

• By measuring learning?  We’ve already pointed out in this blog how very hard it is to do that right.  Teachers use examinations and other forms of assessment.  Are they measuring the right things?  The research that I’ve seen suggests that grades are only rough measures of learning.  If we were going to measure learning as a way of rewarding faculty to incentivize better teaching, we need some external measure of learning apart from grades, and we need that measurement to be meaningful — that it reflects what we really value in student learning.
• By measuring student pass rates?  Wulf might say, “If only!”  He points out that correcting our 50% dropout rate in engineering (and computing!) education would alone dramatically improve our enrollment numbers.  Would we be dumbing down our education offerings?  Honestly, how would we know (see previous bullet)?
• Instead, we most often just ask the students.  “Was this a good class? Was this teacher a good teacher?”  This gets back to student as consumer, which is a step beyond decision maker.  Are they the right ones to make this determination?  Is the end of the class the right time for a student to be able to evaluate if the class was worthwhile?

Higher education teaching will probably improve once we figure out how to give reasonable feedback on teaching quality which could then impact teachers’ perception of benefit or utility.  As Gladwell and Wulf point out, getting it right would have a dramatic improvement on student quality and enrollment.

Correction and Update on APCS enrollment

About a month ago, I blogged on the impact of the Advanced Placement Computer Science (APCS) exam on undergraduate enrollment in computing. I cited some statistics about APCS that I have since discovered were wrong.  In particular, I claimed that there were 26 states whose total enrollment in APCS over the last 25 years has not been over 200.  That’s wrong.

Barb Ericson kindly gave me a spreadsheet with data from all 50 states over the last 10 years, so I can provide some more accurate observations.

• In the 10 year window that Barb gave me, there 9 states whose total number of APCS seats (a student took the APCS Level A exam) is below 100.  Those states are (from fewest taking to most taking) are Montana (at 25 students from 1998 to 2008), North Dakota, South Dakota, Wyoming, Nebraska, Oregon, Kansas, Alaska, and Mississippi (at 198).
• Those are also some of our least populous states, so those low numbers are not surprising.  There are fewer kids there (presumably) to take CS (though probably more than 200 high school kids total…)  What if we balance for state population?  Barb looked up the population in the state (total, not just of high school students, so it’s only a rough scaling factor) and came up a measure of tests taken in 2008 per million people, sortof a seats per capita.  There, Louisiana is lowest, with only 1.36 tests taken per million people. Montana, Wyoming, and Idaho are tied at 2.
• There are 18 states with less than 20 seats (on the 2008 exam) per million, which I’m using as a rough benchmark of “There’s one APCS teacher teaching one class of CS students per million people.” Some of those were pretty surprising to me: Iowa at 7 (21 students took it, with roughly 3 million people), Oregon at 8.78, Arizona at 10.8, Utah at 12.6, and West Virginia at 13.3.  Just outside my metric (tied at 21) are New Mexico, Michigan, and Minnesota.
• Who leads in producing APCS students? Maryland is the highest seats-per-million at 160.  The rest of top 10 are Texas, Virginia, Washington DC (51 students for 600K population), New Jersey, Connecticut, Georgia, Hawaii, California, and New York.

Overall, 15,014 students took the APCS Level A exam in 2008.  Just shy of half of those students (48%) came from three states: California, Texas, and New York.

In contrast, in 2008, 222,835 students took the AP exam for Calculus (Level AB).  57,758 took the AP exam for Physics Level B.  If we were to assume that high schools were perfect economic beings, and the number of those taking the test is a true indication of the importance of the field, then CS is 6% as important as Calculus, and 26% as important as Physics.

The College Board has found in its studies that 58% of high school students who take an APCS course end up taking a computer science course in college, and 19% of those pursue a computing degree – regardless of whether the students even take the exam.  In comparison, only 28% of high school students who do not take any APCS exam go on to take a computer science course, and only 3% of those students pursue a computing degree.  Simply taking the APCS course has an important impact on improving enrollments in computing.

Feynman lectures from Microsoft: A medium for active essays and computing ed

I got to see the Project Tuva videos at the MSR Faculty Summit during the last session of the day Tuesday.  If you haven’t seen them, I recommend them to you.  (Though, as Ian Bogost found out, you have to have the latest version of Microsoft Silverlight to watch them.)  These are Richard Feynman’s “Messenger Lectures” which he delivered at Cornell and recorded by the BBC.  The Project Tuva site enhances the video with the ability to take notes, read others’ notes (synched to the video), see links (that appear at the appropriate points in the video), including links into simulations and even the Worldwide Telescope.  So, when Feynman talks about how stars are formed, you click and go see telescope imagery of new stars being formed.  When Feynman talks about Tycho Brahe, you go to a simulation of planetary orbits so that you can make your own velocity measurements.

While I’m a Feynman fan (as are most scientists, engineers, mathematicians, and computer scientists I know), I was more excited about the medium than I was the Project Tuva lectures themselves.  I’m still looking for the right medium (and authoring tools) to express ideas in computation.  Books don’t cut it, and running programs go too far the other way.

Expressing computation to students is as pedagogically complex as expressing quantum mechanics or electrodynamics to physics students.  Most physics educators whom I’ve asked list those subjects as the most challenging to teach since the phenomena are impossible to directly observe and the behavior is non-intuitive.  All of computing is like that!  I can convince you that Biology is really about cells with a microscope, and most of Physics and Chemistry is about explaining the phenomena that you see every day.

While we see computing everyday, the computation behind those applications is difficult to see.  Just how many for loops are necessary to write that email?  Did you see all those linked lists behind your Powerpoint slide deck?  Applications that we use daily are layers upon layers of computation, such that it’s nearly impossible to see the low-level computation that we want to teach in an introductory course.  This is the problem of not having a microscope for computing.

So, we use books with source code in them.  To imagine the execution of a piece of source code is perhaps the most important and most intellectually challenging goal of an introductory course.  If human intelligence is computable, then the Halting Problem comes into effect, and for some level of complexity, we cannot figure out the execution just by looking at the source code.  Books with source code are a reasonable way of talking about computation, once you have some level of ability to imagine execution.  For rank beginners, it’s almost cruel — it’s like saying, “Let’s have you learn Russian by throwing you into Moscow without a coat in January.  Better figure out what Russian means quickly!”

Alan Kay has argued for “active essays,” a kind of dynamic book with simulations built in.  Ted Kaehler, Mitchel Resnick, and Brian Silverman have all built some reall interesting active essays.  An active essay could expose the source code, with explanation, and allow for execution within the same medium.  As a form of scaffolding, perhaps the source code could be tweaked in some meaningful ways, so that students could see the relationship between changes to the source code and impact on dynamic behavior — which is the most important thing to learn in an introductory class!

The enhanced video mode of Project Tuva could offer a way of doing “active essays” where the base medium is video rather than text.  Active essays have simulations embedded within text explanations.  We could, however, have videos explaining concepts, with simulations embedded within the video.  That’s an intriguing notion.  As I’ve blogged previously, there is psychology evidence that lectures are better for explaining computing concepts than just reading a book. (Yes, I discovered that my old Amazon blog posts are still there, if I can find a direct link to individual posts.)  Maybe with the Project Tuva enhanced video mode, we could finally do the Metaobject Protocol book in a compelling way.

Now, speaking of “books not cutting it,” the post-review-process version of the Data Structures manuscript is due next Wednesday, and I’d better get to it before my co-author discovers I’ve been spending my writing time blogging (again).

Are we measuring what students are learning?

One measure of the success of a talk is how many questions you get in the hallway after the talk.  I got a few yesterday, which suggests that people were still thinking about the points afterwards.

One question I got was about a finding we’ve had in several of the contextualized computing education classes, like robotics and Gameboys for computer organization.  Students report spending extra time on their homework beyond what’s required “just because it’s cool.”  Yet, in some cases, there is no difference in grade distributions or failure rates compared to a comparison class.  What gives?  Isn’t that a bad thing if students spend extra time but it’s not productive time?

Absolutely, that can be the case.  It may also be the case that students are learning things that we don’t know how to measure.  Think about the argument that it takes 10,000 hours of practice to develop expertise (a number that has been recalculated from several sources).  Can we come up with learning objectives for each of those 10,000 hours?  Or is it that we can measure some of those objectives, but others of the items being learned are subtle, or are prerequisite concepts, or are about skills, or even muscle memory?

A famous story in physics education is about how concepts are more complex and have more  facets than we realize.  David Hestenes has developed some sophisticated and multi-faceted assessments for concepts like “force” — a whole test, just addressing “force.”  Eric Mazur at Harvard scoffed at these assessments (as he said at a AAAS meeting I went to a couple of years ago, and quoted in a paper by Dreifus in 2007).  His Harvard students would blow these assessments away!  Gutsy man that he is, he actually tried them in his classes.  His students did no better than the averages that Hestenes was publishing.  Mazur was aghast and became a outspoken proponent of better forms of teaching and assessment.

Building up these kinds of assessments takes huge effort but is critically important to measure what learning is really going on.  For the most part in Computing Education, we have not done this yet.  Grades are a gross measure of learning, and to progress the field, we need fine-grained measures.

Talk at Microsoft Research on IPRE

I gave the Final Report talk on the Institute of Personal Robotics in Education effort at the Microsoft Research Faculty Summit today.  It went pretty well — slides are available.

One of the questions afterward was really intriguing for me.  I raised the issue during my talk, “All of us in the room grew up with computing learning how to compute Fibonacci numbers and the 100th digit of Pi.  What did that work for us when it measurably fails with today’s students?”  The question was, “Maybe because computing Fibonacci numbers and the 100th digit of Pi was the best of what we could do with computing then.  Computing resources were expensive.”  From this perspective, what many of us are doing now is “catching up” with where hardware and software development has brought us.    As my chair, Aaron Bobick argues, there’s a role for cutting-edge computing researchers  in education, in terms of understanding the implications of the latest computing technology for computing education.

That’s been a kind of a theme for the summit today: making things of yesterday finally work or work again.  This morning, Craig Mundie gave a live demo of a surface and wall computing “Office of the Future” that looked like Sun and Apple demo videos of years past.  As Michael Mateas put it, “what we saw is that it took 20 years to get rid of the Apple Knowledge Navigator’s bow tie.”  Andy van Dam just gave a demo of handwritten math recognition that was similar to things running on the System 360 thirty years ago.  Andy pointed out that it’s depressing that it takes so long to get these great ideas in a production system in many people’s hands.  A fascinating and important question is: what are the great, cutting-edge ideas of today that we’ll be working to make happen in products and classrooms over the next 20 years?

Amazon Blog being deleted

My old blog (http://www.amazon.com/gp/blog/A3W4CUXPE1WFNF) is all but gone now.  It looks like Amazon deleted all posts older than three months, so about three years worth of posts and comments are gone.  I’m  contacting them, hoping to get them back and post them somewhere.

Education is to Social Work, as Civil Engineering is to Chemical Engineering

I’m listening to Paul Romer’s Seminar about Long Term Thinking, and got to thinking about the SALT podcasts and TED talks.  These really are remarkable educational opportunities — really smart people, who are also really good at communicating their ideas to a lay audience.  These are not necessarily front-line scientists.  Michael Pollan and Malcolm Gladwell, for example, are both journalists who focus on taking important ideas from science (and economics and…) and making them accessible.  Why is that uncommon? We have relatively few people who do this kind of thing, as opposed to all scientists or even all educators.  Is it because that combination of talents is so rare, or because there is little market, interest, or demand for it?

Seymour Papert once argued that educational curricula should be evaluated like art — don’t try to identify the best, but instead argue about how well this example expresses something, or how accessible another one is, or how another one leaves people thinking and talking for years later.  Compare curricula for how they reach and engage people, not for a measurable, numeric bottom line.  Wouldn’t it be great to have so many compelling CS1 curricula that we could have a CS1 “art gallery” and compare them along the lines Seymour described?

Let’s imagine that we wanted to have more education that was engaging, compelling, and explained things to people.  We’d have to re-organize how we teach and structure education.  In fact, that would go against the basic structuring mechanisms of universities.

When I was at the University of Michigan, there was a lot of excitement about the proposed increased connections between the School of Education and the School of Social Work.  At some places, like Northwestern University, these are housed in the same schools.  That makes sense because the goals of Social Work are very similar to the goals of Education — improved human development, meeting human potential, individual self-reliance, and so on.

However, if we grouped scholars in terms of methods, we would structure universities very differently.  I’ve always found it odd that Physics and Mechanical Engineering are in separate schools/colleges at most Universities, and the same with Chemistry and Chemical Engineering.  Aren’t these really the same things, relying on the same theories, doing similar experiments?  Instead, we group by outcomes.  Civil, Chemical, and Mechanical Engineering are all about applying science to solve problems for people, at a large scale (by creating bridges and buildings, chemical plants, manufacturing capacity).  Never mind that what I see faculty in Chemistry and Chemical Engineering doing much more similar things than faculty in Civil Engineering and Chemical Engineering.

If we did group by methods rather than outcomes, what disciplines would be the natural collaborators for Education?  What disciplines would lead us to think about how we do things, so that we could create the kind of curriculum-as-art that Seymour described?

• Journalism, which also cares about methods for finding “truth,” for conveying that to people in ways that are understandable and compelling, and for structuring the story so that the punchline is up front, and the greater detail is at the end.
• Theater, because lecture is a kind of performance. Experimental Theater does a better job of getting the audience interacting with the performance than do most lectures!
• Medicine, which is (much more than Education) about meeting individual needs and figuring out how to tailor broad approaches to health for the individual’s particular combination of strengths and illnesses.
• Film and Television Studies, which know a lot about using multiple media for creating a compelling story.  Everyone who does On-Line/Distance Education should take a Film Class, to figure out how you package a compelling story/experience for others whom you never see.
• Theme Park Designers (yeah, I know it’s not an academic discipline, but maybe it should be).  I’m a big Disney Imagineering fan.  Imagineers know how to draw you into the ride with the prestory, setting expectations and explaining the context, and then giving you an experience that you talk about and remember later.
• Economics, because in the end, most Educational decisions are economic ones.  We know how to get two-sigma improvements in learning — give everyone a personal tutor.  That’s too expensive to do at scale.  Everything else we do is a step down from that, and if we knew how economists think about these trade-offs, it might help us in Education recognize our trade-offs and where we’re making them.
• Psychology, because Education is just Psychology Engineering.  If in a methods-oriented University we lump Chemistry and Chemical Engineering together, we certainly should put the Psychologists and the Education faculty in the same building.

Okay, I’ll get back to my Faculty Summit talk preparation now, but I’m thinking about how the quality of education should be as much about the student’s experience as about the student’s performance on the test.

Point/Counterpoint: CS Education headed in wrong direction?

The July 2009 CACM also has a great point/counterpoint piece by Robert Dewar and Owen Astrachan asking “Is CS Education in the US headed in the wrong direction?”  I’m not going to give my personal opinion on the pieces, because I solicited both articles for CACM.  I am really pleased with both position papers, and how the overall piece came out.  It’s really about what do we want from our education system — what does an undergraduate degree in computing mean?

(For testing purposes: xky7qhc4r2)

I am a nearly-50 year old curmudgeon: Kids today are like kids yesterday

Despite the fact that my wife and I work on the same campus, we don’t often get the chance to travel in together.  She’s got off-campus work (like visiting high school teachers), and our schedules don’t always align, and there are the activities of three kids to juggle. This morning we did get to drive in together, and I exposed my curmudgeon-ness.

“I think there is something to the claim that this generation is different.  They’re more into immediate feedback, like Twitter. They’re less interested in more complex activities that require sustained effort and are sometimes tedious.”  My wife then went on to explain, in great detail, all the ways that kids today do engage in complex and even tedious actitivies.

• Our son is in his high school’s marching band and orchestra.  Over 1/3 of the school is in the music program–public high school, and that’s not really that unusual (at least around Atlanta).  Learning to play an instrument well is really complex, requires sustained effort, and is frequently painfully tedious.  Yet, lots of kids do it, and from watching the marching bands around here, they’re really putting in the time.
• The Harry Potter books are long and are the best-selling book series ever.  Kids today don’t read?
• Ever watched kids playing video games?  Talk about complex!  Ever seen a kid decide to beat a game?  Starting and restarting the game, making the same mistakes over and over.  Tedious, complex, and sustained effort.
• How do kids text so fast on those normal numeric-keyboard cellphones?  That’s a serious study in tediousness.  Typing semi-colons in Java is childs play in comparison!

“Then,” I asked, “how do you explain the decline of interest in the Western world in science, mathematics, engineering, and computing?”  “Fads,” she replied.  Our parents thought that we were uninterested in tedious, complex, sustained effort, too — just because we weren’t interested in their activities.  We had our own activities.  Our kid’s have their own.  Things become popular, and less popular.  Motivation rises and falls.  In the end, getting more kids into STEM disciplines is about motivating them, showing them why these fields are valuable, interesting, and worth the effort.

Good teaching helps.  Our daughter is a natural scientist.  She is forever “trying this out,” and even setting up comparison cases.  Her bedroom is filled with dead bugs, rocks, and solutions that I pitch out when I find them.  Yet, she claims that she “hates science.”  Science for her is filling out worksheets.  The teachers aren’t comfortable doing any of the real experiments, the real science, that are in the textbook.  The teachers don’t like science, so they teach it poorly.

Computing is fading from students’ interests because, in the end, it’s about student interests.  No, they may not be our “customers,” but they are certainly our “audience.”  Gaining and sustaining student interest is a critical factor in students pursuing computing education and careers, and good teaching by teachers who themselves like and know computing is a critical factor in gaining and sustaining student interest.

I’ve got a really smart wife.  I need to ride in with her more often.  And I probably need to be less of a curmudgeon.

(I’m doing two posts today because I won’t be able to post for the next week or so.  I’m giving a plenary talk at the Microsoft Research faculty summit next Monday.)