Posts tagged ‘learning’

Time spent is about expertise developed: Programmer productivity is an education problem

Joel Spolsky argues that the time spent on an assignment by students is not correlated with the results.  This is part of his argument that there is a 5x to 10x productivity gap between the best and average programmers.

There’s just nothing to see here, and that’s the point. The quality of the work and the amount of time spent are simply uncorrelated. I asked Professor Eisenstat about this, and he pointed out one more thing: because assignments are due at a fixed time usually midnight and the penalties for being late are significant, a lot of students stop before the project is done. In other words, the maximum time spent on these assignments is as low as it is partially because there just aren’t enough hours between the time the assignment is handed out and the time it is due. If students had unlimited time to work on the projects which  would correspond a little better to the working world, the spread could only be higher.

via Hitting the High Notes – Joel on Software.

I did my Blog@CACM post in November on this argument (see the post here).  Joel is measuring the wrong thing in his experiment.  Time spent doesn’t correlate with quality.  Time on task correlates with learning.  More knowledge correlates with greater quality, but you don’t know much about how much is already known by the students in the class that he’s studying.  Therefore, you can’t possibly make an argument about productivity when the critical variable is unknown.

There may be a productivity gap between the best and average programmers.  There’s a productivity gap between the best and average practitioners in any field.  The issue is whether we can generate the performance at the top end.  Can we make someone highly productive?  It’s a design problem.   It’s an education problem.

November 28, 2014 at 8:40 am 8 comments

Doubts of my students: Expert teaching is no better than good-enough teaching

Teaching is a great job.  I particularly appreciate how teaching keeps me thinking and questioning, which is particularly important for an education researcher.  I’m teaching two classes this semester.  I’ve mentioned recently how my data structures class has me thinking about new kinds of practice activities.

I am also teaching a course on educational technology, where we’re reading How People Learn.  Chapter 7 is a fascinating read with three detailed accounts of high-quality learning environments with expert teachers, one each in history, mathematics, and science.  The chapter includes some strong claims about teaching:

The interplay between content knowledge and pedagogical content knowledge illustrated in this chapter contradicts a commonly held misconception about teaching–that effective teaching consists of a set of general teaching strategies that apply to all content areas. This notion is erroneous….These examples provide glimpses of outstanding teaching in the disci- pline of history. The examples do not come from “gifted teachers” who know how to teach anything: they demonstrate, instead, that expert teachers have a deep understanding of the structure and epistemologies of their disciplines, combined with knowledge of the kinds of teaching activities that will help students come to understand the discipline for themselves. As we previously noted, this point sharply contradicts one of the popular—and dangerous—myths about teaching: teaching is a generic skill and a good teacher can teach any subject.

We had a great discussion in class about this last night.  HPL is claiming that an expert teacher has (1) discipline knowledge, (2) understanding about teaching and learning, (3) understanding of conceptual barriers that students face in the discipline, and (4) a set of effective strategies for addressing those conceptual barriers.  (3) and (4) on that list is what we call pedagogical content knowledge, discipline-specific knowledge for how to teach that discipline.  My students don’t argue that CS PCK (pedagogical content knowledge about teaching CS) doesn’t exist.  They just argue that it’s not necessary to be “effective.”

It may be a “dangerous myth” but my students cling to it pretty stubbornly.  “If you know the content, and you know about how people learn, then you can teach that content.  You may not be as good as a teacher with years of experience, but you’re good enough.”  That’s almost an exact quote from one of the students in my class last night.  I tried to argue that, not only is it better to have CS PCK, but we can also teach CS PCK, so that a first year teacher can be much more effective than a brand new teacher who doesn’t know anything about student problems or teaching strategies.  They pushed back.  “How much more does PCK contribute to being a good teacher, beyond just knowledge of the discipline and knowledge of learning sciences?”  Since I don’t know how to measure knowledge of CS well, nor how to measure CS PCK, I have two unknowns, so I can’t really answer the question.

One way of interpreting my students’ comments is sheer hubris.  These are young, smart Georgia Tech undergrads (and a smattering of grad students).  In their minds, they are intellectually invulnerable, able to tackle any academic challenge, and certainly better than any teacher from a school of education.  Several of them mentioned Teach for America in their comments, an organization whose existence encourages them to think that teaching is not so hard.  Maybe their comments also are the thoughts of expert learners — these students have had to teach themselves often, so they don’t see expert teaching as a necessity.

Another way of interpreting my students’ comments which is much more intellectually challenging is that the difference between an effective and expert teacher is hard to see.  A recent NYTimes article speaks to the enormous value of expert teachers — over a student’s lifetime.  Barbara has pointed out that, in her experience, the first year that a teacher teaches AP CS, none of his or her students will pass the AP CS (with a score of 3 or better).  Even some veteran teachers have few test-passers, but all the teachers who get many test-passers are veterans with real teaching expertise.  But how do you make those successes visible?  As we’ve talked about here before: How do we measure good teaching?

As a teacher of education research, I wasn’t so successful yesterday.  I failed at convincing my class (at least, a vocal group of students in my class) that there is some value in expert teaching, that it’s something to be developed and valued.  What I worry is that these are not just the thoughts of a few undergraduates.  How many more people think that it’s easy to learn to be a teacher?  How many other adults, voting citizens, even members of school boards agree with my students — that expert teaching is not that much better than effective teaching, so hiring a bunch of young, smart kids to teach is good enough?

February 24, 2012 at 10:27 am 23 comments

Bret Victor’s “Inventing on Principle,” and the trade-off between usability and learning

I have had several people now send me a link to Bret Victor’s video on Inventing on Principle. It is a really impressive demo!

His system reminds me of Mike Eisenberg’s work on SchemePaint.  Mike wanted the artist to be able to interleave programming and direct manipulation.  In SchemePaint, you could draw something by hand, then store the result in a variable to manipulate in a loop.  Or you could write some code to tesselate some graphical object, then add tweaks by hand.  It was beautiful.  The work that Mike did on SchemePaint led to his wonderful work on HyperGami, a CAD system for origami, which was the start of his Craft Technology group. That’s the group from which Leah Buechley graduated — she did the LilyPad.

People are sending me Bret’s video asking, “Wouldn’t this be great for learners?”  I bet it could be, but we’d have to try it out. At one point in his lecture, Bret says, “Why should I have to simulate the computer in my head?”  Because that’s the point of understanding computer science.  Bret’s system looks like a powerful visualization system, and visualization can be used to lead to real understanding, but it isn’t easy to design the visualization and context such that learning occurs.

The problem is that visualization is about making information immediate and accessible, but learning is about changes in the mind — invisible associations and structures.  Sometimes good usability makes it easier to make these associations and structures.  Tools like Scratch and Alice increase usability in one direction (e.g., syntax) while still asking students to make an effort toward understanding (e.g., variables, loops, and conditionals).

My first PhD student was Noel Rappin, who explored the features of modeling environments that lead to learning.  He had a CHI paper about his work on helping chemical engineers learn through modeling.  Our colleagues in chemical engineering complained that their students couldn’t connect the equations to the physical details of the pumping systems that they were modeling. Noel built a system where students would lay out the physical representation of a pumping system, then “look underneath” to see the equations of the system, with the values filled in from the physical representation (e.g., height difference between tanks).

He ran a pilot study where students would lay out a system according to certain characteristics.  They would then manipulate the system to achieve some goal, like a given flow rate at a particular point in the system.  When Noel asked the pilot students if they gained any new insights about the equations, one student actually said, “What equations?”  They literally didn’t see the equations, just the particular value they were focusing on.  The system was highly usable for modeling, but not for learning.

Noel built a new system, where students could lay out a model, and values from the model were immediately available in an equation space.  To get the flow rate, the student would have to lay out the equations for themselves.  They would still solve the problem by manipulating the physical representation in order to get the right flow rate, and the system would still do all the calculations — but the students would have to figure out how to compute the flow rate.  The system became much harder to use.  But now, students actually did learn, and better than students in a comparison group.

Bret’s system is insightful and may have some terrific ideas for helping learning.  I’m not convinced that they’re new ideas yet, but an old idea in a new setting (e.g., JavaScript) can be powerful.  I worry that we get too entranced by improvements in usability.  In the end, learning is in the student, not in the system.

February 21, 2012 at 7:50 am 17 comments

What I learned from working on the play

I mentioned a while ago that I decided to be part of a play, inspired in part by Seymour Papert’s recommendation to always go learn something new. I have been meaning to report back on some of what I learned.

The play performances were in February, and it’s taken me quite a while to catch up after the play.  (There is a video on-line of the one scene where me, Barb, and our daughter were all on stage.)  A couple weeks before the first performance, I was asked by one of the directors to be part of the stage crew in the rest of the scenes.  In terms of the time commitment, agreeing to do that was an enormous time cost, since I had to be there early to set up, never had down-time during the play, and had to stay after for cleaning up.  In terms of learning, it was terrific — I got to see and do even more.

The amount of planning required was absolutely astounding.  Every scene was meticulously choreographed, both off-stage and on-stage.  One example (out of 20-30 similar activities): While Maria was singing about “having confidence,” I had to hand Steve (another member of the set crew) the drapes to go on the flats that would soon become the walls of Maria’s bedroom, which were the same flats that used to have stained-glass windows in them for the Mother Superior’s office, and I had to be done in time to be stage left to move another flat during the next scene change to create the Von Trapp’s home.  The directors made up maps and plans for every scene and every scene change which told us who did what jobs, where the smaller pieces should be found off-stage (e.g., the drapes were hanging on the closet door in the music room) and where the larger pieces should be moved (e.g., the stained glass windows went off-stage left for re-installation at intermission).  The number of moving parts, moving people, and props and set pieces was beyond my ability to juggle objects without variable names or data structures.  (My biggest challenge in the play: Moving a seven foot tall wardrobe up and down a ramp in the dark. I so worried about it tipping over and taking out the front row!)

There was a lot more mathematics and geometry than I had realized previously.  Part of the set for the church where Maria and Captain Von Trapp marry was this huge, beautiful, hand-painted triptych.  One panel was double jointed, which seemed unnecessary to me when I first saw it.  I then realized that the double joint was key to getting the triptych unfolded and folded (part in the stage wing, where there was enough room) without touching the curtain.  I don’t know who figured out how that could possibly work, but somebody did, and we had mere inches to spare.

There was new vocabulary to learn.  “Upstage/downstage” and “stage left/right” are not that hard to understand, but I had to rehearse for some level of automaticity .  When the stage manager whispers to you, with only a couple measures left for the orchestra before the lights come up, “The stage left tombstone is positioned wrong — move it three feet upstage,” you have to be running in the dark while you finish mumbling “stage left from the actor’s perspective, upstage away from the audience.”

As a computing educator, I look for examples to use in class when talking about computing.  A play is all about algorithm — it’s all about getting a process down that you can repeat, flawlessly, for every performance, both backstage and on-stage.  However, there’s almost no abstraction.  There are roles, but when it comes to the stage crew, everything was special cased.  For example, Mark gets the stage left flat unless Keefe has to be changing into his Nazi costume or Mary’s not available from the last quick-change of the children so Mark has to get her table and someone else gets the flat but it’s all different if Mark is changing into his tux for the party scene.

Something I found interesting was the work and my relation to it.  I didn’t need my college degrees for this.  Age didn’t matter either — past experience on other plays and practice on this play were what mattered.  My skill became more specialized by the time of the performances, but I was the least skilled because I was least experienced.  I was the newbie on the stage crew — everyone else had done it many times before, from the mid-50’s leaders of the stage crew, to the two 15 and 11 year old boys (who had worked on several plays previously) with whom I worked most closely.  It was very satisfying work–maybe it was the sense of community in working with the large group, or maybe it was the good cause.  (We ended up raising $17K for a local homeless shelter and food bank.)  At one point, the director asked the stage manager if she could use me to take the place of an actor who was ill, and the stage manager exclaimed, “You can’t have him! You can’t have the bedroom or church scenes if you take him!” And I was quite pleased — I liked the fact that my work was valued, despite the fact that just about anyone could have learned to do the job.

I had enormous fun doing the play.  I enjoyed working with everyone, I liked doing the dancing and being on-stage, I enjoyed feeling that I had a job that needed doing, and I thoroughly enjoyed seeing all that talent.  Wow!  The singing and dancing ability of my neighbors and fellow parishioners so impressed me.  I don’t know yet if I’ll do the play again next year (or in future years).  I work a lot in the evenings, like most academics.  I lost a lot of that time for well over a month.  I got teased for doing grading off-stage, holding my stage crew flashlight under my chin while I worked.  (A role for the stage crew that I never realized before: As the only people onstage with flashlights, we were often used as movable lightpoles — “Go stand there and shine your light on the floor so that the children can find their way off stage.”)  We’ll see how much I remember the fun and forget the pain by Fall when auditions for the next play start.

April 12, 2011 at 8:23 am 3 comments

Finding hope in a book-less world

I also finished the Smithsonian magazine 40th anniversary issue on the way back from China.  (It’s a REALLY long trip.)  There were three pieces that I think speak to each other, to point to danger in a book-less world, and a possible reason to have hope.

Kevin Kelly explicitly predicts the end of the book over the next 40 years, and describes how the book-less world will be different:

“In books we find a revealed truth; on the screen we assemble our truth from pieces. On networked screens everything is linked to everything else. The status of a new creation is determined not by the rating given to it by critics but by the degree to which is linked to the rest of the world.”

Vint Cerf’s interview points out (one of) the dangers of this future world:

“[The Web is] a little bit like television. When it arrived there were many expectations that it would improve education and everything else. But what we discovered is there’s a finite amount of quality in the universe, and when there are more channels it has to be cut up into smaller and smaller amounts until finally, every channel delivers close to zero quality, and that’s where we are today, with a few exceptions.”

So Kelly is saying that students won’t read to learn truth from a master — they’ll construct their own truth out of the wide range of what has been written.  And Cerf is saying, “And there’s almost nothing good out there.”  When I read the two of these pieces, I felt dismayed.  It feels like Meno’s paradox. How can you find truth, if you don’t know the truth already?  And isn’t it all the more harder if there’s no or little truth out there to work from?

The hope comes from considering Cerf’s caveat “with a few exceptions.”  There’s something out there.  The student must be diligent in finding it and careful in evaluating it.  The student needs a critical eye.  Maybe that’s actually a huge advantage over where we are today, where students tend to memorize more and sense-make less.

Pre-Web, most people in the United States got their news from only one source.  Even today, how many people get most of their news and viewpoints from Fox News?  How many teachers teach using only one textbook or resource, and how many students use only a single source for learning a given curricular topic?  (And how does that contrast with the number of sources they use when they care about the topic?)  In contrast, how many scientists or doctors use only a single source for all their decisions?

It’s a positive direction for people to learn to work to gather information, to have to evaluate it, and to keep going until they come to a personal understanding.  As teachers, we’ll have to help students learn these skills.

I particularly liked the interview with Sabiha Al Khemir, an expert on Islamic art.  I thought that her comments encouraged this style of thinking, to rely less on the expert, and more on the individual student’s effort at assembling “small” pieces at the “intimate” level:

“Making that effort and wanting to find out is part of the duty of each one of us. Most Islamic art is not even signed; most is anonymous. The concept of a masterpiece is not the same as in the West. The concept of the artist is not the same. This is not art that was produced to be hung on the walls. The scale is much smaller, which calls for an intimate relationship. Basically, it is calling you to come close and look, to accept that it is different and try to understand that even though it’s small, it might have something to say. Maybe it’s whispering. Maybe you need to get closer.”

November 11, 2010 at 9:49 am 9 comments

Good teachers are born not made?

American Radioworks is making two claims in this piece that I find disturbing (though both could very well be true). First, that it’s not possible to make teachers effective — they’re either good or they’re not.  From the below quote, the authors of the piece aren’t comfortable with that idea either, since they quickly shift to a project showing progress in improving teachers and thus dispute Hanushek’s claim.

But the scarier claim is the implicit challenge to the old Bruner claim: “We being with the hypothesis that any subject can be taught effectively in some intellectually honest form to any child at any stage of development.”  I’m presuming that adults (even ineffective teachers) are a kind of “child at any stage of development.”  At some point, is the Bruner hypothesis false, and we just have to give up? That there are humans who lose the plasticity of their cognitive systems and can no longer be reshaped and reformed?

Is it possible to take ineffective teachers and make them better? Economist Eric Hanushek, who has done some of the most influential research about the importance of teachers, thinks the answer is, “no.”

Hanushek: My interpretation of the evidence is that teachers are born and not made.

There have been only a few big studies of programs that are supposed to help teachers improve, and the evidence is: they don’t work. That’s why Hanushek thinks the focus should be getting rid of bad teachers, and recruiting better ones. But there are more than three million teachers in the United States. If every child is really going to have a good teacher, there needs to be some way to help teachers improve.

via Testing Teachers — American RadioWorks — Transcript.

September 3, 2010 at 1:39 pm 14 comments

Guitar Hero as a Form of Scaffolding

My daughter turned 12 on Tuesday, and unfortunately, she was ill.  Dad hung out with her, and played whatever video games she wanted.  One of those she picked was Guitar Hero, so I finally got time to play it repeatedly.  Y’know — it was kind of fun!

Back in December, when I first got Guitar Hero, I wrote a blog post where I agreed with Alan that Guitar Hero is not nearly as good as learning a real musical instrument.  At that time, I wrote:

Guitar Hero might still be fun.  But it’s just fun.  I might learn to do well with it.  But it would be learning that I don’t particularly value, that makes me better.

Now I’m thinking that I might want to eat those words.  I found Guitar Hero hard.  I own a guitar and have taken guitar lessons for two semesters.  (Even putting it in terms of “semesters” suggests how long ago it was.)  Some of my challenges in learning to play a guitar included doing two different things with my hands, and switching chords and strumming to keep the rhythm.  I noticed that that’s exactly what I was having a hard time doing with Guitar Hero.  I also noticed the guitar parts of rock songs — songs that I had heard a million times before but never had noticed all the guitar parts previously. I noticed because I missed my cues, and so those guitar parts were missing.  While I have known Foghat and Pat Benatar for literally decades, Guitar Hero had me listening in a different way.

It occurred to me that Guitar Hero could be a form of scaffolding, a reduction in cognitive load that allows one to focus on one set of skills before dealing with all the skills at once.  Cognitive scaffolding is much like the physical scaffolding, “a temporary support system used until the task is complete and the building stands without support.”  Now, Guitar Hero would only be successful as a form of scaffolding if it actually leads to the full task, that it doesn’t supplant it.  In education terms, if Guitar Hero could fade and if it doesn’t lead to negative transfer, e.g., “I’m great at Guitar Hero, but a real guitar is completely different.”

I did some hunting for studies that have explored the use of Guitar Hero to scaffold real music education.  I could not find any educational psychology or music education studies that have explored Guitar Hero as a form of scaffolding or as a tutor to reduce cognitive load.  I did find papers in music technology that hold up Guitar Hero as a model for future educational music technology! My favorite of these is a paper by Percival, Wang, and Tzanetakis that provides an overview of how multimedia technolgoies are being used to assist in music education.  They point out additional lessons that students are learning with tools like Guitar Hero that I hadn’t noticed.  For example, the physical effort of playing an instrument is more significant than non-players realize, and Guitar Hero (and similar tools) build up the right muscles in the right ways (or so they theorize — no direct studies of Guitar Hero are cited).  The paper also argues that getting students to do something daily has a huge impact on music learning and performance, even if it’s a tutorial activity.

Now here’s the critical question: Does Guitar Hero lead to real music playing, or is it a stopping point?  Nobody is arguing that playing Guitar Hero is making music, that I can see.  Does it work as scaffolding?

I don’t know, but I’m now wondering: Does it matter?  If Guitar Hero stops some people from becoming musicians, then it is a problem.  If some people, who might have pushed themselves to become musicians, decide that Guitar Hero is hard enough, then Guitar Hero is doing a disservice.  But if that’s not true, and people who never would become musicians, have a better appreciation for the music and a better understanding of the athleticism of musicians because of Guitar Hero, then Guitar Hero is providing a benefit.

These are computing education questions.  You have all heard faculty who insist on using Eclipse in their introductory classes, because that’s what real software engineers use.  We have recently read in comments on this blog that students should use “standard tools” and “learn science the way scientists understand it.”  We also know from educational psychology that engaging introductory students in the same activity as experts only works for the best students.  The bottom half of the students get frustrated and fail.

We need Guitar Hero for computer science.  We need more activities that are not what the experts do, that are fun and get students to practice more often, that are scaffolding, and that reduce cognitive load.  We have some, like Scratch and eToys.  We need more. Insisting on the experts’ tools for all students leads to the 30-50% failure rates that we’re seeing today.  We have to be doing more for the rest of the students.

October 29, 2009 at 9:52 am 13 comments

The Learning Process for Education Research

One of the more influential projects in physics education (and learning sciences over all) was the effort by Jill Larkin and Herb Simon to characterize how students and experts solved physics problems. They found that students tend to look at physics problems at a shallow level, while experts see deep structure.  Students tend to look at what variables are present in the problem, match them to the equations given in class, and see what they can compute with those variables.  Experts look at a problem and identify the kind of problem it was, then work out a process to a solution.

My son is currently taking AP Physics, and I’m seeing this same process when he asks me for help.  My dissertation work was about teaching students kinematics by having them build simulations, so I’m familiar with some of the content.  I’m no expert, but am a bit closer than my son.  Matt brought me a problem then started with, “I can figure out delta-Y here, but can’t see why that’s useful.”  He knew the equation that matched the variables.  I drew a picture, then figured out what we needed to compute.  I then remembered the wrong equation (evidence that I’m no expert) and came up with an answer that clearly couldn’t be right. (Kudos to Matt for realizing that!)  Matt then figured out the right equation, and we came up with a more reasonable answer. I worked from the problem situation to an equation, and Matt started by looking for an equation.

I’ve been seeing this same process lately in how people come to understand education research.  I’m teaching an undergraduate and graduate (joint) class on educational technology this semester.  (We just started class last week.)  In the first week, I had them read two chapters of Seymour Papert’s Mindstorms; the paper “Pianos, not Stereos” by Mitchel Resnick, Amy Bruckman, and Fred Martin; and Jeanette Wing’s “Computational Thinking.”  I started the class discussion by asking for summary descriptions of the papers.  A Ph.D. student described Jeanette’s position as “Programming is useful for everyone to understand, because it provides useful tools and metaphors for understanding the world.”  I corrected that, to explain that Jeanette questions whether “programming” is necessary for gaining “computational thinking.”  The student shrugged off my comment with a (paraphrased) “Whatever.”  For those of us who care about computing education, that’s not a “whatever” issue at all — it’s a deep and interesting question whether someone can understand computing with little (or no?) knowledge of programming.  At the same time, the student can be excused for not seeing the distinction. It”s the first week of class, and it’s hard to see deep structure yet.  The surface level is still being managed. It’s hard to distinguish “learning programming” and “learning to think computationally,” especially for people who have learned to program.  “How else would you come to think computationally?”

This last week, we’ve been reviewing the findings from our first year of our Disciplinary Commons for Computing Educators where we had university and high school computer science teachers do Action Research in their own classrooms. Well, we tried to do Action Research.  We found that the teachers had a hard time inventing researchable questions about their own classrooms.  We ended up scaffolding the process, by starting out with experimental materials from others’ studies, so that teachers could simply pick the experiment that he or she felt would be most useful to replicate in his or her classroom.  We then found that the teachers did not immediately see how the results had any implication for their own classrooms.  It took us awhile to get teachers to even ask the questions: “The results show X (e.g., most students in my classroom never read the book).  What does that mean for my students?  Does that mean X is true for all my students?  Should I be doing something different in response?”

These results aren’t really surprising, either — at least in hindsight.  High school and university teachers have their jobs not because they are expert at education research.  University researchers typically are expert at some computing-related research, not computing education research, and a general “research perspective” doesn’t seem to transfer.  Our teachers were looking at the surface level, and it does take some particular knowledge about how to develop researchable questions and how to interpret results into an action plan afterwards.

Education research is a field of study.  I’ve been doing this kind of work for over 20 years, so you’d think I’d have realized that by now, but I still get surprised.  Simply being a teacher doesn’t make you an expert in education research, and being a domain researcher doesn’t make you an expert in education research in that domain.  It takes time and effort to see the deeper issues in education research, and everyone starts out just managing the surface features.

August 24, 2009 at 3:27 pm 7 comments

Are we measuring what students are learning?

One measure of the success of a talk is how many questions you get in the hallway after the talk.  I got a few yesterday, which suggests that people were still thinking about the points afterwards.

One question I got was about a finding we’ve had in several of the contextualized computing education classes, like robotics and Gameboys for computer organization.  Students report spending extra time on their homework beyond what’s required “just because it’s cool.”  Yet, in some cases, there is no difference in grade distributions or failure rates compared to a comparison class.  What gives?  Isn’t that a bad thing if students spend extra time but it’s not productive time?

Absolutely, that can be the case.  It may also be the case that students are learning things that we don’t know how to measure.  Think about the argument that it takes 10,000 hours of practice to develop expertise (a number that has been recalculated from several sources).  Can we come up with learning objectives for each of those 10,000 hours?  Or is it that we can measure some of those objectives, but others of the items being learned are subtle, or are prerequisite concepts, or are about skills, or even muscle memory?

A famous story in physics education is about how concepts are more complex and have more  facets than we realize.  David Hestenes has developed some sophisticated and multi-faceted assessments for concepts like “force” — a whole test, just addressing “force.”  Eric Mazur at Harvard scoffed at these assessments (as he said at a AAAS meeting I went to a couple of years ago, and quoted in a paper by Dreifus in 2007).  His Harvard students would blow these assessments away!  Gutsy man that he is, he actually tried them in his classes.  His students did no better than the averages that Hestenes was publishing.  Mazur was aghast and became a outspoken proponent of better forms of teaching and assessment.

Building up these kinds of assessments takes huge effort but is critically important to measure what learning is really going on.  For the most part in Computing Education, we have not done this yet.  Grades are a gross measure of learning, and to progress the field, we need fine-grained measures.

July 14, 2009 at 7:53 am 11 comments

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 9,032 other followers


Recent Posts

Blog Stats

  • 1,996,053 hits
January 2022

CS Teaching Tips