Carl Wieman Finds Colleges Resist Measuring Teaching

July 12, 2013 at 1:05 am 12 comments

I’ve just started my subscription to The Chronicle of Higher Education, and the first print issue I received had a great article about Carl Wieman, whom I have written about previously (here and here and here, for just three).  The story (online here: Crusader for Better Science Teaching Finds Colleges Slow to Change – Government – The Chronicle of Higher Education) was about his efforts to get the White House to measure teaching practices.

At the White House, Mr. Wieman tried to figure out what might actually get colleges and their faculty members to adopt proven teaching practices. His centerpiece idea was that American colleges and universities, in order to remain eligible for the billions of dollars the federal government spends annually on scientific research, should be required to have their faculty members spend a few minutes each year answering a questionnaire that would ask about their usual types of assignments, class materials, student interaction, and lecture and discussion styles.

Mr. Wieman believed that a moment or two of pondering such concepts might lead some instructors to reconsider their approaches. Also, Mr. he says, data from the responses might give parents and prospective students the power to choose colleges that use the most-proven teaching methods. He hoped the survey idea could be realized as either an act of Congress or a presidential executive order.

I hadn’t heard about this survey, but my immediate thought was, “What a great idea!”  We need better ways to measure teaching (like with Sadler’s recent work), and this seems like a great first step.  I was surprised to read the response

College leaders derided it as yet another unnecessary intrusion by government into academic matters.

“Linking federal funding for scientific research to pedagogical decisions of the faculty would have set a terrible precedent for policy makers,” said Princeton University’s Shirley M. Tilghman, one of several presidents of major research institutions who wrote to the White House to complain about Mr. Wieman’s idea. “It is naïve to think that the ‘surveys’ will not have consequences down the line.”

Wouldn’t “consequences” be a good thing?  Shouldn’t we reward schools that are doing more to improve teaching and adopt better practices?  Shouldn’t we incentivize schools to do better at teaching?  I guess I’m the one who is naïve — I was surprised that there was so much resistance.  In the end, Wieman lost the battle.  He’s now left the White House, dealing with multiple myeloma.

Perhaps the saddest line in the piece is this one:

“I’m not sure what I can do beyond what I’ve already done,” Mr. Wieman says.

Is it really impossible to get universities to take teaching seriously?


Entry filed under: Uncategorized. Tags: , , , .

Context matters when designing courses, too: Know Thy Learner Automatically grading programming homework: Echoes of Proust

12 Comments Add your own

  • 1. alanone1  |  July 12, 2013 at 1:55 am

    (Need to subscribe to Chronicle in order to read the article …)

    Only a small percentage of profs and TAs are intrinsically motivated to do anything well — especially teaching, and universities further motivate them against by counting papers and research more heavily than teaching, and especially teaching excellence.

    I found it sad that Carl is referred to as “Mr” Weiman, rather than at least “Prof” Weiman, or “Dr” Weiman or “Nobelist” Weiman — he’s earned real respect for his opinions in many ways, and I think it would be good for most readers of this piece to be reminded of this.

    That said, I have always liked better the scheme his fellow astrophysicist at U of Colo — Dick MacCray — invented for his classes there, which involved creating online “active essays” that the students had to go through and learn before class, so that the class could be used for discussions and questions, not lectures. This is a much better approach for giving students a framework for actually learning content. Carl’s approach is more of a band-aid to lecturing than a real advance in pedagogy. (I should mention that the two approaches are not at odds nor are they distinct — these guys are colleagues and friends.)



  • 2. Mike  |  July 12, 2013 at 3:09 am

    I imagine that “college leaders” object to this in part because:

    1) It’s awfully specific, and therefore sets a precedent that politicians should step in and dictate exactly how colleges are run.

    2) It’s pretty draconian. Not doing this costs your college all it’s federal government funding? I’m assuming his actual proposal said something reasonable like “at least 90% of the faculty” so that the entire college won’t lose it’s funding if, say, a part-time instructor forgets to fill out the survey & then leaves,

    I wonder if it would have been more effective for Nobelist Weiman to lobby for a more general condition and a carrot (instead of a stick). Each college needs to “demonstrate ongoing improvement of teaching” (or something similar) and here’s X$ to do it with.

    • 3. Mark Guzdial  |  July 12, 2013 at 8:31 am

      I can buy that a carrot would go over better than a stick. But even if you provided the carrot, a scientist like Weiman would want to measure if the carrot was having an effect — which would get back to gathering the same data that the college leaders rejected.

      • 4. Mike  |  July 12, 2013 at 1:56 pm

        I definitely agree that measuring the effects of any changes is necessary to know if the changes are effective.

        At the community college where I teach we’ve gathering metrics for a long time; it’s now given (culturally speaking) that you should measure the effects of your actions. We certainly don’t have a problem with producing data for external consumption (and I’m willing to bet that all other colleges have to do this, too). We might not be producing exactly what Nobelist Weiman wants, but there is a significant amount of data available.

        It sounds like Weiman wanted a pretty specific survey done at all colleges. It almost sounds like he wanted all the colleges to provide him with data for a study he’s doing (or would like to do). I’m not surprised that the college leaders rejected this. Colleges already collecting (and sharing) data and the leaders’ jobs are about big picture issues (like getting money for the college), not procedural details.

        This sounds like something that would be better to pilot in a grant before deploying nationally. That would get him money to pay people to (voluntarily) do this, he’d generate some results proving that this change is effective (or not), and he could spend some time figuring out what data’s already being collected & how useful it is.

  • 5. holdenweb  |  July 12, 2013 at 9:32 am

    “Is it really impossible to get universities to take teaching seriously?” In my own limited experience, yes. In modern universities everything is reduced to money. Even thirty years ago being good at teaching was not regarded as significant or any kind of competitive advantage. Nowadays, the quality of education received by the students doesn’t appear on most academics’ radars at all.

  • 6. Peter Boothe  |  July 12, 2013 at 8:04 pm

    Metrics, when used by educators in an effort to improve, are a great thing! Historically, however, the forced introduction of metrics into the educational process has led to a loss of teacher autonomy.

    College professors see what has been done to high school teachers – they are now required to teach to the test! Even more, student reviews of courses were originally for teachers to get better, and are now used in the promotion and tenure process.

    Measuring is important, but until quality teaching is rewarded, measuring teaching can never be anything other than a stick. Even more, I can’t think of any teaching metric that doesn’t fall afoul of the principle that the more important a metric is treated, the more likely it is to be “gamed” and turned useless.

    • 7. Paul T. Corrigan  |  July 28, 2013 at 8:14 pm

      Hear, hear!

  • 8. David Karger  |  July 14, 2013 at 2:52 am

    I think the charitable interpretation of Tighman’s comment is “research and teaching should not be linked.” And with that I agree—a university should not be denied research funds on the grounds that it teaches poorly—no more than a research lab should, that doesn’t teach at all.

    But again, “research and teaching should not be linked.” It’s never been clear to me that the best researchers make the best teachers. I’ve always been told that the top universities *lose money* teaching students. So if those universities are teaching badly, perhaps those students should go elsewhere and the university should turn into graduate-degree-only institutions.

    How to make this happen? Well, if we can develop good ways of assessing learning, and make the results public, then self interested students will start steering away from institutions that do not teach them well—right? But I’d say that rather than surveying how universities teach, which is only a proxy for what we care about, we should instead be surveying how well students learn.

    How do we do that? Well, I can think of one group who are very interested in students learning well: their employers. What kind of evaluation could *employers* do that could yield meaningful public data about which universities are teaching well? And in particular, deal with the confounding effect of the quality of students entering (perhaps we can address that by also asking high-schools to measure something—they also have an interest in finding out what universities will teach best).

    Google is interested in (online) education, and also does a lot of hiring—I wonder what kind of evaluation of their employees by scheel they might be willing to share.

  • […] Guzdial, who pointed me to this article in his post Carl Wieman Finds Colleges Resist Measuring Teaching, […]

  • 10. Paul T. Corrigan  |  July 28, 2013 at 8:45 pm

    The article notes that “[Wieman] believes that without some kind of direct pressure, there’s little reason to believe that most professors will break out of their old, ineffective habits.” I may be as frustrated as he is. But I think that there is little reason to believe that “direct pressure” would accomplish the aims he hopes for.

    It seems that teaching is one of those areas in which you cannot make people do the right thing (i.e. implement effectively approaches to teaching that are known to be more effective). You cannot make colleges support better teaching. You cannot make teachers learn and implement better approaches to teaching. You cannot force graduate schools to better prepare their doctoral students, who are going to go on to teach in college, to teach well.

    You can, of course, force people to PRETEND to do the right thing, go through the motions, check the boxes, etc. With outside mandates, we can easily get what Ted Marchese called “widely observed rituals of compliance.” But that is not only unproductive but counterproductive, wasting time, resources, and will-power that otherwise could go into better teaching.

    So what to do?! I don’t know. But I think that individual profs. and small groups of profs. need to take the lead in reading and implementing the scholarship on teaching and learning. The more who do, the better.

    If some sort of “direct pressure” is used or needed, it will certainly need to be applied skillfully. They survey-or-else idea doesn’t seem like the way to go.

    Paul T. Corrigan
    Teaching & Learning in Higher Ed.

    • 11. Mark Guzdial  |  July 29, 2013 at 3:24 pm

      Paul, you may be more of a curmudgeon than I am! Your list of “You cannot’s” is longer than I’m willing to believe. For example: “You cannot make teachers learn and implement better approaches to teaching.” I actually have a good bit of evidence that GaComputes taught teachers how to implement better approaches to teaching. The problem with arguing what can’t happen is that people are pretty innovative. What couldn’t happen yesterday might happen tomorrow.

      I see that point that you and Peter and others are making, that we can’t force good teaching. However, that’s not what Carl was proposing doing — he was proposing measurement. The rest of the argument (that it would be meaningless, that it would be used to create metrics, that it would be linking research and teaching) is conjecture. It’s really hard to predict the future. You may certainly be right! My point is that you are rejecting Carl’s proposal based on what might have happened, when it’s certainly possible to have invented a scenario where none of those things would happen.

      Nonetheless, we won’t know now. I do hope that there will be more innovative solutions that come along to contradict some of the “cannot’s.” 🙂

      • 12. Paul T. Corrigan  |  July 29, 2013 at 3:41 pm

        I’m not usually called a curmudgeon (at least to my face). I don’t mean to be one. But I do see what you point out, that I put a lot of “can’t”s in my previous comment–or, at least, a good number of variations of the same “can’t” in regards to forcing good teaching.

        I actually think that a nationwide survey of college teaching practices would be good. Researchers have done similar studies. And I would be all in favor for increased federal funding for such research. But I think that’s very different from what I am disagreeing with. Arguing for a “survey-or-else” approach and talking about “direct pressure” as a means to improve teaching seems like a lot more than just “metrics.”

        To clarify, I absolutely believe that we can teach teachers to teach more effectively . . . if they want to. I also think that I agree with you on that point (though your link to GaComputes seems to be broken or their site is down, so I can’t be exactly sure what you mean). Moreover, I even think that we can even help teachers want to improve. (This is what Teaching & Learning in Higher Ed. is all about.) But “teaching” and “helping” and “inspiring” and “encouraging” teachers to improve are much different than “forcing” or “making” them.

        Perhaps we will also need to find ways to be subtly coercive (that’s probably the worse word to use for what I mean), i.e. ways to shift the playing field in ways that encourage teachers to improve without making them feel “forced.”

        Paul T. Corrigan
        Teaching & Learning in Higher Ed.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 9,052 other followers


Recent Posts

Blog Stats

  • 2,030,117 hits
July 2013

CS Teaching Tips

%d bloggers like this: