Student Course Evaluations Can’t Measure Teacher’s Knowledge: But We Could

It’s that time of year when Deans and Chairs start prodding students and teachers about course evaluations. What do the students think about their courses? What do the students think about their teachers?

There is a significant body of evidence that suggests that course evaluations are a stable measure about the teachers themselves. For example, the scores for a teacher are consistent across instantiations of the course over time (see Nira Hartiva’s work). However, they still might not be measuring something that we consider significant about teaching.

For example, it’s a mistake to think that student course evaluations tell us what a teacher knows about teaching. The teacher’s pedagogical content knowledge is invisible to the student. The student only sees what the teacher has decided to do to interact with the students. The student can’t see the knowledge that the teacher used in making that choice.

Three kinds of knowledge that are particularly relevant to a CS teacher are:

• Knowledge about how to teach. A good teacher knows more than one way to teach a particular subject, and knows to change methods for a given student or to change the pace of a class. When I see students driving away in the back of my class, I know that it’s time for a Peer Instruction activity.
• Knowledge about misconceptions. As was shown in Phil Sadler’s exceptional study (see blog post), a characteristic of teacher expertise is knowledge about what students typically get wrong. Based on that knowledge, teachers can address those misconceptions, and lead students to discover and correct the misconceptions themselves.
• Knowledge about how to broaden participation in computing, which is particularly relevant to CS teachers. These include how to teach avoiding stereotype threat and triggering the imposter phenomenon, about how to give everyone a voice in the class and not let the loud and boisterous define the teacher’s perceptions of the course. I can offer a negative example, taken from real life but might have been invented after reading the negative examples in Unlocking the Clubhouse.

Teacher: How many of you students had Python in a previous class?
(Most students raise their hands, since it’s the language used in the pre-requisite class.)
Teacher: Well, you learned a terrible language. You’ll have to forget everything you know if you want to pass this class.
(Every student suffering imposter syndrome at this point decides to drop.)

This teacher actually has quite high course evaluation scores — and double the drop rate of every other teacher of that class.

Pedagogical content knowledge (PCK) is the key difference between novice and expert teachers, but is invisible to students. This is another reason why student evaluations of teaching (aka, Student Reviews of Instruction (SRI)) are inadequate as measures of teaching quality. They can’t measure a key indicator of teacher expertise.

I’ve been wondering how post-secondary teaching might change if we were to take a PCK perspective seriously. The knowledge of good teaching is definable and measurable.

• We might define courses not just in terms of learning objectives but in terms of what knowledge the teacher should have to teach the class effectively.
• We could evaluate University and College teachers based on their PCK — literally, taking a test on what they know about teaching the class.
• PCK tests would finally create an impetus for University and College faculty to pursue professional development — that’s where they’d learn the teaching methods, student misconceptions, and methods for encouraging BPC that they need to answer the PCK tests. One might even imagine teachers taking a class on how to teach a new class that they’ll be offering in the future — preparing for a course by developing expertise in teaching that course. What an interesting thought that is, that higher education faculty might study how to teach.