Proposal #2 to Change CS Education to Reduce Inequity: Make the highest grades achievable by all students

July 27, 2020 at 7:00 am 13 comments

What does an “A” mean in your course? The answer likely depends on why you teach. Research on teacher beliefs suggests that grading practices are related to teachers’ reasons for teaching. Joe Feldman points out that our grading relates to who we are as teachers, and it is passionately held:

Conversations about grading weren’t like conversations about classroom management or assessment design, which teachers approached with openness and in deference to research. Instead, teachers talked about grading in a language of morals about the “real world” and beliefs about students; grading seemed to tap directly into the deepest sense of who teachers were in their classroom.

Feldman, Joe. Grading for Equity (pp. xix-xx). SAGE Publications. Kindle Edition.

I’ve used the Teacher Perspectives Inventory (TPI) (see link here) with dozens of CS teachers. The most common teaching perspective I see among CS teachers is “Apprenticeship.” CS faculty see themselves as preparing future software developers. They value demonstrating and modeling good software practices. An “A” for an apprenticeship teacher is likely to indicate that the student produces good code. An “A” is reserved for “excellence” (as one CS teacher told me recently). An “A” indicates that the student has risen above his or peers in producing “high quality products” (as another CS teacher posted on Facebook recently). An “A” means that, in this teacher’s opinion, the student is recommended to go on to a highly-desired software development job, perhaps at a place like Google or Amazon.

A student with less computing background is much less likely to earn an “A” in an Apprenticeship-oriented class. If you bring more experience to the table, you have a head start on producing higher-quality products than other students in the class. Their products are more likely to be marked as “excellent.”

In my opinion, the teacher attitude of “rugged individualism” defined in SIGCSE 2020 paper by Hovey, Lehmann, and Riggers-PiehlLinking faculty attitudes to pedagogical choicesmeshes with the TPI category of “Apprenticeship.” “Rugged individualism” teachers believe that “learning and success are the individual student’s responsibility.” (Hovey et al did not make this claim or compute the correlation — this is my prediction.) They showed that teachers who believe in “rugged individualism” are less likely to use student-centered teaching practices and more likely to lecture. Students with less computing background do better with student-centered teaching practices.

This is my third post in a series about how we have to change how we teach computing to reduce inequity (see last post). The series has several inspirations, but the concrete one that I want to reference back to each week is the statement from the University of Maryland’s CS department:

Creating a task force within the Education Committee for a full review of the computer science curriculum to ensure that classes are structured such that students starting out with less computing background can succeed, as well as reorienting the department teaching culture towards a growth mindset

The style of grading that means to identify “talent” or “excellence” is inherently inequitable. It presumes a fixed mindset. If you believe that there is a random distribution of “talent” or “ability” or “Geek Gene” in the course, and (critically), there’s nothing much that teaching can do to change that, then it makes sense to grade to the curve. There can be only a few “A” slots, more “B” slots, and so on. Empirical evidence suggests the opposite — good teaching can trump a lot of other factors. Belief in a growth mindset leads to better learning outcomes and better performance. If we value teaching, and believe that students can get better at computer science, then over time, we should teach better and students should learn more. If students learn more, they should get a higher grade It’s not a fixed-result game.

Measure learning or progress towards objectives, not code quality

My teacher perspective is that we are in the job of maximizing individual human development. It’s our job to help each student achieve as much as they can within our discipline. We should measure achievement in terms of learning, not product quality. I tend to align with the “nurturing” perspective on the TPI, but that’s not what I’m going to argue for here.

It is not at all the same thing to grade for excellence and to grade for learning. Writing code isn’t the same as learning. We have evidence that writing a given piece of code is not indicative of understanding that piece of code. The recent ITiCSE 2020 paper by Jean Sala and Diana Franklin showed that use of a given code construct was not correlated well with understanding of that code construct (see paper here).

I mentioned in the last post that I’m reading Grading for Equity by Joe Feldman (see his website here). He points out all the other factors that influence grades that have nothing to do with learning. Some of these extra-curricular factors — like the ability to meet deadlines that presume privileged, full-time student status without outside pressures — are less likely for Black, Hispanic, poorer, or first-generation college students. These factors influence the production of high-quality code even more than they influence learning. These pressures are going to be even greater during a time of a pandemic.

We should give grades depending how much is learned in a given course. That’s hard to do without measuring students as they enter the course and as they leave the course, and grading on the delta. There is a movement suggesting that “labor-based grading” leads to more compassionate and equitable grading (see article here). I’m not arguing for that.

State your criteria clearly and be objective in grading

I’m arguing that we aim for a Teaching Perspective most closely aligned with the one called “Transmission.” The goal of a Transmission teacher is for students to learn what is needed for the next course or to meet the course objectives. Assessments of understanding should be as objective as possible. Grades should represent achievement of the learning objectives and nothing else.

There is a lot in any given CS course that is not about preparing students for the next course in the sequence. We cover a lot of material. In some courses, we’re preparing students for the imagined technical interview with Google or Amazon. That’s not fair to require understanding and performance on standards that go beyond the course objectives.

I recommend setting the objectives clearly, announcing them on the first day, and grading to those. It’s okay to aim for the targeted average on each assignment to be low but passing (e.g., a “C”), as long as you’re clear and fair. In my classes, I rely heavily on weekly quizzes because those are more likely to lead to learning (see post here). I give points for writing code, but that’s to encourage the activity, not to make-or-break the students’ grade. Programming is for learning, and the quizzes, midterm, and final exam are assessments.

Go ahead and bore your best students

Students with a lot of computing background get an easy “A” in my courses. That’s fine. I expect that. I explicitly tell my students that I teach to the bottom third of the course. I want to move B and C students up into A’s and B’s. I give out a lot of A’s. In years past, I did a series of blog posts on “Boredom vs Failure” (here’s the first post in the series, and here’s the last one). The question is: which is worse, to bore and give easy A’s to the most privileged and most prepared students, or to fail (or discourage to the point that they drop) the students with less privilege and the least computing background? Think about the students who might fail or drop out in a system that makes sure that the most well prepared students are challenged. Each one of those students who continues on does more to change the status quo than does keeping the more privileged students from getting bored. Helping the students with less computing background succeed makes a much bigger difference for society long-term than does keeping entertained the most privileged students.

One response to this proposal is that I’m degrading the value of past A’s. The A’s don’t mean the same thing anymore. That’s true. I take a historical perspective. Those A’s were earned when the students with less computing background were not being taught with methods that helped them succeed (Proposal #1). Those A’s were earned in unfair competition, where the students with prior computing background were compared to students with less computing background. I’m proposing a more just system where the students with less computing background have a chance at the highest grades, where they’re taught in ways that meet their needs, and where their teachers believe that they can grow and improve. I’m not particularly concerned about preserving the past glories of those who won in an unjust system.

If we pre-allocate, ration, or otherwise curve “down” grades so that the top scores are a scarce resource in a competitive system, we are privileging the most prepared students and disadvantaging the least prepared students. I am proposing differentiated instruction. Teach explicitly for the least-prepared students. You will likely have to give up on pushing your top students to greater excellence — that’s the kind of privilege which we have to be willing to surrender. Aim to help every student achieve their potential, and if you have to make a choice, make choices in favor of the students with less privilege and less computing background.

Entry filed under: Uncategorized. Tags: , , , .

Proposal #1 to Change CS Education to Reduce Inequity: Teach computer science to advantage the students with less computing background Proposal #3 to Change CS Education to Reduce Inequity: Call a truce on academic misconduct cases for programming assignments

13 Comments Add your own

  • 1. alanone1  |  July 27, 2020 at 8:01 am

    Well, what if this were music? What should we be doing?

    If we are interested in “learners first”, then we have to ask “What is the point in a public grade for music?” (It’s an art form.) What is the point in evaluating other than the student’s playing (i.e. multiple choice tests etc), and doing that evaluation other than for the student’s help? Students need to have some sense of where they are are how they are doing, and whether they are on “fruitful pathways”, etc.

    If the teacher doesn’t have a very good sense about where each of the learners are, then there is something terribly wrong with the educational setup. Maybe there aren’t enough teachers for the number of learners, etc. Maybe the teachers don’t really care, and are not really teachers at all. Etc.

    My main beliefs are in the paragraphs above.

    However, here is a secondary scheme I encountered as a student that was also helpful. I was in a biology program that was coextensive with pre-med (and this was highly rated by med schools — we got lots of super smart kids who were trying to get into good med schools, etc.).

    Grading on the curve would have been even more ridiculous than it is anyway. The biology school said: “We have absolute standards here, and everyone can get an A — you just have to get to 93 and above”. This worked extremely well for many reasons, not the least of which is that the students were not competing at all and there were many processes for the students helping each other learn as much as possible so all could also get good grades.

    The good med schools were not selling MDs for money but had also had absolute standards for who should be a doctor and who shouldn’t. (Their MDs were like getting a 93 in med school.) The prep at the college I went to really helped the kids be better learners in med school.

    I feel clear about the above two schemes. I feel less clear about what is called “CS”, especially at the undergrad level. And I don’t have a good sense now as to when in college remediation should not be the main goal — maybe we’ve gotten to the place where “college is to make up for the bad high school and other experiences”, and that is what it is for now. I think this is in essence what Mark is suggesting when he says: “Helping the students with less computing background succeed makes a much bigger difference for society long-term than does keeping entertained the most privileged students.”

    Maybe so, but I prefer that my doctor have either experience 1 or 2 above.

    One way to fix this for a while — that I’ve advocated over the years — is to not try to fix the “computer science” problem, but to just set up a different field — let’s call it “Systems Science” — that is actually serious about its content and its standards, and doesn’t worry about what industry is doing (except to try to improve it).

    I say “for a while” because guess what will happen if it starts to seem like a path to a job? (Hint: the good stuff is not a “job”, it’s a “calling” …). Just keeping these very different perspectives separated out would help everyone.

    Reply
  • 2. Steve  |  July 27, 2020 at 10:13 am

    Your post brings up two things that I’ve thought about for a while related to equity, and while they’re not the main focus of your post (grading), I think they are worth addressing in the context of this blog post.

    First, you mention performance issues “that presume privileged, full-time student status without outside pressures.” This problem is huge at institutions like mine (as opposed to other places, like the places I went as a student, where most students are devoted full-time to being students). Real life issues intrude a lot. Where this particularly bothers me: The AAC&U has done a lot on “high impact experiences” and how these are particularly advantageous to students who struggle academically or have not had many opportunities in the past. So something like research experiences, extra-curricular activities, etc. are great because they bring students into a feeling of community and involvement, but these experiences are most difficult for the students that would benefit most from them. How can we address that? I’ve been involved in a few BPC initiatives that have been successful at getting students involved with stipends, but when the funding runs out then it’s back to business as usual. Asking these students to go above-and-beyond their required classwork, with no incentive other than “it’s good for you,” doesn’t work when they are struggling with just their classwork. And funding will always run out…

    Second, you talk about “Boredom vs Failure.” If you could personalize material for each individual student, then this wouldn’t be such an issue, but that’s obviously impractical. A first-order approximation of this could be something like the different flavors of the intro course at Harvey Mudd, or extending that into the curriculum where it seems like it would take the form of an honors program (in name or in practice). This accommodates more students at a level that is appropriate to them. In one way this “better fit for all students” seems to promote equity, but two distinct tracks — two different “qualities” if you will — is the height of inequity.

    I don’t think there are any good answers here. In addition to helping my struggling students, I really am concerned about students being bored. These deserve an education that challenges them and gives them good learning experiences, rather than just having them go through the motions and getting easy A’s “because they can.” But when I think about solutions to this, it’s hard not to feel like I’m giving different levels of opportunities to different students, reinforcing the inequities they start with (the “rich get richer”….).

    Reply
  • 3. kirkpams  |  July 27, 2020 at 11:07 am

    The link to the TPI doesn’t work. The page doesn’t render with an HTTPS link, but it works fine with HTTP.

    Reply
  • 5. cognitionmachine  |  July 27, 2020 at 12:06 pm

    I’m all in favor of dedicating relatively more education system resources to address inequity. But are you proposing that we maintain the status quo of all these students sitting in the same classroom? If you are explicitly planning for students with a stronger background to sit around bored, couldn’t they at least do some kind of independent study? If that leads to greater inequity, that’s quite an indictment of the education system, and worth knowing about.

    Reply
  • 7. emon  |  July 27, 2020 at 1:50 pm

    Second, you talk about “Boredom vs Failure.” If you could personalize material for each tech to the point zone individual student, then this wouldn’t be such an issue, but that’s obviously impractical. A first-order approximation of this could be something like the different flavors of the intro course at Harvey Mudd, or extending that into the curriculum where it seems like it would take the form of an honors program (in name or in practice). This accommodates more students at a level that is appropriate to them. In one way this “better fit for all students” seems to promote equity, but two distinct tracks — two different “qualities” if you will — is the height of inequity

    Reply
  • […] Some students cheat because they think that they have to. “If I don’t cheat and everyone else does, I’m at a disadvantage.” That’s only true if student grades are comparative. That’s why Proposal #2 is a critical step for Proposal #3 — stop pre-allocating, curving, or rationing grades. Use grades to reward learning, not “rising … […]

    Reply
  • 9. David Karger  |  July 30, 2020 at 3:56 pm

    I’m thinking about this post in light of my experience at MIT. I do think that an A grade should indicate mastery, not just effort or progress. But I also believe that almost all of our students are brilliant and are capable of mastering the material in any of our classes. The only variable is in how much effort they have to invest to do so. Unfortunately, if we try to increase the A’s by making courses easier, students respond by taking more courses, so they have less time per course, so they don’t achieve mastery or get more A’s.

    I think this is a consequence of the “all you can eat” model which leads student to think can get more learning for the same price by taking more courses. Which is false of course. I’d like to see us move to a model where you pay a flat fee for a full college tuition, regardless of how long it takes. This might permit those with less background to spend more time acquiring it.

    Reply
    • 10. Mark Guzdial  |  July 30, 2020 at 5:15 pm

      Do students ever get less than an A in courses at MIT, David? Do they ever fail? From your message, I’m guessing that your explanation for any grade less than an A is that the student simply didn’t invest enough effort.

      Reply
      • 11. David Karger  |  July 30, 2020 at 5:37 pm

        To first order, yes, I believe that is the case. But not in the dismissive, students are lazy sense. Rather, they are workaholic and take on too much, such that they are unable to achieve to their full potential on any of it. And for some of them, even the “normal” load of 4 courses may be too much to let them achieve mastery in any. I don’t think that we should penalize such students.

        Reply
        • 12. Mark Guzdial  |  July 30, 2020 at 6:34 pm

          No, I wasn’t implying that any students are lazy. It does seem sad, though. An MIT degree is worth a lot. If students have to be declared “brilliant” before they walk in the door, then only the most-privileged, most well-prepared get access to that degree. I’m guessing that the definition of “brilliant” is going to be fairly traditional — fairly white/Euro-centric.

          It also seems sad to be a teacher there. If it’s only student effort that maters, then the teacher doesn’t. Not the curriculum, not the pedagogy in the classroom, not whether you use evidenced-based methods. There’s no value-added there as a teacher. It’s just about setting up hoops and puzzles and filters for these brilliant students to test themselves against.

          You don’t paint an inviting picture of MIT, David. I would like to believe that an MIT education could be about helping a diverse range of students could get access to the enormous opportunities that MIT can offer.

          Reply
          • 13. David Karger  |  July 30, 2020 at 6:45 pm

            Let me try to be clearer. It’s not only student effort that matters, because a teacher can make it easier or harder to learn. A good teacher will make it easier for most students, and a bad teacher will make it harder (though of course not uniformly, due to different learning styles). I certainly get the fulfillment of feeling like my teaching has made the learning easier for everyone. So not sad.

            More students than I would like do seem to find the teachers redundant. They take classes but we only see them when they pass through to drop off their homework. Probably because they are enrolled in another class meeting at the same time (that’s how some students manage to complete 8 or 9 courses in a semester). They are basically engaged in a self-study course, using the faculty only for assessment/accrediation (hoops and filters).

            We can only teach the students we admit. I’m sure you’ve heard the aphorism that the hardest part of [your favorite elite college] is getting in, and our job as teachers is simply not to damage the talent we get. It is clear that the elite colleges admit only a very narrow segment of the population deemed “most likely to succeed.” I’ve written emails suggesting that we should instead be admitting students for whom the admission would most significantly increase welfare in our society, but haven’t gotten traction.

            Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed


Enter your email address to follow this blog and receive notifications of new posts by email.

Join 7,966 other followers

Feeds

Recent Posts

Blog Stats

  • 1,783,595 hits
July 2020
M T W T F S S
 12345
6789101112
13141516171819
20212223242526
2728293031  

CS Teaching Tips


%d bloggers like this: