Reading student writing: The value of what can’t be automated

April 23, 2013 at 1:24 am 7 comments

I really liked this post, in part because of how differently it is being interpreted within my department.  I posted it on a school-wide discussion list, to emphasize the value of what we do that cannot be automated.  However, my MOOC-favoring colleagues read this post in exactly the opposite way to how I interpreted it. “Anyone can do this kind of grading, so we shouldn’t waste our time at it! Instead, we should abandon all courses that require this kind of grading.”  What can’t be automated isn’t worth doing?

I know that a lot of MOOC-proponents are pushing automatic grading of papers as a cost-effective way to handle classes with over 1000 students.  Quite frankly, the idea appalls me—I can’t see any way that computer programs could provide anything like useful feedback to students on any sort of writing above the 1st-grade level.  Even spelling checkers (which I insist on students using) do a terrible job, and what passes for grammar checking is ludicrous nonsense.  And spelling and grammar are just the minor surface problems, where the computer has some hope of providing non-negative advice.  But the feedback I’m providing covers lots of other things like the structure of the document, audience assessment, ordering of ideas, flow of sentences within a paragraph, proper topic sentences, design of graphical representation of data, feedback on citations, even suggestions on experiments to try—none of which would be remotely feasible with the very best of artificial intelligence available in the next 10 years.

via Reading student writing | Gas station without pumps.

Entry filed under: Uncategorized. Tags: , , .

Universities should stand by core values NSF TUES is Cancelled: Where will CS Ed funding come from?

7 Comments Add your own

  • 1. Errol THompson  |  April 23, 2013 at 5:27 am

    Interesting reaction. Does that mean that those who see automated grading as the solution do not see us as being able to learn from what students submit?

    I lean heavily toward things that can not be marked by automated tests. My reasoning is:

    1) automated assessment tends to focus on giving a correct answer and not on the variety of possible answers. This leads to students focussing on what they perceive as correct rather than on creative thinking and the development of their potential.

    2) like the author of the piece you quote, I see my feedback on improvements to program code as being more valuable than the mark. A student may get close to full marks but I will still provide feedback that I hope will help them develop their skills as a programmer.

    3) automated assessment removes some of the master / apprentice relationship that I believe is crucial to the development of a master programmer. I see automated assessment as doing away with the valuable interaction that allows our students to enter the community of software developers.

    As I am becoming less and less happy with the emphasis on grades and grading, maybe, I no longer belong in a formal education context and should consider other alternatives.

  • 2. Kathi Fisler  |  April 23, 2013 at 7:09 am

    CS has the unfortunate trait that many of our assignments _can_ be graded automatically, which discourages us from taking the time to critique work more deeply. Code reviews are our form of grading “writing” below surface level. How many of us (still) grade code manually, deducting points for style, in large classes?

    Much as I value (and still try to provide) that kind of feedback in
    intro-level courses, I’m finding it increasingly hard: with 200+
    students and grad student TAs who never had their own code critiqued, it isn’t working. Curious to hear what techniques others use for giving students feedback on code beyond its behavior in large classes.

    I’m toying with the idea of training up a bunch of our strongest
    undergrads (who’ve been through my course) to run code critiques, then requiring each intro-class student to have a small number of live code critiques during the course. Anyone have experience with a model like this in a large class?


    • 3. gasstationwithoutpumps  |  April 23, 2013 at 10:18 am

      I have found it impossible to give meaningful feedback in classes of more than 40 students and difficult in classes of more than 20. I think that the question should not be “how do we teach better in megalecture classes?” but “why the f*** are we doing megalecture classes when we know that we can’t provide the most important part of teaching in them?”

      • 4. Errol THompson  |  April 24, 2013 at 7:19 am

        I have now taught a classes of over 100 for a couple of years although this year it has reduced to 50. The key to being able to provide feedback is to have a good team of helpers. I am also keen to have a tool that focuses on providing standard feedback rather than on marking but I need some time in my busy schedule to develop it.

    • 5. Errol THompson  |  April 24, 2013 at 7:17 am


      I always try to provide feedback on how the code might be improved. Novice programmers tend to apply solutions that they have already seen to programming problems. The result can be very messy code. Providing feedback on how it might be improved gets the better students thinking.

      For less able students, I tend to focus more on what they need to correct in their code to make it work.

  • 6. alanone1  |  April 23, 2013 at 7:19 am

    I couldn’t agree more with “gas” on this one. The desire for “fake scaling” (going larger only by providing less than is needed) in universities has always been a problem (in my lifetime anyway). It has now ballooned with the baby boom and now the “jobs require a college degree” boom into the something that is hard to distinguish from any pop culture “designer jeans” substitute for the real thing,

  • 7. Academic-Zone  |  May 1, 2013 at 9:21 am

    I think the best approach is to combine the use of both automated and teacher grading.Let the technology handle the areas it can grade and let the teacher cover the rest. Technology offers so many opportunities to enhance education. There is nothing wrong with traditional learning and grading, but with technology so readily available, we might as well use it to our advantage.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Trackback this post  |  Subscribe to the comments via RSS Feed

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 9,045 other followers


Recent Posts

Blog Stats

  • 2,025,046 hits
April 2013

CS Teaching Tips

%d bloggers like this: