Audrey Watters (who is one smart person) has a post on the University of Akron’s robot essay grader paper. Tossing Sabots into the Automated Essay Grading Machine
It provides a good overview of the issue from both sides.
My personal hightlights are:
Comment IV
This is an example of an absurd answer that recieve 6/10.
It opens with the statement
“In today’s society, college is ambiguous. We need it to live, but we also need it to love. Moreover, without college most of the world’s learning would be egregious.”
Check out the automated feedback, which includes this statement
“* Uses grammar and mechanics correctly (virtually free of errors) and demonstrates understanding of correct usage”
Comment VII
“Robots can give a grade. That grade, according to the study by Shermis and Hamner, approximates that of a human. But a human can offer so much more than just a grade. And without feedback – substantive, smart, caring, thoughtful, difficult feedback – what’s the point?”
Comment VIII
“What do we want from students’ essay writing? Why do we assign them? What are we assessing? Critical thinking? Grammar? Persuasiveness? Vocabulary? Content knowledge? Comma placement?”
http://www.hackeducation.com/2012/04/15/robot-essay-graders/