ROBOT !!! (Photo credit: m a c h i n t o y)
I was recently quoted in Devon Haynie’s U.S. News and World Report article on the topic of Automated Essay Scoring, and when I saw the finished article I realized that there’s more to say.
Ms. Haynie’s article was intended to briefly introduce students to the concept of AES, not to be a comprehensive statement on the state of the technology– much less the human and philosophical issues that surround it. There’s more to this issue that I stressed to her in our phone interview, and that I’d like to stress here.
Ted Curran, an education and technology blogger and instructional technologist for Pearson, believes the technology can be helpful to teachers and students alike if used correctly. In his mind, professors could use the technology to scan essays for areas of weakness, freeing professors to read the work more deeply with a focus on improving students’ critical thinking.
Whenever a technology comes along that could potentially replace human workers, it’s important to ask ourselves if it should. I stressed to Ms. Haynie that we shouldn’t look at robo-graders as “a way to do the same thing only cheaper”. We should be looking for ways to “do education better”, and this means building upon the unique strengths of human and machine graders.
Right now, even in higher education, even with human teachers, the vast majority of instruction measures just the lowest forms of thinking– recall and understanding. Higher order critical thinking skills (those slippery things) are labor-intensive to teach– they require a lot of the teacher’s attention focused on personalized feedback, revision, and discussion with each student. Our precious human graders are failing to provide this high level of support to students.
Firing human teachers and switching to robo-graders won’t magically solve this problem. AES systems are designed to watch the way a human grades a paper, notice the characteristics of papers that earn different grades (over hundreds of tries), and then learn to match new papers to the grades the human would assign them. It’s not thinking, it’s not evaluating, it’s mimicking. If it watches mediocre grading practices, then that’s what it reproduces.
However, what it is doing is handling the boring, tedious, repetitive part of grading that numbs teachers’ minds. You don’t need to pay a Ph.D professor to catch students’ grammar and mechanics mistakes, but computers are great at that. What you pay the Ph.D for is to develop students’ thinking, spot misconceptions, and challenge the student to think critically in ways that only a highly trained human being can.
“Though I don’t know that it’s realistic,” he says. “My fear is that people are just going to want to feed everything through it and fire all of the teachers.”
Internet technology has replaced millions of workers in a variety of industries. Travel agents, bank tellers, telephone receptionists, bookstores, record shops– the list goes on. All of these middle-class jobs are being destroyed in the name of efficiency and lower prices with little thought for what else we’re losing in the process.
In all these industries, some administrator made the decision that machines could do the same work as humans, and do it more cheaply. If you’ve ever been trapped in a phone answering tree when you just needed to speak to a person who could understand your problem and help you find the right answer, you know how irreplaceable humans really are.
When I gave Ms. Haynie this quote above, I was expressing fear that school administrators would fail to understand the true value of human graders, and make a simplistic decision to switch based solely on the economic bottom line. I know that most human graders are not currently giving students the quality of feedback they need, and I can see frustrated administrators saying “why should I pay you to give bad feedback when a machine could do it at a fraction of the cost?”
What needs to be said– over and over again, every time the idea of robo-graders comes up– is that we need a hybrid approach, where humans and machines both do what they’re best at so we can raise the overall quality of feedback that students receive. Give students lightning-fast feedback on their writing (even as they’re writing it!) so they can improve the overall level of their work. Give professors semi-graded papers where they can focus only on students’ mastery of learning outcomes instead of being distracted and demoralized by careless student mistakes. Don’t let this decision be only about dollars– it needs to be about improving the quality of student achievement across the board.
Related articles
Liked this post? Follow this blog to get more.
