Tuesday, January 12, 2010

It "couldn't cope with metaphor..."

excerpted from the AWP's The Writer's Chronicle:

Computerized Exam Markers Fail Hemingway, Churchill, Golding

Some of the world’s most well known writers have received failing marks when submitted to a new computerized marking system for British school essays, the Times Online reports. Winston Churchill’s 1940 speech exhorting his countrymen to “fight on the beaches” had a style that was too repetitive according to the computer. The speech was rated below average. William Golding and Ernest Hemingway came up short as well, ranking less than standard in the American equivalent of an A-level English exam. A passage from Golding’s Lord of the Flies was docked for its two-word paragraph: “A face.” Graham Herbert, deputy head of the Chartered Institute of Educational Assessors (CIEA), an umbrella body for exam boards and other organizations, said: “The computer was limited in its scope. It couldn’t cope with metaphor and didn’t understand the purpose of the speech. We also tried a passage from Hemingway. It couldn’t understand the fact that he had a very spartan style and (it) said he should write with more care and detail. He was also rated less than average.” This system, already in use in the United States, was created using a range of comments by human graders in response to exam papers. While the program recognizes sentence structure, other elements such as style and purpose are not recognized. According to Herbert, some children in America had “cracked the code” by learning to write in a style that the computer understood. This was called “schmoozing the computer,” he said. “At the moment we do not have a reliable and valid way of assessing English language using a software package, although this is something for which there is demand.”


  1. But why? Why is there demand? Dreadful.

  2. Because administrators are obsessed with finding a so-called "objective" assessment tool by which to measure success in the humanities and arts. So they're willing to kill original thinking. A two-word paragraph? Bah.

    They want a box around the final answer, as with the algebra homework. Sigh. Ask me why I left teaching.

  3. Oh yeah, I get constant citations from the grammar police on Word. Kinda fun.

  4. Because I could not cope with metaphor,
    it kindly stopped for me?

    They just want to delete the human aspect of assessment and teaching, and learning. People keep insisting it's a hard science, where really learning is all flesh and bone and blood.

  5. Ditto what Desiree said.

    Objective and writing never go together, and so many, many, many institutions refuse to accept that.

  6. My heart breaks for future generations.

    For a second.

    Then I think, no, they'll beat this.

    It probably comes down to money. There's hope if someone brilliant comes up with a way to do away with money altogether.

  7. Innovation, by definition, is about *breaking* the rules. If you elevate the rules to the point of absolute authority, you will quash innovation and raise a generation of yes-men and yes-women.


  8. You should be teaching. Your insights into the process are valuable.

    In the plastic arts "breaking the rules" is never questioned. It's a modernist notion that fine artist' carry with them daily. Because I earn my bread doing public art, my fellows often repeat the mantra "I could never work with people telling me what to do." This cloak of freedom they THINK they're under is arguable. I say every time an artist puts pencil to paper they have to contend with what they can't do under the mantra "it's been done before."

    I feel like we should be sitting down to a couple of mugs of spiked coffee

    (this time I'm saving this comment on glowing desktop

  9. PA, That coffee break sounds good. And, yeah, the tenets of modernism apparently never made it to the social sciences, where, if you can't label it, you have no use for it.

  10. I've been working on a program whereby students could write essays in the form of multiple-choice tests. I haven't made much progress.

  11. Linda!!!! I LOVE YOUR COLLECTION!!!! I'm dumb at saying why. You make me see in ways I've not seen. You make things rich and new.
    I write declarative sentences. I miss ya. Wow am I happy for you. I remember you liked my rainbow poem. It's snowing here. Julie sends a great big hi. Me, too.
    Jack (Ridl)

  12. Thanks, Jack! Coming from you, that means a lot. I miss you guys, too. I think of you often, and your dogs, and your house with the awesome floors. I don't miss the snow. ;-)

  13. Jim and I had such a great time Saturday night. Thanks to you and your husband from both of us!

    I am really enjoying your poetry and it was especially meaningful to hear you talk about your inspiration for several of the poems. I love the fact that you write about art and your word choices are so visual.