AI are machines that can learn, reason, and act for themselves. You’ve probably come across this before: Maybe you’ve called a business and a computer asks you questions instead of a human. Apple’s Siri is another example. Or Google Assistant.
These AIs are super helpful: you don’t have to get up from the couch to change your favourite tunes. Just tell your AI that you want to listen to newest hits and, voilà, it starts playing! But what if it gets a bit more complicated than selecting songs?
AI in English language testing
We know you don’t do an English language test for fun. Sometimes a University needs evidence of your English language proficiency before you can enrol. Immigration departments in Australia, New Zealand, the UK, the US and Canada also need you to prove your English language skills before you can get a visa.
So, when your future depends on an English language test, you want a test that is fair, accurate, and accepted by your institution.
English language tests with AI marking are often promoted as fairer. That’s because they say machine marking treat every test taker the same, regardless of where they come from or what they look like.
But, it’s not fairer. Researchers from MIT found that AI is biased. When training an AI –for example to mark English language exams– there are ways that bias can occur.
Bias in machine marking of English tests
AI’s are trained through deep-learning: A machine learns from large amounts of data. If the data contains more spoken English samples from, say, European candidates, the resulting marking system would be worse at recognizing spoken English from Indian, Chinese or Latin American speakers. There are even examples of native speakers failing the AI marking system. Even native English teachers can fail. The machine just wasn’t built for recognising these native speakers. Human examiners can understand different accents, but computers clearly can’t.
More bias, or unfairness, sneaks in when machines are judging English language, but the AI is not advanced enough. It can’t always assess complicated answers, jokes, or slang words used in one English speaking country but not others. A human examiner has been trained to think about the social context. They know the appropriate use of a word. Machines don’t – and you won’t want to lose marks as a result of this.
The MIT research also found that it is possible to introduce bias during what they call the “data preparation stage”. This happens when people select which points of your language they want the machine to consider when marking you. So, when they tell a machine that pauses in your answer are bad, you get marked down – even when some pauses are completely acceptable like when you are thinking about the best way to respond or using a pause to emphasise a point. A machine might ‘time out’ when it thinks you’ve finished your answer. At some tests, this happens as quickly as 3 seconds of silence.
Humans vs Machines: who do you want to check your high-stakes English test?
AI is useful, but machines still make mistakes. Researchers from Swiss Federal Institute of Technology Lausanne found machines to mistake a joystick for a Chihuahua, for example, or a coffee-maker for a cobra. Yet the human brain would have no problem correctly identifying the objects.
We’ve already covered some points where machines marking your English test can be unfair: they can’t always recognise different accents. Machines also don’t know jokes or slang words.
Human examiners are language experts. They have years of experience working with students of different levels of English, and with people coming from all over the world. This means they’ll understand your accent and won’t penalise you for it.
Human language experts are trained to assess your English language skills. Apart from your grammar and spelling in writing or pronunciation when speaking, the way you present your ideas is also assessed. We call this cohesion. Cohesion is when you successfully present your ideas and arguments related to the given topic in a related manner. So, your points must be related to the topic and be convincing with supporting examples. Humans will understand how your examples relate to a question, but machines can’t always do this.
Finally, humans can understand emotions. When your future depends on an English language test, it can be stressful. Humans are able to understand when you talk a bit quieter because you are nervous (and won’t fail you for it). In fact, human examiners can help you feel at ease.
Take a test that’s fair
While English language tests with AI marking are often promoted as fairer, researchers from MIT found that AI is biased. When you need an English test for your study, work or migration, you want a test that you know. A test that is fair. A test that is open and transparent about the marking criteria. And a test that supports you.
The IELTS Writing test is marked by humans. Not once, but twice. Your IELTS Speaking test is done face-to-face with a real, human examiner. This way you know that there is no machine bias.