This isn’t the first account of bias in artificial intelligence this year, with researchers from MIT reporting it was possible to introduce bias into AI during the data preparation stage. What do these findings mean for English language testing, where machines are used for marking purposes?
Chances are you’ve used AI to make your life easier at some stage recently. Opened the Uber app on your phone? Your estimated fare and arrival time are based on machine learning. Friendly with Alexa or Siri? Your voice activated-assistants can do more than tell the time, they can now debate the meaning of life – all thanks to AI. But where do we draw the line when it comes AI marking high-stakes English testing?
The world’s most trusted English test
IELTS is designed to assess the language ability of people who need to study or work where English is the main language. As the world’s most trusted test of English, IELTS is recognised by more than 11,000 universities and employers in Australia, Canada, New Zealand and the United States. It is also recognised by professional bodies, immigration authorities and government agencies.
But what makes IELTS the most recognised and trusted English test worldwide? The answer: the method in which IELTS is conducted and marked ensures this quality.
Speak with a human, not a computer
Unlike other high-stakes English language tests, the speaking component of IELTS is conducted face-to-face with an IELTS examiner – not a machine. This approach makes it fair to test takers, as it prompts a more lifelike interaction and examiners can recognise different accents and pitch of voice. IELTS test takers also benefit from taking their Speaking test in a private room, allowing them to perform at their best, free from distraction. Human examiners can also ask a broader range of questions and accept a broader range of responses to exam questions because of the breadth of their knowledge of English. Machines have to be trained in their knowledge of English and therefore for them to be fair, they must have every possible question type and every possible accurate response entered into them (not only the key words) in order to accurately mark a person’s English.
In a story that went viral in 2017, a native English speaker with two degrees failed the speaking test of another test provider who uses machine-based marking. The test taker, who was seeking permanent residency in Australia at the time, should have received an accurate and fair speaking result – however, AI failed to understand her accent.
Ensuring consistent, fair assessment
The IELTS Writing and Speaking tests are marked by trained and certified examiners, who are qualified English language specialists. IELTS markers (not machines) work to clearly defined criteria and are subject to real-time monitoring and quality control procedures to ensure consistent and fair assessment for all test takers.
Writing tests which are anonymised before examiners see them are marked by a minimum of two examiners, and in some instances, tests may be double-marked by a team of principal examiners, with routine analysis conducted on each test to ensure accuracy and reliability of results.
There is no space for “computer says no” with IELTS, with multiple people reviewing and marking every written and spoken test.
Humans or robots?
“Hey, Google! What’s the weather tomorrow? When’s my dentist appointment? Skip this track! Dim the lights.”
It’s no denying AI can be useful in everyday life. In fact, technology plays a vital part in developing and delivering the IELTS test. And, as more test centres around the world switch to computer-delivered IELTS, it’s apparent that technology not only makes our lives better, but our test days easier and more accessible. With the introduction of computer-delivered IELTS , we’ve been able to release results faster, test 7 days a week, multiple times per day – and increase test day efficiencies.
But do you trust AI to not have bias when marking your high-stakes English test? The choice is very clear.
AI out, humans in
In light of the allegations made the Apple Card customer, Apple co-founder Steve Wozniak said that “Algos (algorithms) obviously have flaws. A huge number of people would say, ‘We love our technology, but we are no longer in control.’ I think that’s the case.” Industry regulators are now investigating these claims of AI discrimination by Apple Card.
For English test takers, IELTS is the fair and reliable choice for study, migration and work goals. With more than 40 test centres Australia-wide , and a growing number of computer-delivered centres in all major cities, take the test that is marked by humans, for humans.