Commissioner Warns of Discriminatory AI and Worsening Racism

Australian Human Rights Commissioner Lorraine Finlay has warned that artificial intelligence (AI) could worsen racism and sexism in the country unless it is subject to strong regulatory oversight. According to her, algorithmic bias—where biases from datasets are transferred into machine learning systems—can combine with automation bias, leading people to place excessive trust in AI outputs and accept discriminatory results as neutral or objective. This could cause discrimination to become "invisible" and deeply embedded in automated decision-making processes.

Risks of Discrimination in Recruitment

Australian research published in May 2025 estimates that about 30% of Australian employers use AI tools for recruitment, and this figure is expected to grow over the next five years. The study, conducted by University of Melbourne Law School researcher Natalie Sheard, highlights the risks of discrimination against job candidates interviewed by AI systems. For example, people with accents or disabilities affecting their speech could be disadvantaged because the datasets used to train AI often do not reflect Australia's demographic groups.

In her research, Sheard interviewed 18 human resources professionals in Australia, 13 of whom used AI systems for recruitment. Résumé analysis systems were the most common, followed by video interview systems. One example from the research shows that a company with an AI system had only 6% of its training data from Australia or New Zealand, while 36% of the applicants in the data were white. The word transcription error rate for native English speakers in the US is less than 10%, but for non-native speakers with accents from other countries, such as speakers from China, it rises to between 12% and 22%.

Research participants described how non-native English speakers or people with speech disabilities may have their words transcribed incorrectly, resulting in a low score from the algorithm. Two of them asked the AI software provider for assurances that the system did not disadvantage candidates with accents and were told that the AI was "really good at understanding accents," but no evidence was provided.

Transparency and Regulation

Lorraine Finlay emphasized the lack of transparency in AI training datasets, which makes it difficult to identify and correct inherent biases. She warns that the pursuit of productivity through AI must not come at the expense of human rights and anti-discrimination objectives. These warnings come amid a political debate within the government over AI regulation. For example, Senator and former AI researcher Michelle Ananda-Rajah argues that making Australian data available for AI training could reduce bias by enabling systems to better reflect local diversity, rather than perpetuating bias from models trained overseas.

Ananda-Rajah opposes a standalone AI law but supports compensating Australian content creators for the use of their data in training. Media and arts organizations have expressed concerns about the potential misuse of Australian content by major technology companies and are calling for stronger copyright and privacy protections.

Recommendations and Next Steps

In her study, Sheard emphasized that no case of AI-driven discrimination has reached the courts in Australia—all cases must first go through the Australian Human Rights Commission. In 2022, the federal Merit Protection Commissioner overturned 11 promotion decisions at Services Australia after the agency outsourced the process to a specialist using AI techniques, including psychometric testing, questionnaires, and video responses recorded by candidates. The process was found to have "not always met the key objective of selecting the most capable candidates."

Sheard suggests that Anthony Albanese's government should consider a dedicated AI law to regulate its use and strengthen existing anti-discrimination laws to protect against AI-driven discrimination. Without these measures, according to Finlay and other stakeholders, AI could deepen existing racial and gender inequalities in Australia, making them more difficult to detect and address.