The Australian branch of Deloitte, one of the Big Four accounting firms, has recently come under public scrutiny over a document submitted to the Australian government that was allegedly riddled with errors generated by artificial intelligence (AI). Experts say the incident, involving a well-known institution in the corporate finance sector, should serve as a wake-up call for all practitioners using AI tools without adequate safeguards.

"This situation delivers a critical lesson for finance professionals: AI is not a 'truth-teller' but a tool designed to provide answers that fit your questions," said Bryan Lapidus, FP&A Practice Director at the Association for Financial Professionals, in an email. The association provides training and certification for finance practitioners.

According to the Associated Press, the report submitted by Deloitte Australia to the Department of Employment and Workplace Relations (DEWR) was initially published on the department's website in July. A revised version was released earlier this month after the original was found to contain errors, including fabricated citations of Federal Court rulings and references to fictitious academic research. Deloitte Australia agreed to partially refund the A$440,000 (about US$290,000) the Australian government paid for the document, which reviewed the department's IT systems used to automate penalty decisions in the welfare system.

"Deloitte conducted an independent assurance review and confirmed that some footnotes and references were incorrect," a DEWR spokesperson said in an email, adding that the "substance" of the review was retained. The spokesperson said Deloitte had agreed to refund the final payment under the contract. Deloitte Australia did not immediately respond to a request for comment.

University of Sydney researcher Chris Rudge is credited with first exposing the errors. According to an article published by The Australian Financial Review in August, he found the Deloitte report was "riddled with citation errors," raising concerns that the report may have been generated by AI.

Nikki MacKenzie, an assistant professor at the Scheller College of Business at Georgia Tech, said the incident reminds businesses that AI is "smart but not infallible." "We keep hearing about how 'intelligent' AI is becoming, which can lead people to trust it too much," she said in an email. "Consciously or not, we start to over-rely on it."

Tools such as generative AI chatbots are prone to "hallucinations," producing outputs that are "nonsensical or completely inaccurate," according to an explanatory article published by ChatGPT developer OpenAI. Hallucinations can occur when AI models are trained on datasets containing biased or unrepresentative data. AI tools can also be subject to adversarial attacks, where malicious actors manipulate outputs by subtly altering input data, the article said.

A study released by KPMG in April showed that nearly six in ten (59%) employees admitted to making mistakes at work due to AI errors. Additionally, about half of employees were unsure whether they were allowed to use AI in the workplace, and more than four in ten (41%) used AI "knowingly inappropriately," the study found.

Experts say the Deloitte incident, while alarming, is unlikely to lead to a pause in AI adoption. "I believe companies will treat it as a normal cost of doing business," MacKenzie said. "Just as employees make mistakes, tools make mistakes. The goal isn't to avoid AI errors—it's to ensure we're smart enough, as the ultimate decision-makers, to catch them."

The Deloitte incident is just the latest high-profile case of risks posed by the rapid deployment of AI across industries. In January, Apple suspended an AI feature used to summarize news alerts after complaints that it generated false and inaccurate information. In 2023, a federal judge in New York sanctioned two lawyers who submitted a legal brief containing fabricated case citations generated by ChatGPT.

Jack Castonguay, an associate professor of accounting at Hofstra University, said Deloitte is the first large accounting firm he knows of to have a report with AI-generated fabricated details. "It seemed like only a matter of time," he said in an email. "Frankly, I'm surprised it took this long for it to happen at a large firm."

MacKenzie said the incident highlights the need for organizations to establish AI safeguards or internal controls to mitigate risks such as hallucinations. "Responsibility still lies with the professionals using it," she said. "Accountants must take ownership of their work, review outputs, and apply judgment rather than copy-pasting whatever the system produces."