What happened
In March 2023, Guardian Australia reported that one of its journalists had defeated the voice identification used on Centrelink’s phone self-service line, using an AI clone of his own voice. The Guardian reported that about four minutes of audio was enough to generate the clone with an online voice-cloning service (tools of this kind, it noted, are offered free or for a small fee), and that the cloned voice, together with the journalist’s customer reference number, passed the “voiceprint” check and opened access to his Centrelink self-service account. The same underlying technology is used by the ATO, though the Guardian noted the ATO applies it during conversations with staff, which may be harder to exploit in real time.
The scale is what made the demonstration land. The Guardian reported that 3.8 million Centrelink clients were using voiceprint at the time, and that more than 7.1 million people had verified their voices with the ATO. (The phrase customers repeat to enrol, “in Australia, my voice identifies me”, is the system’s own summary of the promise.) Services Australia defended the technology: its spokesperson said voice ID was a highly secure authentication method and that the agency has the capacity to continually assess risks and update its processes, though the Guardian reported it declined to say whether the technology would be changed or removed. The system, as reported, treated a matching voice plus a reference number as a verified identity; the Guardian’s investigation reported that a synthetic voice could produce that match, while it noted the reference number is not treated as securely as a password and appears on Centrelink correspondence.
What an auditable version would have shown
When a biometric gate is the front door to millions of accounts, the governing questions are: what evidence did the system accept, what score did it assign, and what did it do when the evidence was synthetic? An auditable version records each authentication event with the inputs evaluated, the liveness and spoofing checks that ran (or did not run), and the threshold that admitted the caller, signed at the time. When a clone gets through, the record shows exactly which check failed to fire, and a claim like “highly secure” can be tested against the system’s own logged behaviour rather than asserted over it.
Where the gap was
A match score, plus a reference number that appears on paper correspondence, was treated as proof of identity. A VerificationGate routes the question “is this a live human, and the right one?” to trusted checks beyond the model’s own similarity score: liveness detection, a second factor, a channel the caller cannot synthesise. The model proposes a match; something the attacker cannot fake has to dispose. A ConductRecord preserves every authentication decision with the evidence behind it, so an agency can measure how often synthetic audio is probing the gate instead of learning about it from a newspaper.
What governance should have looked like
Biometric convenience is a promise about attackers, not just customers, and by 2023 the attacker had a voice synthesiser. A verification system that accepts the very artefact AI has learned to fabricate, with nothing in the record to show a liveness check stood in the way, is security by assumption. The governing principle is simple: never let the thing a model can generate be the only thing a gate accepts. And keep a signed record of every decision, so when the assumption breaks, the system’s owners are the first to know, not the last.
The reference implementation of VerificationGate and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- AI can fool voice recognition used to verify identity by Centrelink and Australian tax office (The Guardian)
- Incident 523: Australian Journalist Able to Access Centrelink Account Using AI Audio of Own Voice (AI Incident Database)
- The Guardian: AI Can Fool Voice Recognition Used to Verify Identity by Centrelink and Australian Tax Office (NSW Council for Civil Liberties)