What happened
In October 2017, a Palestinian construction worker posted a photo of himself at work with a simple greeting in Arabic that meant “good morning.”
Facebook’s automatic translation got it badly wrong.
Instead of translating the post as a friendly greeting, it reportedly translated it as “attack them” in English and “hurt them” in Hebrew. The difference came down to a single letter in the original Arabic.
Israeli police saw the translated post and arrested the man on suspicion of incitement. According to public reports, no Arabic-speaking officer checked the original post before action was taken. Once investigators realised the translation was incorrect, the man was questioned and released.
Facebook apologised for the mistake and said it would continue improving its translation system.
What an auditable version would have shown
An auditable translation system would make it obvious when a translation had been generated by AI rather than a person. It would keep a record of the original text, the translated version, and how confident the system was in its translation. For short phrases where a single letter can completely change the meaning, it would flag the translation as uncertain rather than presenting it as fact. Most importantly, it would show that a significant decision was about to be made based on a machine translation that no human had verified.
Where the gap was
The problem wasn’t that the AI made a translation mistake. The problem was that the translation was treated as if it were a verified fact. Machine translation is often least reliable with short phrases, slang and local dialects. Yet there was nothing in the process requiring someone who actually spoke Arabic to read the original message before action was taken. The technology made the first mistake. The process allowed it to become a human one.
What governance should have looked like
If an AI translation is going to influence a decision about a person, it should not be treated as the final answer. The translation should be clearly marked as machine-generated, include an indication of how confident the system is, and require a human fluent in the language to verify the original before any action is taken. A VerificationGate is designed to pause decisions until that human check has happened. A ConductRecord keeps the original text, the AI translation, and a record of who relied on it. Together they make it possible to understand not only that a mistake occurred, but exactly where it entered the process and why nobody caught it. The goal isn’t to stop people using AI translation. It’s to stop a machine’s best guess becoming the basis for a decision about another person’s life.
The reference implementation of VerificationGate and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed, free for any company to install. The repository is public now.
Sources
- Israeli police arrest Palestinian for ‘good morning’ Facebook post (Times of Israel)
- Israel arrested a Palestinian after Facebook translated “Good morning” as “Attack them” (Quartz)
- Israel arrests Palestinian because Facebook translated ‘Good Morning’ to ‘Attack Them’ (Haaretz)