What happened
It was reported that in 2024, Taco Bell expanded its AI voice-ordering system to more than 500 drive-thru locations across the United States, hoping to speed up service, reduce pressure on employees and make ordering more efficient.
The technology soon attracted attention for reasons the company probably had not anticipated, as customers began sharing videos of conversations in which the assistant struggled with ordinary orders or accepted increasingly ridiculous requests.
In one widely circulated video, a customer asked for 18,000 cups of water, and the system continued responding to the request rather than questioning whether it made sense. Other recordings showed customers becoming frustrated as the assistant misunderstood menu items, suggested things they had not ordered or repeatedly asked for clarification.
Although some of these interactions involved customers deliberately testing the technology, they raised questions about how reliably it could handle the unpredictable nature of everyday customer conversations.
By August 2025, Taco Bell’s parent company, Yum Brands, was taking a more cautious approach, with executives acknowledging that relying entirely on AI at busy drive-thrus might not be practical. Rather than abandoning the technology, the company began giving restaurants greater flexibility over when to use automated ordering and when employees should intervene.
What an auditable version would have shown
The viral recordings demonstrated unusual behaviour, but they could not establish how frequently the system struggled during normal operations or how many mistakes employees corrected before customers received their orders.
An auditable system would have captured what customers requested, how the AI interpreted those requests and whether staff needed to intervene, allowing the company to identify patterns across individual restaurants rather than relying on isolated complaints.
A ConductRecord could preserve the interaction and the resulting order, while a MetricRecord could track repeated misunderstandings, abandoned transactions and human interventions. These records would help distinguish deliberate customer pranks from problems affecting ordinary service.
The public reporting does not establish which safeguards or internal monitoring arrangements Yum Brands had already implemented.
Where the gap was
The request for 18,000 cups of water illustrates the difference between recognising someone’s words and understanding whether an instruction should be carried out.
Headlights identifies unconstrained action, where an automated system proceeds with a request without sufficiently checking whether the resulting action is reasonable or requires human confirmation.
There is also a human behaviour behind this problem that is easy to overlook. Experienced employees make countless small judgements during ordinary conversations, often without recognising that they are doing so, and those judgements can disappear when the task is handed to software.
A person taking an order would probably question an extraordinary quantity before entering it into the register, whereas an automated assistant needs explicit instructions about when to stop, clarify or ask for help.
The available evidence does not establish that Taco Bell lacked all such controls, or that the unusual orders were ultimately fulfilled. The incidents nevertheless demonstrate how easily an automated conversation can move beyond what a customer or restaurant would reasonably expect.
What governance should have looked like
Taco Bell needed clear operating limits that allowed its AI assistant to handle routine orders while recognising situations that required human judgement.
Unusually large quantities, repeated misunderstandings and customers struggling to complete an order should prompt clarification or transfer to an employee rather than allowing the conversation to continue without resolution.
A ConstraintGate could check proposed orders against defined limits before they were processed, while ConductRecord and MetricRecord could help identify recurring problems and determine where human assistance was needed most often.
These controls would not prevent customers from making unusual requests, but they would reduce the risk of the system treating every instruction as something it should automatically accept.
The broader lesson is that automating a conversation requires more than recognising speech. It also requires the judgement to recognise when a request does not make sense and when someone should step in.
Failure Pattern: Taco Bell’s AI drive-thru assistant struggled with some customer interactions and responded to unusually large orders, leading to viral criticism and a more cautious deployment approach from Yum Brands.
Governance Principle: AI systems that turn customer requests into actions need defined operating limits, checks for unusual behaviour and a reliable way to transfer control to a person.
The reference implementation of ConstraintGate, ConductRecord and MetricRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- Taco Bell is having second thoughts about relying on AI at the drive-through (TechCrunch, on the WSJ account, August 2025)
- Yum Brands says AI drive-thru is in more than 500 Taco Bell locations (CNBC)
- [McDonald’s ends AI drive-thru partnership with IBM (Restaurant Busin