What happened
It was reported that in December 2021 a 10-year-old, at home with her mother, asked Amazon’s Alexa for a challenge to do. Reporting indicates Alexa came back with one it had found on the web, the so-called outlet challenge, and suggested plugging a phone charger about half way into a wall socket and touching a coin to the exposed pins, which can cause an electric shock, sparks or a fire. Her mother, Kristin Livdahl, was in the room and stopped it immediately, and her daughter did not try the challenge.
Livdahl shared what had happened and the story quickly attracted attention. Amazon said it took swift action to fix the response once it became aware, and that Alexa was designed to provide accurate and helpful information. Reporting linked Alexa’s instructions to an article on a website called Our Community Now, and somewhere between finding that information on the web and reading it aloud to a child, there was no check that asked a very basic question: could this hurt someone?
What an auditable version would have shown
For an assistant like Alexa, you would want to know where an answer came from and what happened to it before it was spoken. In this case, there should have been a record showing that a request for a challenge, from what appeared to be a child, had been checked for physical danger, and that a suggestion involving a live power socket had then been blocked or replaced with something safe. That record would also tell Amazon whether its safety controls were catching dangerous suggestions before they reached people, rather than finding out after a parent reported one.
Where the gap was
Alexa could find the challenge. What it did not do was recognise that repeating it as a suggestion could be dangerous. The information moved from a web page to a child without a safety check in between, and there was no stronger limit because the person asking appeared to be young. Finding an answer and deciding whether it is safe to give someone that answer are two different jobs.
What governance should have looked like
An assistant used by children needs a firm limit around suggestions that could cause physical harm, regardless of what it finds online. A ConstraintGate is designed to stop that kind of output before it reaches the user, so a dare involving a live socket, a poison or a weapon is caught rather than repeated as a suggestion, with extra care when the user may be a child. A ConductRecord would preserve what was asked, what information was found and what Alexa eventually said, which gives the company something concrete to investigate when a harmful response gets through, and a way to understand exactly where the safety check failed.
The reference implementation of ConstraintGate and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed, free for any company to install. The repository is public now.
Sources
- Amazon’s Alexa tells a 10-year-old to touch a penny to an exposed plug socket (CNN)
- Amazon’s Alexa told a child to do a potentially lethal challenge (CNBC)
- Alexa reportedly told child to touch penny to outlet, mom says (KBTX)