What happened
It was reported that in 2024 Microsoft announced Recall for its new Copilot PCs. Reporting indicates the feature took screenshots of a user’s screen a few seconds apart and turned them into a searchable timeline of almost everything they had seen or done, from web pages to chats, and that the original design was intended to be enabled by default on eligible Copilot+ PCs. The problem was what those screenshots could contain: reporting found that the filtering did not reliably exclude sensitive information, so snapshots could include passwords and financial details.
Security researchers then showed that the snapshots were stored on the computer without proper protection, so that if an attacker already had access to the machine, the data was easy to find and read, and one researcher released a tool demonstrating how quickly the timeline could be extracted. After a strong public backlash, Microsoft delayed Recall in June 2024, changing the design so it was off by default, encrypting the stored snapshots and requiring a Windows Hello sign-in before the timeline could be opened. Microsoft said it wanted more feedback before releasing the feature widely, and it later reintroduced Recall through a staged rollout in its redesigned form. The security and privacy criticism relates to that first design, before those changes.
What an auditable version would have shown
Before launch, you would want a very clear picture of what Recall captured, where that information was stored, how it was protected and who could access it. You would also want evidence that somebody had tested the store from the perspective of a realistic attacker and recorded what happened, and an answer to one more simple question: did the person actually choose to turn this on? With those records, the risks of creating a searchable history of someone’s activity could have been examined and signed off before launch, rather than being demonstrated by outside security researchers.
Where the gap was
Recall was designed to make recording enormous amounts of information easy. Securing that information was the harder part. A searchable history could contain passwords, messages and financial information, yet the original design was intended to be on by default and the store of sensitive information it created was not properly protected. The person using the computer had also not necessarily made an active choice to create that record in the first place.
What governance should have looked like
A feature that records someone’s activity should start with the person’s choice to use it. Then the information it collects needs to be protected, and the system tested against realistic attacks, before it reaches the public. An EgressGate is designed to control what sensitive information a feature captures and where that information can go, so a record of someone’s activity is not simply left exposed, while an AuthorityGate is designed to make the person’s informed choice part of the decision before the capability is turned on. Together, they address two separate questions: should this information be collected at all, and if it is collected, how is it being protected?
The reference implementation of EgressGate and AuthorityGate is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed, free for any company to install. The repository is public now.
Sources
- Microsoft Recall delayed after privacy and security concerns (Malwarebytes)
- Microsoft Recall on Copilot PC: testing the security and privacy implications (Kevin Beaumont, DoublePulsar)
- Microsoft rolls Recall out to the public after privacy backlash (TechRepublic)