What happened
It was reported that in March 2017 Flight Centre held a ‘Design Jam’, an event at which teams competed to build technology, including tools to automate parts of a travel agent’s work, using a dataset the company supplied. Flight Centre intended the dataset to be de-identified, and ran preliminary checks meant to strip out personal information. Those checks did not do what was assumed. The dataset that reached the developers held the personal information of 6,918 customers, including 4,011 credit card numbers, 5,092 passport numbers and 475 sets of login credentials. The company discovered the error after the data had been available for about 36 hours.
It was then reported that in December 2020 the Australian Information Commissioner and Privacy Commissioner, Angelene Falk, published her determination. She found that Flight Centre had interfered with the privacy of the affected customers and had breached three Australian Privacy Principles: the obligation to take reasonable steps to put in place practices that would ensure compliance, the prohibition on disclosing personal information for a purpose the customer had not consented to, and the obligation to take reasonable steps to protect personal information from unauthorised disclosure. The Commissioner decided no further action was warranted, noting that Flight Centre had cooperated with the investigation and had taken steps to reduce the harm, including paying at least $68,500 to replace customers’ passports. The finding was not that a hacker had broken in. It was that the company had handed the data out itself, to people outside its own walls, in the belief that a file it had not properly inspected was safe to share.
What an auditable version would have shown
The failure sat in the gap between “we de-identified this” and “we checked that this was de-identified”. Those are different claims, and only the second is a record. An auditable version treats any dataset crossing from the company to outside developers as an egress event that must be classified and logged: what fields the file actually contains, scanned at the point of release rather than assumed from an earlier step, the classification of each field, the destination it is going to, and the named approval that let it go. Run against the Design Jam file, that check reads the credit card and passport numbers still sitting in the data and stops the release, and it leaves a signed record showing the check ran and what it found. Instead, the first time anyone established what the file truly held was after it had already been in outside hands for a day and a half.
Where the gap was
Sensitive personal data left the company’s trust boundary for an external event, on the assumption it had been stripped out, with no check that verified the file’s contents before it went. An EgressGate performs exactly that check: it classifies the data in an outbound file and tests the destination against a standing rule, so that credit card and passport numbers cannot leave for an outside team without an explicit, recorded exception. A ConductRecord preserves what was released, to whom, and which de-identification steps were actually run and with what result, so that a de-identification claim is a verifiable log entry rather than a belief the organisation holds about its own process.
What governance should have looked like
De-identifying a dataset is a claim about a file, and a claim about a file can be checked against the file. Once data has been distributed to sixteen teams it cannot be recalled, only apologised for and, in the case of passports, paid to replace, which puts the weight on getting the check right at the point of release. A project that needs real data to build automation places that data in front of more people, in more places, than usual, which raises rather than lowers the value of a check at the boundary. Best practice would be for an organisation releasing customer data to outside developers to be able to show, from a signed record made at the moment of release, what was in the file and who cleared it to leave, so that “we thought it was de-identified” is a claim the logs can confirm or refute.
Failure Pattern: sensitive personal data was released to outside developers on the assumption it had been de-identified, with no check that verified what the dataset actually contained before it left.
Governance Principle: data leaving the trust boundary must be classified and verified against a standing rule before release, not released on the belief that it is already safe.
The reference implementation of EgressGate and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- Flight Centre found to have interfered with privacy (Office of the Australian Information Commissioner)
- Travel agent leaked customer data by giving it away in a hackathon (The Register)
- Australian travel agency criticized over coding event that exposed sensitive user data to external software developers (The Daily Swig)
- Flight Centre’s Design Jam Data Breach: What can we learn? (Privacy108)