What happened
It was reported that home insurers in the United States increasingly decide whether to renew a policy by running artificial intelligence over aerial and satellite images of the house, rather than by sending an inspector. Specialist analytics firms sell insurers this capability, using computer vision to estimate a roof’s age and condition, identify its material, measure tree cover and judge other signs of risk from an overhead picture. A survey by insurance regulators found that most home insurers were already using or building such tools, including in underwriting.
It was reported that homeowners have been dropped or told to make expensive repairs on the strength of these images, sometimes wrongly. A woman near Houston, Alaina Callahan, said her insurer told her to replace a roof that was five years old and sound, and that she had, in her words, no recourse as a homeowner, none whatsoever. Another homeowner, Tracy Gartenmann in Austin, said Travelers emailed her two grainy overhead photos and told her to cut back trees near the house or lose cover. In one case a homeowner lost cover after the insurer relied on the wrong image, and homeowners were often never shown the picture the decision rested on. The Texas Department of Insurance said it had taken complaints about the practice. Insurance regulators have started to respond: in 2023 the National Association of Insurance Commissioners issued a model bulletin telling insurers to govern their AI, watch for bias, check the data and oversee outside vendors, and about half the states have adopted it, while a proposed law in California would require an insurer to show a homeowner any image used to cut or refuse cover and let them dispute it.
What an auditable version would have shown
An insurer’s decision to drop a house or demand a new roof, made from a picture taken from above, is one the homeowner cannot check unless they can see the picture and what was read from it. An auditable version keeps, for each decision, the image used, the date it was taken, what the model concluded, and the basis, and makes it available to the homeowner, so a roof that has since been repaired, or a tree that has since been removed, or simply the wrong house, is something the owner can point to rather than a verdict they cannot reach behind. It also keeps the aggregate, how often the model’s reading is later found to be wrong, so the accuracy of the tool is a figure the insurer and a regulator can see.
Where the gap was
A decision that cost a homeowner their cover was made from an image, and a model’s reading of it, that the homeowner never saw and could not contest. A ConductRecord keeps the image, its date, the model’s finding and the basis for the decision, and puts it in the homeowner’s hands, so a mistake, a repaired roof, a felled tree, the wrong address, can be shown and corrected. A MetricRecord keeps how often the model’s readings turn out to be wrong, so the reliability of the tool is measured rather than assumed. The regulators’ model bulletin and the proposed California law point at the same things a record holds: show the person the image, let them dispute it, and check how well the system performs.
What governance should have looked like
Where an insurer makes an adverse decision about a home from an automated reading of an image, the homeowner should be shown the image and the reading, be able to dispute them, and the insurer should know how often the readings are wrong. Best practice would be for an insurer to keep, for every decision of this kind, the image used and its date, what the model concluded, and the ground for the decision, to give that to the homeowner, and to measure across all such decisions how often the model is later found to be mistaken. The homeowners who were dropped over a repaired roof or a vanished tree had no way to reach the picture behind the decision. A record they can see, and a measured error rate, are what turn an image taken from the sky into something a person can answer.
Failure Pattern: an insurer made an adverse decision about a home from an automated reading of an aerial image, and the homeowner could neither see the image and the reading nor contest them, while the model’s error rate went unmeasured.
Governance Principle: where an automated reading of an image drives an adverse decision about a person’s property, the person must be shown the image and the reading and be able to dispute them, and the model’s error rate must be measured.
The reference implementation of ConductRecord and MetricRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- Insurance companies using aerial imagery to determine if they’ll renew home coverage (NPR)
- NAIC members approve model bulletin on use of AI by insurers (National Association of Insurance Commissioners)
- Artificial intelligence is inspecting your roof and deciding your insurance future (Property Insurance Coverage Law Blog)