Klarna's CEO announced an AI customer service assistant was doing "the work of 700 agents", then walked it back a year later

What happened

On 27 February 2024, the Swedish buy-now-pay-later company Klarna issued a press release announcing that an AI customer service assistant, built in partnership with OpenAI, had handled 2.3 million customer conversations in its first month of operation. The release framed the achievement in a striking comparison: the bot was doing "the work of 700 full-time agents." Klarna projected the deployment would drive a US$40 million improvement in profit in 2024. Customer satisfaction scores were comparable to human agents. Resolution times had dropped by an average of 9 minutes per chat. Repeat inquiry rates had fallen 25 percent.

OpenAI published a case study. The financial press picked it up as the breakthrough proof point for AI in customer service: a real company, at real scale, with real metrics, demonstrating that AI could replace customer service agents wholesale. CEO Sebastian Siemiatkowski did several podcast interviews and a substantial press tour. The "work of 700 agents" framing became the most-cited single statistic in the AI-replacing-humans discourse of 2024.

Underneath the framing, the actual figure was more nuanced than the press tour acknowledged. Between 2022 and 2024, Klarna had reduced its global headcount from roughly 5,527 employees to 3,422, a reduction of around 40 percent. The "700 agents" comparison did not refer to seven hundred specific people who had been laid off. It referred to the additional agents Klarna would have needed to hire to handle the conversation volume during a growth phase if humans had been doing the work. Klarna had avoided hiring, not laid off. The distinction matters less when the headline framing carries the implication that AI directly displaced workers, which it did, in net effect, just not via the specific causal chain the framing implied.

Through 2024, complaints accumulated. Customers reported that the bot handled simple queries, refund status, account balance, basic transaction questions, well, but degraded on anything that required nuance, judgement, or sustained context across multiple messages. Cases involving disputes, fraud, hardship, or complex multi-product interactions were particularly poorly served. Customer satisfaction in the cohort of complex cases declined materially. The bot occasionally hallucinated, in the customer-service idiom: invented refund policies, claimed account features that did not exist, gave timelines for actions it could not actually take.

In May 2025, in a Bloomberg interview, Siemiatkowski acknowledged that the company had over-deployed. "We focused too much on efficiency and cost," he said. "The result was lower quality, and that's not sustainable." In the same cycle he made a more striking point: "From a brand perspective, a company perspective, I just think it's so critical that you are clear to your customer that there will always be a human if you want." Klarna announced a new approach: human agents would handle complex cases, AI would handle routine ones, and the company would explicitly hire for an Uber-style flexible workforce, remote agents, including students and parents, on flexible schedules. The "work of 700 agents" framing was retired. The pivot was reported widely as the most visible public reversal of an AI-replaces-humans deployment to date.

What an auditable version would have shown

The 2024 metrics Klarna published were aggregate. Chats handled. Average resolution time. Customer satisfaction. Each is informative. None of them, on their own, distinguishes between the bot being good at customer service and the bot being good at the easy half of customer service.

An auditable version of the metric stack would slice every quality measure by case complexity. Klarna's case management system already classified cases by type, refund status, balance inquiry, transaction dispute, fraud claim, hardship request, multi-product query. The aggregate CSAT score across all cases looked acceptable through 2024. The CSAT score on the dispute, fraud and hardship segments declined materially during the same period. The aggregate number, the one Klarna published, the one the press repeated, concealed a degrading distribution within it.

A structured per-case record would have made the pattern visible at the operational level long before the public reversal. Each case: type, route (AI-handled, human-handled, AI-then-escalated), resolution outcome, customer satisfaction, complaint trail. Aggregated by case type, the data would have shown, and almost certainly did show, in Klarna's internal systems, that the bot's value-add was concentrated in the easy cases and that the hard cases were being handled worse, not better, than under the previous human-only regime.

The question is not whether the bot was succeeding overall. It clearly was on volume and on routine cases. The question is whether the bot was succeeding in the cases where customer trust is built or destroyed: the dispute, the fraud claim, the hardship request. Trust is a low-frequency, high-stakes signal. It does not move the aggregate CSAT line for months, and by the time it does, the reversal is what makes the news, not the deployment.

Where the gap was

The gap was in segmentation discipline, not in the underlying technology.

The OpenAI-powered assistant was capable of handling the routine 70 percent of customer-service queries to a standard customers found acceptable. That part of the deployment worked. What was missing was a structural control that prevented the AI from being deployed against the cases it was not yet capable of handling well, and a clear escalation path that routed complex cases to humans early and unambiguously, rather than after the customer had spent forty minutes failing to make progress with the bot.

The publicly available accounts of the deployment suggest the bot was given a broad mandate: handle any customer interaction it could, escalate when it could not. The escalation criterion was the bot's own confidence, a function the bot could be confidently wrong about. Customers in dispute or fraud cases routinely encountered a bot that believed it could help, persisted in trying to help, and resolved the case in a way that resolved it for the metrics but did not resolve it for the customer.

A typed-case routing system, where case types known to require human judgement bypass the AI entirely, and case types where the AI is competent are handled by the AI with explicit human-escalation tripwires on negative-sentiment signals, would have surfaced the bot's strengths without exposing the customers in the hardest cases to its weaknesses. Klarna's pivot in 2025 was, in essence, the implementation of this control after the fact.

What governance should have looked like

For AI deployed in customer service, the governance question is not how much volume the AI can absorb. It is how the institution segments the work and how it captures the structured signal that tells it the segmentation is wrong.

from headlights import (
    ConductRecord,
    MetricRecord,
    PersonaGuard,
    sign,
    chain,
)
from datetime import datetime, timezone

# The guard refuses to route certain case types to AI entirely.
guard = PersonaGuard(
    ai_handles=["refund_status", "balance_inquiry", "shipment_tracking",
                "basic_transaction_query"],
    human_only=["dispute", "fraud_claim", "hardship_request",
                "complex_multi_product"],
    escalation_signals=["negative_sentiment_repeated",
                        "customer_request_to_speak_to_human",
                        "case_duration_over_threshold"],
)

case_type = classify(incoming_message)   # routed at intake
if not guard.allows_ai(case_type):
    route_to_human_agent()
else:
    response = ai_handle(incoming_message)

# Every case is recorded with its handler, its outcome and its
# customer-side signals.
record = ConductRecord(
    workflow="customer_service_case",
    case_id=case_id,
    case_type=case_type,
    handler="ai" if ai_handled else "human",
    ai_model_version="gpt-4-1106-preview" if ai_handled else None,
    resolution_outcome=outcome,
    customer_satisfaction_score=csat_score,
    complaint_lodged=complaint_lodged,
    escalation_triggered=escalation_triggered,
    timestamp=datetime.now(timezone.utc),
    previous_record_hash=last_record.hash(),
)

signed = sign(record, key=klarna_private_key)
chain.append(signed)

# Aggregated metrics, sliced by case type and handler, are published to
# internal dashboards continuously. The aggregate CSAT line is one of
# many, never the only one watched.
metrics = MetricRecord(
    period_start=period_start,
    period_end=period_end,
    by_case_type_and_handler={
        ("refund_status", "ai"): {"csat": 4.6, "complaint_rate": 0.012, "n": 84_321},
        ("dispute", "human"):   {"csat": 4.2, "complaint_rate": 0.024, "n": 1_823},
        # Each segment a separate row. Aggregate masks segment drift.
    },
)

The first move is to classify each case at first contact and route it based on what kind of case it is, not based on how confident the bot thinks it is. Refund status, balance inquiry, shipment tracking, these go to the AI. Dispute, fraud claim, hardship request, these go to a human, by design, with no AI in the loop. The AI's mandate is bounded explicitly. The bot does not get the chance to attempt cases it is not yet competent at, and customers in the hardest moments do not have to bounce off it before reaching someone who can help.

The second move is to log every case with its type, its handler, its outcome and its customer-side signals, independent of whether the AI or a human took it. The aggregate metrics across all of those records are useful. The segment metrics, sliced by case type and handler, are the operational signal. Aggregate CSAT will look fine for a long time even as the dispute segment quietly degrades. If the segment numbers are the ones in front of the operator, the segment problem surfaces before the brand problem does.

The third is to make the escalation criterion the customer's experience, not the bot's self-assessment. Negative-sentiment patterns across messages, explicit customer requests for a human, case duration past a sensible threshold, each is a recorded signal that pulls a human agent in automatically. The bot does not decide when it has failed. The customer's behaviour decides, and the system listens.

Customer-service work is not homogenous. The "AI does the work of N agents" framing of 2024 quietly assumed it was. The 30 percent of cases where customer trust is actually made or lost are also the cases where AI is least mature, and where deploying it widely costs the brand more than it saves on payroll. Klarna's reversal is the cleanest public statement of that, but it is not the last one, every customer-service deployment of this size will eventually meet the same wall, and the operators with the segment-level records will see it coming first.