Customer Support Replies with AI

The big study: who gained the most

This is the strongest evidence on the subject: 5,172 support agents, roughly three million conversations, rolled out in stages — so not self-reported, but the system's own data.

The single most important sentence out of this: the beginner gained the most. The machine did not make the best better — it narrowed the spread. The learning effect was measurable too: an agent with two months of experience and an assistant performed like one with six months and no help.

And the same study says that among the most experienced, quality slipped slightly. That is not a footnote: it means the assistant is not worth turning loose on everyone in the same way.

The 2026 turn: assistant or replacement

Up to this point the story is a happy one. The recent field experiment picks up where the earlier one stopped — and arrives somewhere else.

In a 2026 randomised experiment the setup was no longer an assistant: AI led the conversation, with a human taking over if needed.

• Conversations got shorter.
• The share of repeat contacts barely moved.
• But on the conversations that could be routed to AI, customer ratings got significantly worse.

The most interesting part, though, is that human takeover does not always help — it depends on why it became necessary:

On emotional escalations the agents wrote fewer messages, asked fewer proactive questions, engaged less with the conversation. The researchers' conclusion is unambiguous: early intervention is essential — after a late takeover the effort no longer recovers.

There was a positive side effect too: the quality of the non-AI-capable conversations improved, because agents could allocate their attention better.

The central tension of this article in one sentence: when AI sits next to a person, everyone wins. When it sits there instead of them, the customer can tell.

Four documented counterexamples

The first: the reversal. In February 2024 a large payments provider announced that its assistant was doing the work of 700 full-time employees. In May 2025 it reversed course and started hiring people for support again. The chief executive's phrasing is the most quotable part: in his view human customer service will always be a premium, high-touch offering. (Precision matters: headcount still fell sharply over the same period — this was not a full return.)

The second: the invented policy. In April 2025 the support bot of a developer tool invented a login rule that had never existed, and delivered it signed with a human name. Users cancelled their subscriptions; the company apologised. This is the most instructive case on the list: the bot did not simply "answer badly" — it confidently stated a company policy that did not exist, which the customer was right to take as the company's position.

The third: the legal precedent. In a Canadian decision from February 2024 the airline argued that the chatbot was a separate legal entity that the company was not answerable for. The tribunal's reply was short: a chatbot has an interactive component, but it is still part of the company's website — and it should be obvious that the company is responsible for everything on its own site.

The fourth: the market turning back. According to a survey of 163 support leaders, half of the organisations that had planned significant headcount cuts because of AI will drop that plan by 2027. 95% want to keep human agents. And the most telling number: 51% of customers trust a human agent most, 7% trust AI — while 62% are worried that reaching a human will get harder.

Brand voice: it does not write badly, it pulls toward the average

Worth knowing up front: there is no peer-reviewed measurement specifically on brand-voice drift — what you find under that heading is marketing material with no method.

What has been measured does transfer, though, and it describes the problem more precisely. A study running across 880,000 texts and seven datasets found that language models cut the variance of written complexity by 21–50%. In their own wording, they amplify the dominant patterns and suppress the rest — they reward conformity over distinctiveness. Another piece of research found the same on creative tasks: individual output rises, collective diversity falls.

So the brand-voice problem is not that the machine writes badly. It is that it pulls toward the average. If your voice lives exactly in that spread — short sentences, first names, a concrete number, humour — then the generated reply measurably drags it back to the mean.

There is one practical consequence: brand voice should not be described, it should be shown. Ten to fifteen of your own earlier replies that you consider good are worth more than any style guide.

What you have to disclose

This part is new in 2026, and many people have the source of the obligation wrong.

Under Article 50 of the EU AI Act, AI systems that interact directly with people have to be designed so that the person concerned is informed that they are dealing with an AI system — unless this is obvious anyway to a reasonably well-informed, observant and circumspect person. The information has to be clear and distinguishable, and it has to be given at the latest at the time of the first interaction.

This has applied since 2 August 2026, and the postponement did not touch it. The 2026 change to the timetable pushed back the deadline for high-risk systems — not the transparency rules.

The penalty band: up to €15 million or 3% of total worldwide annual turnover, whichever is higher.

There is another, less well-known obligation as well: under Article 4 of the Act, since 2 February 2025 organisations operating AI systems have had to take measures to build AI literacy among their own staff. It prescribes no specific level, but the obligation is live.

And one Hungarian clarification that many people have backwards: Hungarian consumer protection law contains no provision whatsoever on chatbots, automated handling or AI. The disclosure obligation comes solely from the EU regulation.

What Hungarian law does require — but only of public utility providers and of those named in a separate act: they have to run a customer service line, take the call and begin substantive handling within five minutes of the connection, make the live voice option for complaints the first menu item without identification, and record telephone complaints with a unique identifier and keep them for five years. Most online shops in Hungary are not required to run a telephone support line at all.

The same disclosure obligation applies on the other side, where the machine assesses not the customer but the applicant: the job applications and interviews post covers that side in detail.

(This is not legal advice. For your own situation the classification decides, and that is worth going through with a professional.)

The recipe

1. ASSISTANT, NOT REPLACEMENT
   The machine writes the draft, a person presses
   send. That is exactly where the studies differ.

2. LET THE BEGINNERS HAVE IT FIRST
   That is where the largest gain was measured. Among
   the best, a small loss of quality was measured.

3. AT AN EMOTIONAL SIGNAL, A HUMAN AT ONCE
   Not after escalation, but at the first sign. After
   a late takeover the effort does not recover.

4. SHOW THE BRAND VOICE, DO NOT DESCRIBE IT
   10-15 of your own earlier replies in the prompt. A
   style guide fails, the model pulls to the average.

5. FACTS ONLY FROM A SYSTEM
   Price, deadline, policy, warranty: from a tool call.
   What the model "knows" we do not tell a customer.

6. THE FIRST MESSAGE HAS TO SAY IT
   That they are talking to a machine. Not in fine print.

Point five is the one that heads off the invented-policy case. The model does not state a falsehood because it is bad — it does so because it has no way to know, and saying something is cheaper for it than saying "I don't know".

Point one, meanwhile, is the whole article in a single line: the studies show a gain when the machine helps; they show a decline when it takes over.

Common misconceptions

• "Everyone gets 15% more efficient." No. Over 30% among beginners, a small speed-up and a small loss of quality among the best. The average misleads here.
• "The customer won't notice anyway." In the 2026 experiment the ratings dropped exactly where AI ran the conversation.
• "If something goes wrong, someone will take over." It depends when. On an emotional escalation a late takeover measurably works less well.
• "I'll describe the brand voice in the prompt." The model measurably narrows the spread. It needs examples, not a description.
• "Hungarian consumer protection law requires chatbots to be labelled." It does not. The obligation comes from Article 50 of the EU regulation.
• "We're not answerable for the bot, it is a separate service." That is precisely the defence an airline ran in 2024, and lost.

What to take away

Customer support is the area with the best measurements — and precisely for that reason you can say exactly where the line runs.

As an assistant it is measurably good, especially for beginners, and it steepens the learning curve too. As a replacement it measurably degrades the experience, and the market has confirmed that: half the organisations that had planned headcount cuts will drop that plan by 2027, and 7% of customers trust the machine most.

The three things that decide it in practice: a person presses send, the fact comes from a system, and the first message says that this is a machine.

How the sorting of incoming mail and the draft replies fit into all this is laid out in the inbox triage post as a recipe; for the broader question of tone and brand voice, the real time savings post supplies the measurement background.