Every time I tell someone AI handles 80 percent of our support, I get the same question.
What if it is wrong?
Great question. What if the humans are wrong?
That is the whole thing right there.
Here is one of the beautiful things about AI. Tell it to do something a certain way once, and it does that same thing on repeat. It does not veer off. It does not have a bad morning. Somebody said to me today that humans hallucinate more than AI does, once you have trained the AI properly. I have been chewing on that all day.
So yes. AI will be wrong sometimes, and it will be confidently wrong.
Good. That part is fine. Plan for it.
Stop asking whether your AI will be wrong. Ask what catches it.
Your body figured this out a long time ago.
Right now, this second, your immune system is fighting off things you will never hear about. It catches threats all day, every day, in the background, without a meeting. You do not thank it. You do not notice it. You only notice the week it fails.
That is what good catching looks like. Quiet. Constant. Built in.
Part one of this series was the Skeleton. Your data.
Part two was the Circulatory System. Your context.
Part three was the Muscles. Your tools.
Part four was the Central Nervous System. The wiring.
This is part five. The Immune System. What catches it when the machine gets it wrong.
TWO WAYS TO FAIL
Most companies fail this in one of two ways, and both come down to risk tolerance.
The first group gets so excited about the AI that they just let it rip. They wire it into support or billing and nothing checks the output. Wrong answers ship straight to customers. Nobody finds out until somebody is angry enough to call. By then it has happened fifty times.
We were excited too. Here is what we did instead, and we gave it a name. See it, use it, love it.
At the start, we see every response before it goes anywhere. If it is good, we use it. If it is a little off, we edit it and that edit goes back into the system. And eventually we get to the stage where we love it, and we stop checking.
I call that the look ma, no hands approach. Think about learning to ride a bike. You were scared and wobbly and somebody was holding the seat. Then one day you are flying down the street with your hands in the air. Nobody handed you the bike and said good luck. And nobody held that seat forever, either.
The second group has the opposite problem, and it looks responsible. They are so afraid of the machine being wrong that they will not let go of anything. It has to be 100 percent right before it gets to run on its own. So every output needs a sign-off. Then a second one. Then a committee.
Think about what we accept from people. You hire someone good, and it takes months, sometimes quarters, before they have really mastered the work. They get things wrong the whole time. And nobody stands behind them correcting every single thing they do, because that would be insulting and it would not work anyway.
We give a new hire a year to get good. We give the machine an afternoon.
Either way, whether you let it rip or never let go, you need the immune system.
THE PERSON WHO LEAVES
Here is the pattern we use.
Every new workflow starts with a person in the loop. Sometimes several people.
We have spent the last six months building our AI renewal system. We meet every Thursday to go through everything. And we have somebody whose actual job is to stay on top of it and course correct, every single day.
Our renewals system has a name. Fiona. And Fiona now catches her own mistakes, flags them, and writes her own improvements.
That is not a day one thing. It took about six months to get there.
One of our teammates just finished this exact arc. He started with renewals under 100k on our smallest products. Then all the products at that size. Now he is working on renewals over 100k, and he is running the same climb again from the beginning.
That is the shape of it. The person in the loop should be working themselves out of the loop from the first day they are in it.
If your human in the loop has been checking everything for a year, you do not have an immune system. You have a hamster running on a wheel that is going to be stuck there forever. You need to get that hamster off the wheel.
TRUST COMES IN COHORTS
Watch what happened to our numbers.
Our AI renewals team started out converting 9 percent better than our human team. Then 14.7 percent. As of today, 19.8 percent better.
That is not one good number. That is a trend, and a trend is what you make decisions on.
So we made the call. Starting January of 2027, every renewal runs through the AI renewal system. Any size. Any contract value.
And we are still five or six months away from that date. On purpose. We keep expanding the cohorts and watching what happens, because a number only tells you the machine is better on the cases you have already seen.
THE MEMORY IS THE POINT
Here is what separates an immune system from a review process.
When your body beats an infection, it builds antibodies. It does not just survive the thing. It comes out of the fight with a permanent defense against it. That is why chickenpox gets you once.
For us, the antibodies are knowledge bases. We keep extensive ones and we are constantly adding to them. New rules. New examples. Every correction somebody makes on a Thursday goes in.
Most companies catch an AI mistake and fix the answer. Then the machine makes the same mistake next week, and they fix it again. Forever.
Fix the answer and you fixed one ticket. Fix the knowledge base and you fixed every ticket like it.
If catching a mistake does not make the next one less likely, you are not building a self-improving system. You are building something you have to maintain forever.
AND BE HONEST ABOUT IT
We still meet every Thursday. We still find problems.
Are there fewer? Absolutely. But they are new ones. Different ones. As we move into more sophisticated workflows, the problems change shape.
And here is the funny part. Sometimes in those Thursday meetings we end up debating what the right answer even is, because we never documented it properly. That is exactly why the machine got it wrong in the first place. It did not have an answer to follow, so it made one up.
That has turned into one of the most useful things this whole project has done for us. It forces us to clean up our own thinking and kill off the tribal knowledge that turned out to be wrong.
So no, the 80 percent does not mean the machine is right and we shrug at the rest. It means 80 percent flows through with the immune system watching, and the hard ones go to people. We drop things too. The system exists because we are not perfect, not because we are.
Anyone who tells you their AI does not need watching is either lying or about to make a very expensive mistake in public.
Kathy
THE AI BOSS
QUESTIONS PEOPLE KEEP ASKING
What happens when AI gets something wrong in my business?
It ships a confident wrong answer, and the damage depends on what catches it. But ask the other half of that question too. What happens when your people are wrong? Humans get things wrong all day and we build for it. AI is actually more consistent once you have trained it properly, because it does the same thing every time. Plan for wrong either way, with detection built in and a record so the mistake does not repeat.
How do I know when to trust AI with more work?
Earn it in stages. We call it see it, use it, love it. First you check every response before it goes out. Then you use what is good and edit what is off, feeding those edits back in. Eventually you trust it enough to stop checking. Nobody learns to ride a bike with no hands on day one.
Do I need a human in the loop for AI?
Yes, at the start of every new workflow, and sometimes several people. But their job is to work themselves out of the loop. If somebody has been checking everything for a year, you do not have a safety system. You have a hamster on a wheel, and you need to get that hamster off the wheel.
How do I know when to remove the human from the loop?
When the numbers earn it and the trend holds. Our AI renewals went from 9 percent better than our human team, to 14.7 percent, to 19.8 percent. One good number is luck. A trend is a decision. The system should also be flagging its own uncertain cases before you step back.
What is the biggest mistake companies make with AI oversight?
Picking one of two extremes. Some are so excited they let it rip with no checks, and wrong answers ship until a customer complains. Others are so afraid of being wrong that they demand 100 percent before they let anything run, which kills the speed they bought AI for. We give a new hire a year to get good. We give the machine an afternoon.
How do I stop AI from making the same mistake twice?
Put the correction in a knowledge base, not just in the answer. Rules and real examples, updated constantly. Fix the answer and you fixed one ticket. Fix the knowledge base and you fixed every ticket like it. If your AI keeps repeating an error, nobody is feeding the corrections back in.
THE BUSINESS BODY SERIES
Part One. The Skeleton. Everyone Wants Beach Muscles. Nobody Thinks About Dense, Durable Bones. Your data, and why your AI keeps guessing.
Part Two. The Circulatory System. Your AI Does Not Know How You Think. Context, your second brain, and what a guardrail really is.
Bonus. Build Your Second Brain. Let Claude Interview You. One hour. Five steps. Do it today.
Part Three. The Muscles. Your AI Strategy Is a Gym Membership You Are Not Using. Why buying the tools does nothing.
Part Four. The Central Nervous System. Even the AI Boss Does Not Take Her Own Advice Sometimes. Own the wiring, rent the tools.
Part Five. The Immune System. You are reading it.
Next up. The Brain. The finale. What stays human when the machines do the work.