Part 1 ended at a line. Below it the agent computes: it moves data across the messy seams and applies the rule you already wrote. Above it a human decides. The whole architecture rests on one assumption, which I quietly skipped: that something can tell, case by case and at speed, which side of the line a case belongs on.
Ask a team how they plan to do that and you get the same answer every time. I have heard it in every design review I have sat in, usually from the calmest person in the room.
The room relaxes, because this one sounds like engineering. It has a mechanism in it. There is a confidence score, there is a threshold, someone will tune it.
It is the same sentence from the intro in a technical costume. Common sense will be applied, by the model, on request. And it fails in a specific direction that makes it worse than having no filter at all.
Confidence measures the wrong thing
A model's confidence is a statement about text. It is high when the input looks like the millions of inputs the model has seen and the answer follows a well-worn path. It falls when the phrasing is strange, the formatting is broken, the message is in a language the model rarely handles, or the customer rambles for nine paragraphs before getting to the point.
Read that list again. None of it is about money. None of it is about trust, or precedent, or whether this action can be undone tomorrow.
A furious customer writing in fragments about a €9 shipping fee reads as unclear. The agent stops, and a human spends four minutes on nine euros. A polite, well-structured message that happens to be the sixth fraudulent refund request from the same person this year reads as perfectly clear, because it is perfectly clear. It is fluent, ordinary, on-policy, and wrong. The model has no doubt to report, so it reports none.
The agent stops on the cases that are hard to read. It should stop on the cases that are expensive to get wrong. Those are different cases, and they are barely correlated.
Difficulty and danger are two separate axes. "Stop if unclear" filters on the first one, and then bills you for the second.
Both errors cost you, and they compound each other. The escalation queue fills with the bottom-right box: noise, low stakes, resolvable in seconds. The human working that queue learns, correctly, that flagged cases are usually nothing. Meanwhile the top-left box never reaches her at all. She is being trained to skim by the same mechanism that hides the cases worth reading.
Go back to the two cases from the intro, the ones that arrive looking identical. Case A is a first-time customer who genuinely forgot to cancel. Case B is the sixth forgotten trial this year across three accounts, same wording every time. A confidence gate treats them the same way, and it is not wrong to: the text really is equally clear in both. What separates them is not in the message. It is in the history, and nobody asked.
The supervisor trap
So you add a second model to check the first. Almost everyone lands here, and the appeal is obvious. It is cheap, it runs in milliseconds, it needs no headcount, and it emits a label you can put in a dashboard: verified.
The second model is trained on the same kind of data as the first, carries the same priors, and reads the same text. So it is not an independent check. It disagrees with the primary exactly where the primary was already hesitant, which is the noise, and it agrees with the primary exactly where the primary was confidently wrong, which is the damage. You did not add a check. You added a second vote from a voter with the same opinions.
Then there is the second-order effect, which is worse and slower to show up. The verified label is read by every human downstream as checked. It lowers the attention of the only genuinely independent reviewer in the system. A supervisor that catches thirty percent of errors while cutting human scrutiny in half has made you less safe and given you a metric that says otherwise. That is not a hypothetical to argue about. It is a ratio you can measure, and you should, before the label ships.
None of this makes a model watching model output useless. It is good at a job I have written about elsewhere: catching drift, tone breaks, policy language that has quietly wandered over six weeks. That is pattern work on the agent's own output, and a model does it well because the signal genuinely is in the text. What it cannot supply is independent judgment about this customer, because it is blind in the same places the primary is blind.
Escalate on the case, not on the model's mood
The fix is to stop asking the agent how it feels and start asking the case what it is. Every useful escalation trigger I have seen is deterministic, computed from data the agent does not generate, and readable by a human in one line.
Reversibility. Can this be undone in one click tomorrow, or does it move money, close an account, or send an email that cannot be recalled? Magnitude. How does the amount compare to the ordinary amount for this product, this segment, this week? Pattern. Has this customer, this device, this address, or this payment method done this before? Novelty. Does this case resemble anything already decided, or is it genuinely the first of its kind? Value at risk. What is this relationship worth, and what does the wrong answer here cost beyond the transaction?
None of those require a model. All of them are auditable, and every one of them has a threshold that a named person sets.
Notice who owns this. The thresholds are not a technical setting, and the engineer tuning the pipeline should not be choosing them alone. Reversibility, magnitude, value at risk: these are the parameters from Part 1, and they belong to the parameter owner. This is the line from the anatomy, written down in numbers instead of gestured at in a meeting.
It also changes what kind of quantity escalation volume is. Under a confidence gate, the number of cases reaching humans is an output: whatever the model happened to find strange this week, drifting silently every time the model is updated. Under deterministic triggers it is an input. You set it. Tighten the thresholds and more cases reach humans, cost rises, risk falls. Loosen them and the reverse. That is a budget, and a budget has a price per unit, which is the number this series ends on.
Keep the confidence score. Just take the gate away from it and give it a smaller job: sampling. Low-confidence cases make a reasonable pile to audit, because they cluster around inputs your process handles badly. As a queue for review, useful. As the thing standing between an agent and a customer's money, no.
What this costs you
Two things get worse before they get better, and pretending otherwise is how this advice gets ignored by month three.
First, somebody has to write the triggers down. "The agent will escalate when it's not sure" requires nobody to define anything, which is the real reason it wins design reviews. Deterministic triggers force the parameter owner to say what counts as consequential, in numbers, on paper, with their name on it. That is the discomfort from the end of the series arriving early and in a smaller size.
Second, a trigger list only catches shapes of trouble you already know. The genuinely novel case, the fraud pattern nobody has seen, passes every rule you wrote and every confidence check as well. There is no gate clever enough to fix this. The mechanism that finds it is the audit wire from Part 1: a random sample of resolved cases, read by a human, selected without reference to any score. Unconditioned sampling is the only way you learn what your rules fail to describe. It looks like waste on a dashboard. It is how the trigger list gets written.
What comes next
So the routing is deliberate now. Cases reach humans because of what they are, not because a model found the grammar unusual, and the humans above the line get the dozen that carry the weight.
Which creates the next problem, and it is a human one rather than a technical one. Give a person a queue of flagged cases and an approve button, and within three weeks the button is faster than reading. The human in the loop becomes a human on the loop, and then becomes a signature. Accountability on the org chart, rubber stamp in practice, and a worse outcome than having no reviewer at all, because now everyone believes the case was read.
The fix is the opposite of what volume suggests: send humans fewer cases, not more.
That is Part 3.