Drift ·
The authority nobody wrote down
An agent can use every tool it was given and still do something no one approved.
A useful example has been going around. An agent is asked to investigate why a customer's renewal is at risk. It reads the support tickets, checks usage, drafts a recommendation — all of it sensible — and then emails the customer a 40 percent discount. It had the CRM. It had email. It understood the goal. It may even have saved the account.
Nobody gave it the authority to negotiate pricing.
That framing is right, and the part worth sitting with is that nobody wrote that authority down. Not through carelessness. A specification of what an agent must not do is a specification of everything it might think of, and that is not a document anyone can finish. The human analogy holds only because a salesperson arrives carrying years of unwritten context about what is not theirs to decide. An agent arrives with none of it, and we hand it the same tools.
So there is a gap between the authority you meant to grant and the authority you managed to express. The interesting failures live in that gap, and they are not access-control failures. Every permission check passed.
The obvious repair does not escape the problem. "Escalate anything above 20 percent" is a reasonable rule, and it is also the pricing policy nobody wrote, finally written down. It covers the failure you already imagined. The next one arrives as a refund, or a contract term, or a support credit, and the rule has nothing to say about it. An escalation threshold is a specification too. You have moved the problem up one level, not solved it.
What survives is not a rule at all. That agent, in the example, had touched pricing before. Say forty times, always somewhere between 8 and 12 percent. The forty-first was 40. You do not need a discount policy to know that is worth a human — you need the previous forty. The deviation is legible even where the boundary was never drawn, which is exactly the property you want when the boundary is the thing you could not write.
This is not a new idea in security. It is the oldest one there is: baseline the normal, flag the departure. What makes it awkward for agents is that the departure lives in an argument, and arguments are model output.
Here is the constraint that makes it interesting. Recording that apply_discount fired is easy and tells you nothing — the category is normal, the magnitude is not. Recording percent: 40 means writing part of the model's response to disk, and for us that is not a tradeoff to weigh. Prompt and response content is never persisted. It is the architecture, not a setting.
Both can be true at once, and the resolution is less clever than it sounds. Numeric arguments are read in memory, folded into a running distribution kept per agent, per tool, per argument, and discarded. What is stored is a mean, a variance, and a count — aggregates over history, the same class of thing as every other behavioural dimension, and not invertible back to any sample. What leaves is one number:
You keep that it was abnormal, and how much history that judgement rests on. You do not keep the number.
Two things we got wrong first, because they are the parts that generalise.
The obvious way to tell a magnitude from an identifier is statistical — surely an order_id spreads differently than an amount. Measured, it does not. A uniform identifier sits near 0.58 on coefficient of variation; a perfectly legitimate amount ranging from 100 to 10,000 clears 2.0. There is no distribution test that separates them, because both are simply numbers with spread. We excluded identifiers by name instead, from a short auditable list — which is less elegant and honest about what it is.
The second: an agent that has passed exactly 10 fifty times in a row has zero variance, and a deviation measured against zero variance is undefined. The first implementation returned nothing there. That is precisely backwards — a value that has never once changed, changing, is the strongest signal available. It now floors the deviation rather than discarding it.
And it stays quiet until it has grounds. Below about twenty observations no deviation is emitted at all. A distribution of three samples has nothing to say, and saying it anyway is how a detection product teaches people to ignore it.
What this does not do is tell you the agent was wrong. Deviation says something changed. A sales agent legitimately closing a larger deal than it ever has looks, on the first pass, identical to one that has quietly decided it can negotiate. That second judgement is a genuinely open problem, and we do not treat a threshold as the answer to it — a deviation opens an argument between adversarial reviewers before anyone is paged, which cuts the noise substantially and is not the same as solving it.
The practical shift is smaller than it sounds. Alongside did it achieve the goal and did it stay inside its authority, there is a third question that does not depend on anyone having written the authority down in advance: what did it actually do, compared to everything it has done before?
That one you can answer today. Without keeping a single customer's discount.