Multi-Agent Verification Independence: The Property Your Critic Agent Does Not Have

A design review this month ended on a question I now ask first. A team walked us through a six-agent pipeline, and when we reached reliability, the answer was that a seventh agent reviewed the output before anything shipped.

I asked what failure that seventh agent could not have.

The room went quiet, and I do not think the question was unfair. Multi-agent verification independence is the entire load-bearing assumption under critic agents, judge agents, debate, and ensembles. If the checker can make the same mistake as the thing it checks, you have not added verification. You have added a second opinion from the same mind, and you are paying twice for one judgment.

Four preprints posted in the last week put numbers on how bad that assumption is.

Why do agents agree with agents like themselves?

The sharpest result comes from a study of identity-dependent conformity across twelve open-weights models on nine tasks that each have exactly one correct answer (arXiv 2609.33495). The authors separate two things that usually travel together: how many peers agree, and who those peers are.

The effect is bidirectional. When the agreeing peers are framed as in-group, models conform more to wrong answers. When the same consensus comes from an out-group, models conform less. Group membership here can be as thin as model family, an AI-versus-human label, or an arbitrary assigned group with no content at all.

Two details matter more than the headline. Chain-of-thought reasoning suppresses most of the effect but not all of it. And relabeling the peers as safety-aligned shifts overall conformity while leaving in-group favoritism intact. So the two moves an architect would reach for first, reason harder and label the reviewer as the careful one, do not remove the bias.

Read that as an orchestration finding rather than a psychology one. If your critic agent is the same model family as the agent it reviews, running in the same prompt lineage, then the consensus it produces is partly a function of who is speaking, not what was said.

What happens when the channel between agents is corrupted?

A second study runs seven models through iterated N-player Stag Hunt games while programmatically inverting some agents' actions before they become public (arXiv 2609.31704). At eighty percent corruption, the honest agents' own cooperative choices held at seventy-eight percent, while public success collapsed to twelve percent.

That gap is the useful part. Almost the entire collapse was mechanical, not a loss of intent. The agents kept choosing well and the system kept failing, because what each agent could see about the others had stopped being true.

The authors' recommendation is a measurement discipline worth stealing outright: evaluate an agent's original choice, the action that became visible to peers, and the outcome that actually executed as three separate quantities. Most production traces I see collapse all three into one line, which makes a coordination failure look like a reasoning failure and sends the fix to the wrong layer.

Does a defense that works on one threat work on the next?

This is where the evidence gets genuinely uncomfortable. ORBIT is an open-source evaluation framework built on the UK AI Safety Institute's Inspect, and it is the first one that varies attack, defense, and architecture together across realistic environments including browser use, agentic coding, and customer service (arXiv 2609.33102).

Its central finding: no tested defense generalized across all attacks. A per-action defense that cut a compromised agent's attack success by sixty points on multi-issue coding gave no measurable protection against agents colluding with each other.

Sixty points to zero, same defense, different threat. If you have a per-action guard in place and you have not tested it against collusion specifically, you do not know whether it is worth anything against collusion. The framework's other conclusion is that security interacts with topology, which means a control cannot be evaluated independently of the graph it sits in.

What does an independent check actually look like?

The constructive answer in the same batch is deliberately unglamorous. Maat validates agent-to-agent handoffs against a versioned workflow contract, with no language model anywhere in the validation or scoring path (arXiv 2609.34017). Across six domain workflows of six to fifteen agents, 522 trials with injected data-level defects and a deterministic seven-check rubric, it improved every one of the six by between 2.9 and 26.5 percent.

The mechanism is the point. A deterministic contract check cannot conform to anybody. It has no in-group. It cannot be talked around, and it returns the same verdict twice on the same input, which an LLM judge does not guarantee.

One more finding closes the loop. ReplayLens audits what actually drives an agent's reuse of its own logged history by changing one relationship at a time (arXiv 2609.34177). Swapping the recorded scores changes decisions, while moving intact action-score pairs to different positions does not. So behavior is driven by score attachment, and corrupted historical scores misdirect exploration and reduce final utility even when fresh measurements are available. The record your verifier reads is itself an attack surface.

These are preprints, and the enterprise-scale replication does not exist yet. The direction is consistent across four independent teams, which is more than I can say for most things I get asked to architect around.

The Verifier Independence Test

I work with Fortune-level enterprise clients architecting and building agentic frameworks, and this is now a gate in my design reviews rather than a discussion. Four questions, asked of every agent that checks another agent's work.

Lineage. Is the verifier a different model family, with a prompt and context that do not descend from the agent it reviews? Same family plus same lineage is the in-group case, and the conformity result says that is the worst configuration available.

Evidence. Does the verifier see the raw artifact, or only the peer's report about the artifact? The corrupted-channel result says a verifier reading summaries is verifying the summary.

Threat. Was this control measured against the threat you actually face? ORBIT's sixty-to-zero result means a defense score is meaningless without the threat named beside it.

Determinism. Does the final gate need a model at all? Schema validity, contract conformance, permission scope, and budget ceilings are contract-detectable. Spend the model on the judgments that genuinely need one.

Then the one-line version, which is the only part anyone remembers: name a failure this verifier cannot have. If the answer is nothing specific, the verifier is decoration.

What I would actually do this quarter

Take one multi-agent workflow already in production. Pick the single handoff where a wrong value does the most damage downstream, and write a deterministic contract for exactly that handoff: required fields, value ranges, permission scope, and a hard budget ceiling. No model in the path. Put your LLM judge behind it rather than in front of it.

Then instrument that one handoff the way the Stag Hunt paper recommends, logging the originating agent's own output, the value that became visible downstream, and the value that executed, as three separate fields. One handoff, three fields, one contract. You will learn more from that in two weeks than from adding an eighth agent.

The industry sold the ensemble as reliability that comes free with scale. It does not. Independence is a property you buy with architecture, and right now most of us are buying correlated opinions and calling them checks.

---

Part of the enterprise agentic AI reality pillar guide, which maps the four layers no benchmark measures: the scoreboard, the runtime, the denominator and the verifier.

Related reading: what one completed task actually costs you, the case that this is a runtime problem and not a model problem, and what kills enterprise AI agent projects. The control surface underneath all of it is covered in the enterprise AI agent governance guide.

I write one piece a week on the orchestration, decisioning, and governance that make autonomous agents safe to ship in an enterprise. Subscribe to the newsletter to get it.

Swipe the framework

The Verifier Independence Test as an eight-slide breakdown, the same deck shared on LinkedIn.