Multi-Agent Consensus Is Not Correctness: How Debate Manufactures Confidence

Five agents independently examine a decision. Three find weak evidence for option A. One finds strong evidence for option B. The fifth is uncertain.

Then they debate.

After several rounds, all five choose A. The system reports unanimous agreement and raises its confidence. But the discussion did not add decisive evidence. It repeated the majority’s framing until the dissenting agent adopted it.

The system now looks least uncertain at the moment its independent error-correction mechanism has disappeared.

That is the central finding of When Consensus Is Not Correctness, presented in the August 9, 2026 program of the KDD Workshop on Evaluation and Trustworthiness of Agentic AI. Inter-agent agreement is not automatically evidence of truth because the debate itself produces the agreement.

Two companion papers make the engineering stakes larger. How Much Coordination Gain Is Real? shows that small multi-agent improvements can sit inside a surprisingly wide local noise floor. Equilibrium-Level Harm in Multi-Agent AI Systems shows how individually acceptable agents can produce a collectively harmful stable outcome after they interact.

Evidence outside the workshop points in the same direction. When LLMs Agree, Are They Right? analyzes 265,000 samples across GPQA Diamond and AIME. Agreement was a positive but regime-dependent correctness signal, with reported correlations of 0.20–0.59. In the most consistent frontier GPQA regime, agreement was at least 0.8 for 77% of case-result entries, yet 48% of those entries were wrong. The paper’s conclusion is appropriately narrow: self-consistency is a conditional proxy, not a standalone confidence score.

This is not an argument that communication is harmful. A 2026 study of competitive, consensus-seeking, and collaborative debate reports that a truth-seeking collaborative protocol beat conventional debate and single-agent methods under matched token budgets on its evaluated tasks. ACL 2025 experiments likewise show that voting, consensus, and discussion depth produce materially different results; additional discussion rounds sometimes reduced performance.

The problem is not communication itself. The problem is treating communication-induced convergence as verification. Protocols that reward new evidence, contradiction discovery, and truth-seeking can outperform both naive consensus and isolated reasoning.

Together these results overturn a common architecture shortcut:

More agents do not create independent verification if they share evidence, imitate one another, optimize the same proxy, or stop when they agree.

This post develops a safer pattern: independent evidence first, bounded deliberation second, failure-mode-independent certification third, and one accountable runtime decision at the end. Its larger thesis is architectural: multi-agent systems need a trust architecture that treats deliberation as an untrusted proposal mechanism. It reviews new research, not settled universal law. Reported numbers belong to the cited protocols, and every production inference should be tested on the target workflow.

Agreement is useful only to the extent that it contains independent information

Ensembles do not require strict independence. They can still help when members are positively correlated; the averaging benefit simply shrinks as error covariance rises. If several estimators fail for sufficiently different reasons, aggregation can cancel some of that error.

Multi-agent debate changes the assumption. Agents read one another’s claims, evidence summaries, confidence, and rhetoric. Their outputs become statistically dependent.

The consensus paper models the effect as rising inter-agent correlation. As debate progresses:

• answers become more similar;
• visible disagreement falls;
• the panel appears more confident;
• the errors stop averaging out;
• the remaining consensus can still be wrong.

The paper calls this diversity collapse. Agreement is endogenous: it is an output of the process being used as evidence about the output.

That creates a measurement bug. A dashboard sees falling disagreement and interprets it as rising certainty, but the same interaction has removed the independence that would make agreement informative.

In the paper’s controlled study across 15 tasks and five model families, agreement-based stopping produced confident errors with 18–47% miscoverage. Its reported calibration gap tracked residual error closely. The authors propose calibrated multi-agent debate with prevention, detection, and a split-conformal certificate under stated exchangeability assumptions.

Effective Agent Count: nominal agents are not independent evidence sources

The paper’s equicorrelation identity suggests a useful diagnostic. Match the panel’s aggregate variance to the variance of an idealized independent panel and define Effective Agent Count (EAC):

N_eff ≈ N / (1 + (N - 1)ρ)

Here N is the nominal number of agents and ρ is their approximate error correlation within a declared evaluation slice. For five agents:

EAC is a ContextOS implementation inference from the paper’s variance identity; the paper does not propose or validate this named metric. It is a variance-equivalence diagnostic under the equicorrelation lens, not a universal count of intelligence or capability. Estimate it offline from repeated, labeled traces using correctness or error outcomes, report it per slice with uncertainty, and retain the full error-correlation matrix when lanes are heterogeneous.

Online traffic usually lacks immediate ground truth. Evidence overlap, retrieval overlap, shared model family, rationale similarity, and counterfactual message influence can warn that dependence is rising, but they are collapse proxies—not estimates of ρ and not substitutes for label-backed EAC.

The general engineering lesson is simpler:

Do not use a signal created by social convergence as if it were an independent measurement of correctness.

Consensus failure has two immediate consequences

“The agents agreed” can immediately conceal an answer failure or an evaluation failure. These are related, but they require different evidence.

A. Epistemic failure: agreement without independent evidence

Agents begin with different evidence or hypotheses, then converge through copying, anchoring, authority bias, or repeated exposure. The minority evidence is lost even though no new fact defeated it.

Observable symptoms:

• later answers reuse the same rationale and omissions;
• citations converge before the underlying evidence is reconciled;
• confidence rises while evidence coverage does not;
• dissent disappears after reading a high-status agent;
• counterfactual removal of the majority message changes every lane.

B. Evaluation failure: apparent coordination gain without adequate controls

Two multi-agent protocols produce different outcomes even when the coordination mechanism is inactive or the API inputs are configuration-equivalent. A small score increase is then attributed to the architecture.

The paired noise-floor paper compares coordination protocols on Claude Haiku 4.5 and τ²-bench Retail. Across two n=100 seeds, its inert configuration-equivalent contrast pooled to +5 percentage points with a Wilson interval of [-2, +12], not statistically significant. The observed paired-gap envelope ranged from -3 to +18 points across seeds. Seven of ten recent coordination architectures reviewed by the authors reported headline effects below that local floor, with another inside the envelope.

That does not disprove those architectures. It says their claimed gain was not tested against this kind of same-model paired replication.

A deeper problem: interaction can create new failures

Agents can pass isolated tests and still interact into a bad equilibrium.

The equilibrium-harm paper gives a narrow synthetic proof-of-mechanism. One agent passes a coverage audit for the smallest group. Four agents optimizing in competition converge to a stable profile in which that group falls below the coverage threshold. The objective did not change; the interaction structure did.

The paper identifies three risk predictors:

• correlated objectives such as engagement, conversion, throughput, or rank;
• constrained shared resources such as attention, liquidity, review capacity, or bandwidth;
• skewed outcomes with demographic or stakeholder correlation.

In those settings, testing every agent alone is structurally incomplete. The harm lives in the game.

The progression matters: correlated evidence can produce the wrong answer; weak controls can produce the wrong measurement; strategic interaction can produce the wrong system equilibrium.

Role prompts do not guarantee independence

“You are the optimist,” “you are the skeptic,” and “you are the judge” can diversify surface language while leaving the underlying errors correlated.

A controlled logical-reasoning study found intrinsic reasoning strength and group diversity to be the dominant drivers of debate success, while debate order and confidence visibility produced limited gains. Role diversity may change output diversity. It does not, by itself, demonstrate that the lanes fail differently on the cases that matter.

Role prompts fail as an independence mechanism when agents share:

• the same model family and decoding defaults;
• the same context pack and missing evidence;
• the same retrieval ranking;
• the same tool outputs and stale state;
• the same reward proxy;
• the same parent summary;
• the same judge;
• the same prior agents’ answers.

A skeptic who sees only the proposer’s evidence is not an independent reviewer. It is a differently prompted reader of the same evidence boundary.

Useful diversity must exist in the experiment:

None guarantees independence. Each makes dependence more measurable.

The safe sequence: isolate, deliberate, preserve, certify, decide

Use four evidence phases with different contracts, then one accountable runtime decision. The trust architecture is explicit:

• agents generate hypotheses;
• evidence supports or defeats claims;
• deliberation discovers contradictions;
• evaluation estimates uncertainty and dependence;
• policy constrains effects;
• certification controls whether a claim meets its declared evidence guarantee;
• the DecisionRecord establishes accountability;
• the runtime—not the crowd—owns execution.

Phase 1: isolate the first pass

Give each lane a declared question, evidence boundary, tool budget, and output schema. Do not expose peer conclusions before the lane commits its initial artifact.

Record:

• claim;
• evidence references;
• unresolved assumptions;
• confidence with a defined meaning;
• conditions that would falsify the claim;
• recommended action or abstention;
• model, context, tool, and budget identity.

The first-pass artifacts are the independence baseline. Never overwrite them after debate.

Phase 2: deliberate over evidence, not status

Deliberation should target named contradictions and missing evidence. Do not run a free-form “debate until consensus.”

For each disputed claim, require:

1. the evidence supporting each position;
2. the specific incompatibility;
3. a retrieval, test, or state check that could resolve it;
4. a bounded number of rounds;
5. a terminal state: resolved, unresolved, or not material.

An agent changing its mind is not itself evidence. The record must show which new observation caused the change.

Phase 3: preserve the minority report

Consensus must not delete dissent.

The final deliberation artifact should retain:

• the initial vote distribution;
• the post-deliberation distribution;
• the claims that changed;
• new evidence introduced during deliberation;
• unresolved minority evidence;
• dependence warnings, such as shared sources or shared model family;
• the cost and latency of reaching the terminal state.

A minority claim can be wrong and still be valuable. It identifies the uncertainty the majority may have socially compressed.

Phase 4: certify outside the debate

The final action gate should not ask the debating agents whether their debate was correct.

External means failure-mode independent, not merely separately invoked. A verifier that shares the panel’s model family, evidence boundary, retrieval path, stale state, or evaluation rubric may remain correlated with the failure it is expected to detect. Four independence dimensions should be recorded:

A separate agent call may satisfy none of these. A verifier need not be independent on every dimension, but its shared failure domains must be declared and tested.

Conformal coverage is one certification mechanism for suitable prediction problems under an appropriate calibration population, score construction, and exchangeability assumptions. It is not a universal correctness proof for arbitrary agent trajectories. State-changing workflows additionally require policy, authority, state, integrity, idempotency, and recovery invariants.

Use controls that do not depend on their agreement:

• deterministic schema and policy validation;
• independent evidence and freshness checks;
• read-only precondition checks against current state;
• calibrated held-out evaluators;
• a domain reviewer for high-consequence ambiguity;
• abstention or escalation when required evidence is unresolved.

For a state-changing workflow, correctness is not “four agents approve.” Correctness is that the proposed effect satisfies policy, authority, evidence, idempotency, state-integrity, and recovery contracts at the action boundary.

One runtime must still own the decision

Multi-agent participation does not remove accountability.

Workers can produce findings. Reviewers can veto or request evidence. A deliberation process can synthesize positions. But one parent runtime must emit the final DecisionRecord and own whether the system acts, abstains, or escalates.

The parent should record why it accepted the panel’s result and which controls remained independent of the panel.

An illustrative, non-normative record:

deliberation_record:
  question: approve_supplier_change
  initial_findings:
    - lane: policy
      verdict: deny
      evidence_refs: [policy:vendor_change:8.2]
    - lane: operations
      verdict: approve
      evidence_refs: [ticket:4821, runbook:onboarding:4.1]
    - lane: fraud
      verdict: uncertain
      evidence_refs: [account_graph:result_91]
  shared_dependencies:
    model_family: [policy, operations]
    evidence_snapshot: [policy, operations, fraud]
  deliberation:
    rounds: 2
    changed_positions: [operations]
    new_evidence_refs: [registry:beneficiary:result_17]
    unresolved_minority: "ticket requester identity is not independently verified"
  certification:
    policy_gate: deny
    state_precondition: pass
    human_review: required
  parent_decision: escalate

The important field is not the vote. It is certification.

Replace consensus stopping with evidence stopping

“Stop when all agents agree” optimizes directly for the failure the research identifies.

Safer terminal conditions are:

• every material claim has an eligible evidence reference;
• required independent source classes have been covered;
• named contradictions are resolved or explicitly escalated;
• additional rounds have not changed supported claims on a validation set;
• the next round exceeds its expected information value or budget;
• the external verifier passes;
• the policy gate allows the proposed effect;
• the system abstains because the evidence threshold cannot be met.

Agreement may be reported. It should not be the permission primitive.

The multi-agent calibration scorecard

Before shipping a debate, voting, reviewer-panel, or society-style system, measure all of the following:

Run paired seeds and publish uncertainty. If an improvement sits inside the inert or configuration-equivalent noise floor, call it inconclusive.

Production release gates

The research does not justify an arbitrary calendar. It does justify four gates that every candidate must clear.

Gate 1 — Establish causality

• compare against single-agent, independent-N, compute-matched, and inert-coordination baselines;
• use paired tasks and repeated seeds;
• report uncertainty, cost, latency, wrong-confident outcomes, and human rescue;
• reject claims whose improvement remains inside the local noise envelope.

Gate 2 — Measure dependence

• commit initial findings before peer exposure and retain them after deliberation;
• estimate the label-backed error-correlation matrix and EAC by evaluation slice;
• monitor evidence, retrieval, model-family, and rationale-overlap proxies online without presenting them as ground-truth correlation;
• trace counterfactual message influence, minority rescue, and minority erasure.

Gate 3 — Separate proposal from certification

• document the verifier’s evidence, method, authority, and state independence;
• enforce evidence-based stopping, abstention, deterministic policy, and current-state gates;
• test wrong-unanimous cases and correlated-verifier failures;
• require one parent runtime to own the final DecisionRecord and any effects.

Gate 4 — Adversarially test interaction

• replay cases where a valid minority must overturn an incorrect majority;
• test correlated objectives, shared-resource contention, persuasion incentives, and harmful equilibria;
• measure stakeholder and system outcomes, not only per-agent success;
• canary only where the multi-agent path beats simpler baselines on certified outcomes under the same operating budget.

The longer-term research roadmap is clear: better dependence measures for model panels, calibration certificates under realistic non-exchangeable traffic, equilibrium tests for cross-vendor agents, attribution across compressed handoffs, and benchmarks that price minority preservation as a safety feature rather than conversational inefficiency.

Where multi-agent coordination still helps

This research is not an argument for isolated agents forever.

Communication can improve coverage on decomposable, interdependent tasks. AgentRadio reports a controlled gain from passive cross-lane awareness on codebase understanding, while also documenting higher cost and some rubric-level regressions. The correct response is not to suppress coordination. It is to make its influence bounded and measurable.

Use multi-agent deliberation when:

• lanes can access meaningfully different evidence or methods;
• the cost of missing a contradiction is high;
• worker artifacts are independently verifiable;
• the parent can preserve provenance and replay every influence;
• a verifier outside the panel controls authority;
• the gain survives single-agent, independent-N, compute-matched, and inert-coordination comparisons.

In ContextOS terms, deliberation is an untrusted proposal process inside the Decision plane. The Trust plane owns evaluation, policy, certification, approval, and release control. The Action plane executes only through the Tool Gateway after those controls pass. The crowd can inform the decision; it does not own permission.

Otherwise, several agents repeating one another are not a team. They are one correlated error with a larger token bill.

Research base

• When Consensus Is Not Correctness: Diversity Collapse and Manufactured Overconfidence in Multi-Agent LLM Debate, KDD 2026 workshop paper.
• How Much Coordination Gain Is Real? A Paired Noise-Floor Protocol for Multi-Agent LLM Benchmarks, KDD 2026 workshop paper.
• Equilibrium-Level Harm in Multi-Agent AI Systems: Why Individual Evaluation Fails, KDD 2026 workshop paper.
• KDD Workshop on Evaluation and Trustworthiness of Agentic AI 2026, August 9, 2026 program.
• When LLMs Agree, Are They Right? Auditing Self-Consistency and Cross-Model Agreement as Confidence Signals, arXiv:2607.08065v2.
• When and Why Does Multi-Agent Debate Fail and Does It Really Underperform?, collaborative-debate protocol and matched-budget experiments.
• Voting or Consensus? Decision-Making in Multi-Agent Debate, Findings of ACL 2025.
• Can LLM Agents Really Debate? A Controlled Study of Multi-Agent Debate in Logical Reasoning, controlled study of team composition and debate structure.

What to read next

• Multi-Agent Systems Need Replay Before More Agents
• Passive Awareness Is the Missing Multi-Agent Primitive
• The Control Tower Pattern for Product Managers
• Bounded orchestration