Reviewer Agents

Reviewer Agents are specialized harness components that inspect proposed changes—pack diffs, policy edits, tool additions, planner skill updates, agent-generated PRs—before they reach the independent promotion authority selected by policy. In the high-assurance reference profile on this page, that authority is human. Reviewers reduce the linear burden of mechanical checks so accountable people can focus on judgment, architecture, and risk.

A reviewer agent is itself a versioned harness component: it has a skill, a rubric, a locally versioned output schema, a golden set, and a release gate. It is treated like any other component of the Improvement Loop—its findings produce candidate proposals that land in the same change-control queue as everything else. The reviewer-output envelope on this page is a local deployment protocol, distinct from the published execution/evidence attachments.

Why specialized reviewers, not one giant reviewer

A broad “review the change” rubric can fail in three ways:

1. Concerns get blurred. Security findings get diluted by stylistic noise; cost regressions hide behind feature wins.
2. Severity calibration drifts. A reviewer that sees everything calibrates to the median, missing tail-risk findings.
3. Ownership becomes diffuse. Nobody owns the rubric, so it ages silently.

Specialized reviewers make each concern and owner explicit. Calibrate severity against concern-specific goldens and measure missed/incorrect findings; specialization or agent consensus alone cannot establish correctness.

The human promotion policy here is a local reviewer deployment profile, distinct from the immutable contextos.high-assurance@1 control definition. Reviewers remain scoped agents: admit their actual features, current delegated authority, boundary/data destinations and original aggregate budget. Their findings are evidence inputs; runtime conformance additionally requires independently verified current control/fault evidence for the exact release/environment/workflow.

The seven reference reviewers

Architecture reviewer

Owns: plane boundaries, dependency direction, primitive layering.

Checks include:

• No business-agent path that bypasses the Tool Gateway to call a database or external system directly; an admitted DB adapter may access its declared backend inside the gateway boundary.
• No cross-domain dependency without an adapter manifest.
• No governed business action without its exact decision_key resolved in the Decision Catalog; a plan checkpoint’s decision_id is a separate field referencing that key.
• No tool added without a schema and capability class.
• No agent response written without a trace_id propagated end-to-end.

Severity floor: any finding that breaks the reference architecture is blocker.

Security reviewer

Owns: sensitive data, secrets, auth, sandbox profile, injection, isolation.

Checks include:

• No plaintext sensitive field in logs or traces (policy_id: no_plaintext_sensitive_data_logging).
• All secrets resolved through the secrets adapter; no inline credentials.
• Tool sandbox profile matches the tool’s side-effect classification.
• Principal chain, tenant, workload identity, and stable subject/evidence refs are preserved across every Tool Gateway call. SIDs are model features and never substitute for audit identity or authority.
• User-controlled/retrieved content retains data-only provenance and cannot create instructions, grants or privileged arguments without verified-source/user-intent policy checks; escaping text alone cannot prove injection resistance.
• Current boundary/egress, brokered credentials, actual processors/recipients and source-label policy match the enabled code/tools/data path; declarations alone are insufficient.

Severity floor: any sensitive-data leak is blocker and emits a security event regardless of approval state.

Reliability reviewer

Owns: timeouts, retries, fallbacks, idempotency, rollback paths.

Checks include:

• Every tool call has an explicit timeout; no inherited defaults from a generic HTTP layer.
• Retries are scoped (max attempts, backoff, idempotency key); no unbounded retry loops.
• Every state-changing tool declares idempotency and recovery behavior; reversible and compensatable actions expose the corresponding mechanism.
• Irreversible actions declare that fact explicitly and require the stricter evidence and authority obligations selected by policy.
• Failure-playbook coverage exists for the typed verdicts the change can produce.
• Lost acknowledgement, cancellation races, stale ownership generations, checkpoint migration and provider idempotency-retention expiry preserve unknown effects and original reservations; recovery is tested rather than inferred from a key.
• New planner / executor / critic skills declare their loop guard and budget posture.

Severity floor: a state-changing tool that falsely claims reversibility or omits required idempotency/recovery semantics is blocker.

Product reviewer

Owns: user experience, intent fidelity, edge cases.

Checks include:

• Response is clear, minimal, and useful at the user surface.
• Edge cases on the intent (empty result, expired state, partial failure) are explicitly handled.
• User-facing messages do not expose internal system details, internal IDs, or stack-shaped strings.
• Clarification questions are asked only when the harness genuinely lacks the information; otherwise the agent acts on its best evidence and says so.
• Tone, naming, and surface area match the product principles registered in the repo.

Severity floor: leaking internal details to the end user is blocker.

Data reviewer

Owns: event schema, evidence coverage, analytics impact.

Checks include:

• Each decision’s required evidence names are covered by admitted current verified artifacts or authenticated structured context; the DecisionRecord.evidence_refs resolve those artifacts rather than merely repeating requirement names.
• New event types are registered in the schema registry before they ship.
• Analytics events include the canonical correlation IDs (trace_id, run_id, decision_id) — no event is “for product” only.
• Memory proposals route to the appropriate tier through review with required consent, contradiction, namespace/readers, retention and copy-propagation evidence; the four tiers are not a mandatory sequential promotion ladder.

Severity floor: a DecisionRecord without resolvable evidence_refs is blocker.

Cost reviewer

Owns: tokens, infra cost, retrieval cost, run-budget headroom.

Checks include:

• Token usage per decision stays within the intent’s RunBudget band; new pack versions cannot regress economics by more than the configured guard.
• Tool-call count per decision is bounded; loop guards activate before the budget is exhausted.
• Retrieval breadth (top_k, max_hops) is justified by an evaluator metric, not by intuition.
• Caching occurs only within admitted purpose, tenant/owner/namespace, processor, retention and data policy; reusability alone cannot authorize a cache or avoid lifecycle propagation.
• Parent, child, retry/repair/fallback and online reviewer work share original limits and deadline, with exact units and verified settlement; unknown usage remains visible in cost per accepted outcome.
• Model selection: the AI Gateway & LLM Router profile is appropriate for the native risk dimensions and evaluation requirements; v1 risk class remains a historical slice.

Severity floor: a regression on economics_cents_per_decision greater than the guardrail is blocker.

Compliance reviewer

Owns: audit, consent, regulated actions, native ActionRisk, policy obligations, and the v1 ApprovalMode projection.

Checks include:

• Every regulated action has the typed obligations and named accountable role required by applicable policy; where approval is required, it has a named gate and approver role.
• Native effect, authority, reversibility, interaction, data scope, and decision TTL match the capability and concrete call. The v1 ApprovalMode fields remain conservative compatibility projections.
• The accountable workflow owner identifies the applicable consent/purpose and other required data conditions; the runtime verifies those conditions rather than treating a generic consent flag as permission for every regulated use.
• Audit trail covers the full lifecycle: proposal, approval, execution, reversal.
• Regulatory mapping (NIST AI RMF, ISO/IEC 42001, sector-specific frames) is current for any new control.

Severity floor: an action missing a policy-required gate, consent record, authority binding, or native risk declaration is blocker.

Output schema

Every reviewer adopting this reference profile emits the same locally versioned envelope, regardless of concern:

{
  "review_id": "rv_2026_05_08_a17",
  "reviewer_agent": "security-reviewer",
  "reviewer_version": "1.4.0",
  "subject": {
    "kind": "pack_change",
    "pack_id": "ctxpack.support",
    "from_version": "5.1.0",
    "to_version": "5.2.0",
    "diff_ref": "git:repo@a13b9c2..7777eaa"
  },
  "status": "fail",
  "findings": [
    {
      "finding_id": "f_01",
      "severity": "blocker",
      "file": "harness/tools/payment.refund.json",
      "issue": "The compensatable effect declares no compensation evidence in the tool result contract",
      "policy_id": "no_state_change_without_recovery_evidence",
      "recommendation": "Add a protected compensation reference and observable postcondition to the result contract."
    }
  ],
  "evidence_refs": ["dr_2026_05_07_q88"],
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "completed_at": "2026-05-08T09:14:00Z"
}

Within this profile, severity vocabulary is fixed: blocker, must-fix, should-fix, nit. A blocker from any reviewer prevents promotion past the relevant rollout stage; a must-fix requires either the fix or a change-control decision explaining the deferral. That change-control artifact is not the governed business DecisionRecord unless it actually satisfies that schema and decision binding.

Integration with approval gates

Reviewer findings are inputs to the governance approval gate, not a replacement for it.

proposal arrives
    ↓
reviewers fan out in parallel
    ↓
findings aggregate by severity
    ↓
if any blocker: gate denied, proposal re-enters change-control as 'changes_requested'
if any must-fix: gate held; either fix lands or ADR is filed
if only should-fix / nit: gate goes to human approver with summary
    ↓
human approver makes the final call (judgment, risk, scope)
    ↓
if approved: rollout begins at 0%_shadow per the harness rollout stages

This high-assurance profile deliberately reserves the final promotion decision for a human: reviewers may auto-deny a blocker but never auto-approve. Other profiles may use a bounded automated authority only when policy explicitly names it and independent evaluation proves non-vacuity; the proposer still cannot approve itself.

Reviewer skills are first-class harness components

Each reviewer is a versioned skill in harness/reviewers/:

harness/reviewers/
  architecture/
    skill.md               # role, scope, rubric
    policies.json          # JsonLogic checks the reviewer applies
    fixtures/              # known-good and known-bad changes
    golden_set.json        # labeled findings for evaluator scoring
  security/
    skill.md
    policies.json
    fixtures/
    golden_set.json
  ...

The skill follows the same minimal-skill pattern used elsewhere in the harness (Meta-Harness Appendix D calls skill quality a strong lever on search quality): constrain what is forbidden and what artifacts to produce, while leaving the reviewer free to inspect any material inside its declared authorization and data scope.

A reviewer’s golden set is exercised on every reviewer-skill change. False-positive rate, false-negative rate, and severity-calibration drift are tracked the same way evaluator metrics are tracked in Evaluation and Observability.

Failure modes

• Reviewer becomes a style cop. Findings devolve into nits and humans stop reading. Mitigation: severity floor by concern; cap nit-rate per review.
• Reviewer over-blocks. Every change becomes a blocker; throughput collapses. Mitigation: track per-reviewer block rate; rebaseline rubric quarterly.
• Reviewer drifts. New patterns appear in the codebase that the rubric doesn’t cover. Mitigation: feed Improvement-Loop insights of kind gap_detected back into reviewer rubrics.
• Reviewer is bypassed. A team adds an “emergency lane” that skips reviewers. Mitigation: emergency handling is an explicit policy exception with scoped authority, expiry, a security event, and mandatory post-hoc review—not a new approval-mode tier.
• Reviewer hallucinates findings. Mitigation: every finding must cite a policy_id from the policy bundle or a rule from the reviewer’s own rubric; ungrounded findings are dropped at the gate.

Operational concerns

• Reviewers run in parallel; the slowest reviewer sets gate latency.
• A separate offline change-review job has its own admitted root budget and data authority; online reviewer work consumes the production run’s aggregate capacity. Budget exhaustion cannot silently waive required review; emergency handling uses explicitly authorized scope, expiry and later evidence review.
• Reviewer skills are versioned independently and live gating uses an approved pinned revision. An unreviewed downgrade cannot bypass current required controls; replay retains the original revision and comparison policy.
• Reviewer outputs are append-only and replayable. Deterministic validators must reproduce exactly from pinned inputs; model-based review is compared through required findings, forbidden misses, calibration, and scorecard properties rather than assumed to emit identical prose.

Evaluation metrics

• Block rate per reviewer — fraction of reviews ending in any blocker. Each deployment calibrates an evidence-backed band by concern; movement outside it triggers miscalibration review rather than automatic threshold relaxation.
• False-positive rate — blocker findings overturned at human approval.
• False-negative rate — incidents whose root cause was a finding the reviewer missed.
• Severity calibration drift — distribution of severities over time; sharp shifts trigger rubric review.
• Time-to-finding — median wall-clock from proposal arrival to reviewer verdict.
• Coverage — fraction of diffs that exercise at least one rule in the reviewer’s rubric.

Example: a security reviewer catching a regression

A pack diff adds a new tool adp_support.fetch_thread. The Architecture reviewer passes; the Security reviewer fails. This is an abridged display of the local finding envelope:

{
  "review_id": "rv_2026_05_08_b04",
  "reviewer_agent": "security-reviewer",
  "reviewer_version": "1.4.0",
  "subject": {
    "kind": "tool_addition",
    "tool_name": "adp_support.fetch_thread",
    "diff_ref": "git:repo@7777eaa..b21c5d4"
  },
  "status": "fail",
  "findings": [
    {
      "finding_id": "f_01",
      "severity": "blocker",
      "file": "harness/tools/support.fetch_thread.json",
      "issue": "Output schema includes raw user_email and phone_e164 with no redaction profile bound",
      "policy_id": "no_plaintext_sensitive_data_logging",
      "recommendation": "Bind and verify the permitted source-to-result transformation and actual reader/processor route before model or telemetry exposure."
    },
    {
      "finding_id": "f_02",
      "severity": "must-fix",
      "file": "harness/tools/support.fetch_thread.json",
      "issue": "The proposed route includes identity-mapped fields without verified purpose, recipient and required consent-policy evidence",
      "policy_id": "data_flow_matches_verified_scope",
      "recommendation": "Verify current data-flow authority and required source/recipient/consent conditions; renaming the capability class cannot authorize disclosure."
    }
  ]
}

The gate denies promotion; the Improvement Loop captures the pattern as gap_detected so the Security reviewer’s rubric can be extended; an ADR is filed if the team wants to defer the must-fix.

See also

• Harness Engineering — the discipline that this primitive composes into
• Governance — native action risk, v1 approval-mode compatibility, and policy bundles
• Improvement Loop — how reviewer findings feed change control
• Evaluation and Observability — how reviewers are themselves evaluated