A Critic helps a runtime challenge a plan, inspect an outcome and explain unresolved work. It cannot create authority by agreeing with the Planner, and a typed report cannot prove its own evidence is trustworthy.
October 16 revision: the former “80 lines” implementation is withdrawn. Its evidence check accepted any truthy classification, including unrelated evidence, and its score wrapper accepted not_evaluated Policy and Safety because neither said fail. It used incompatible reference fields, a mode ranking, guessed budget defaults and an ambient timestamp while claiming deterministic replay. No production Critic service or executed refund is supplied here.
The canonical sources are Cognitive Core, the execution contract and conformance.
Three responsibilities, separate from enforcement
plan proposal
→ verify declared requirements and supplied evidence
→ independently admitted/fenced dispatch
→ score trusted observations and postconditions
→ consolidate actual state, evidence and limitationsA pre-execution review can reject an incomplete proposal. It does not reserve an external balance, freeze provider state, authenticate an approver or prevent a stale worker from calling a tool. Those properties belong to enforced boundaries and protected stores.
A post-execution score can identify failure. It cannot undo a committed effect. Consolidation must retain that distinction: “not accepted as a task outcome” and “not dispatched” are different facts.
Verify the exact proposal
Map every required output to an explicit check; a declared output string is a promise, not proof it will be produced. Validate actual capability names, full nested arguments and evidence references against the current admitted surface. A model can invent an unsurfaced name even if it never saw that schema, so the dispatcher must reject it independently.
Evidence binding needs the required source class, artifact identity/digest, ownership, valid time, access and unresolved-conflict rules. A public classification label is not proof of a refund window. A reference shaped like kg:refund_window:... is not proof that the source exists or supports the claim. Semantic sufficiency needs an explicit trusted observation or an appropriately bounded judgment with limitations.
The five v1 approval modes remain compatibility labels. They do not replace independent native risk dimensions or current owned tenant/audience/purpose grants intersected through every ancestor’s restrictions and revocation. Full-action/evidence/immutable-pin stored approval and current entitled independent quorum remain distinct from native guardrails, actual active containment/credential/data controls, provider preconditions and available shared capacity. Applicable policy-expression errors stop compilation with structured diagnostics.
Do not invent fallback budget fields. The protected ledger accounts exact cumulative spent/reserved use against original immutable ceilings. Admission must include children, model calls, evaluators, retries and repair, rather than trusting an estimated plan sum or refreshing capacity at resume.
Score complete evidence, not a favorable label
The repository’s assessConformanceReport is a deterministic reference for a bounded question: whether supplied current verified evidence supports the exact selected controls and workflow/release/environment scope. It is not a general semantic quality evaluator or dispatcher. The following wrapper uses its actual public API:
import {
assessConformanceReport,
type ConformanceAssessmentInput,
} from "@/lib/contextos/conformance"
export function inspectControlEvidence(input: ConformanceAssessmentInput) {
const assessment = assessConformanceReport(input)
return {
outcome: assessment.outcome,
reasons: assessment.reasons,
missingControls: assessment.missing_controls,
limitations: assessment.limitations,
}
}The outcomes are passed, failed and insufficient_evidence. A complete simulated test can exercise the passing branch; it does not become real runtime evidence. Production adapters must establish observer identity, current trust, actual artifact bytes and freshness. Merely submitting verification_status: "verified" cannot authenticate a report.
Missing required cases, stale or mismatched artifacts, unsupported mandatory controls, not_evaluated and synthetic runtime evidence are insufficient. Verified failures remain failures. Useful positive cases prevent an always-deny system from passing by avoiding all work. Required denial and fault cases complement utility; one global average cannot hide a mandatory control failure.
Keep a separate domain scorecard for actual task utility, supported claims, latency and economics. A high semantic score cannot override a failed control, and a valid control assessment does not prove the final answer is correct.
Consolidate without manufacturing completion
The final record should bind actual inputs, current operation state, trusted control/effect observations, exact approvals, original accounting and unresolved limitations. Do not merge all status vocabularies into an article-local completed flag.
| Layer | Canonical vocabulary |
|---|---|
| Run lifecycle | created, running, waiting_approval, waiting_external, suspended, cancel_requested, completed, failed, cancelled, expired |
| External effect | not_dispatched, confirmed, not_applied, partial, unknown |
| Decision | DECIDED, DEFERRED, REJECTED, ESCALATED, IN_FLIGHT, CLOSED |
| Control assessment | passed, failed, insufficient_evidence |
The owning runtime applies legal transitions in protected atomic stores with current revisions/leases and enforced generation fencing. A Critic report does not acquire ownership or permit a new effect. Wake/resume rechecks authority, exact approval, active boundaries, data and provider state while preserving operation identity and original shared capacity.
A timeout after provider commit leaves uncertainty even if review failed. Retain reservations and reconcile under verified provider semantics; a key string cannot promise one effect. Cancellation intent and future tuple rollback do not prove stop or reversal. Compensation needs a new currently governed action.
A refund example with honest outcomes
A plan proposes a refund but has no eligible refund-window evidence. Verification rejects or escalates the proposal; dispatch remains blocked. Record the missing obligation without leaking raw customer data.
A second proposal supplies a valid source. That fixes one obligation, not the entire permission decision. Recheck the actual order state, exact amount in minor units/currency, current full-action approval/quorum, authority, boundaries and shared capacity. If independently admitted dispatch times out after the provider may have committed, record an unknown effect and reconcile. Do not claim successful refund merely because all score numbers exceed .9.
Only current attributable provider/postcondition evidence can support the relevant confirmed outcome. No payment is executed in this example.
Govern the Critic’s own data and cost
A reviewer sees sensitive evidence too. Admit actual readers, processors, destinations, purpose, residency and retention before sending review prompts or storing reports. Apply this across all nine data stages, including telemetry, cache, memory, export and delegation. Track derived report/trace/example copies and target-specific correction/deletion receipts. Audit convenience does not justify permanent raw-content retention.
Record explicit trusted evaluation time and immutable versions. Compare deterministic artifacts exactly under identical retained inputs; evaluate model-dependent judgments against declared properties. Missing retained evidence and output drift are different from supported tampering evidence. Offline replay never dispatches external effects.
What to verify before a deployment claim
The conformance tests exercise missing cases, always-deny utility, failed mandatory cases, stale/wrong artifact bytes, wrong workflow/release/environment, duplicate evidence, synthetic runtime evidence and deterministic diagnostic ordering. They test the reference logic, not deployed observer authenticity, store atomicity or provider mediation.
Add workflow-specific tests for unrelated evidence, forged current approval, revoked ancestor, changed nested arguments, stale worker, partial/unknown provider outcomes and exhausted shared capacity. Freeze candidates before a separately protected final evaluation and bound repeated feedback. Select exact control profiles and collect current independent useful, denial and fault evidence for the actual implementation, release and environment.