Skip to content
Back to Blog

Wiring the Five Evaluators: Policy, Utility, Latency, Safety, Cost

Published Updated Editorial policy & corrections

Share:XBSMRedditHNEmail
Wiring the Five Evaluators: Policy, Utility, Latency, Safety, Cost illustration

A candidate can improve task utility while introducing unauthorized actions. Evaluating only the final answer or one average hides that trade. Keep Policy, Utility, Latency, Safety and Cost visible as separate dimensions, with application-specific checks beyond those five where needed.

October 16 revision: the former minimal TypeScript evaluator/gate implementation is withdrawn. An isolated probe returned merge for two empty scorecard arrays. Another showed that one destructive call without approval produced Safety warn, which the gate also merged. The old Policy evaluator treated an empty event list as perfect, used unverified self-reported fields and did not detect missing required decisions. These are unsafe defaults, not a production gate.

This article supplies explicit design and test obligations rather than a runnable scoring/release service. All examples and thresholds are illustrative; no production regression, benchmark gain or deployment is claimed. The canonical sources are Evaluation and Observability, conformance and the execution contract.

Establish evidence before scoring

An evaluator consumes observations whose origin, completeness, scope and valid time must be checked. A type declaration or signed-looking field does not establish those properties.

Bind each result to the exact workflow, implementation/version, release pins, environment, dataset, evaluator and actual artifact bytes. Distinguish three outcomes: verified support for a claim, verified failure and insufficient evidence. Missing, malformed, nonfinite, stale, mismatched, synthetic runtime or not-evaluated observations cannot become passing defaults.

The current ConformanceReport and assessConformanceReport reference implement bounded evidence-sufficiency checks for exact selected controls and useful cases. They do not ingest/authenticate production telemetry or replace domain-specific evaluators. Trusted adapters must establish report/observer authenticity and current entitlement. Synthetic fixtures exercise the reference; their results must not be relabeled as deployed control evidence.

1. Policy — compare intended permission with actual dispatch

A policy denial that stopped an unsafe proposal can demonstrate correct enforcement even when it indicates poor model behavior. A denied action that reached the provider is an enforcement failure. Also test the opposite direction: did useful permitted work complete, or did the system simply deny everything?

Required checks include:

  • every relevant dispatch has the required current policy decision;
  • actor/tenant/audience/purpose and all ancestor grants/revocation match;
  • actual capability, nested arguments, evidence and immutable pins match;
  • full exact stored approval has current entitled independent quorum where required;
  • native risk, guardrails, actual active containment/credentials/data and provider preconditions hold;
  • required denial blocks dispatch, including equivalent shell/browser routes.

An empty policy-event list means insufficient evidence unless the declared workflow genuinely has no applicable obligation and the trusted coverage assessment establishes that fact. Applicable compiler policy-expression exceptions stop compilation with structured errors; they are not false predicates or a zero-violation score.

2. Utility — verify the promised outcome

Define task-specific acceptance before seeing candidate results. Evidence-class presence and a terminal label are prerequisites in some workflows, not a universal utility formula. Verify that the cited source actually supports the material claim and that observed external state satisfies the intended postcondition.

For a refund, a syntactically valid receipt does not prove the right account received the right amount. For a report, correct arithmetic on the wrong cohort is failure. Preserve appropriate abstention and escalation as valid outcomes where the task requires them.

Include useful positive cases to prevent vacuous always-deny success. Record attempted work and the denominator, partial results, missing dependencies and human correction burden. Do not drop failed or abandoned cases after inspecting which candidate they favor.

3. Latency — separate objectives from hard deadlines

Measure intake/compile, model queue/generation, tool waits, approval, verification, retry and accepted-outcome time. State whether a human wait is included. Report distributions and tails for the declared cohort, not a per-run number mislabeled p95.

A smooth score can compare performance against an application SLO after validating finite nonnegative observations and declared targets. It cannot soften a mandatory deadline or authorize more work after expiry. An unknown intent cannot silently inherit a guessed budget/target.

Concurrent tools can reduce wall time while increasing resource use. Retain both elapsed time and actual tool/model usage, including cold starts, cache misses and failures.

4. Safety — mandatory violations are not warnings

A required approval bypass, forbidden disclosure or containment escape must block dependent work/release under the applicable policy. It cannot be diluted into a high score or downgraded merely because there was only one finding.

Inspect actual data at all relevant boundaries, not only final model output. A classification label does not authorize a flow. Redaction declarations require transformed bytes and trusted source/artifact/action/policy-bound proof; the compiler does not implement a universal sanitizer.

Semantic judges can add bounded signals, with calibrated rubrics, recorded versions and uncertainty. Their agreement does not establish statistical independence or replace deterministic controls. Test false denials and interruption burden alongside missed failures.

Govern actual readers/recipients/processors, purpose, residency and retention across collection, inference, tool input/output, telemetry, cache, memory, export and delegation. Inventory derived evaluator/trace/example copies and require target-specific correction/deletion receipts. Preserve pending, failed and external unknown propagation honestly.

5. Cost — use exact aggregate accounting

Distinguish an optimization target from the run’s immutable hard ceilings. A score curve around an illustrative dollar target cannot enforce a budget.

The reference ledger uses exact decimal cost/counters, root/child aggregate spent/reserved capacity, atomic revisions and idempotent reserve/settle transitions. Include model, tools, children, graders, shadow, fallback, retries and repair under the original shared limits. A wake, child or reviewer cannot refresh the allowance.

Retain unknown provider effect/usage reservations until authoritative reconciliation. Cancellation or task failure does not prove no spend occurred. Late facts may settle accounting without granting new dispatch or ownership. Business transaction value and runtime compute cost are separate dimensions.

Map unique invoice lines to disjoint provider-native billing categories; reasoning may already be included in billed output, and retry/evaluator/tool-result labels overlap content or work purpose. Count each charge once. Report zero accepted outcomes and total spend instead of dividing by a fabricated success count.

Compose a report without averaging away failures

The report should retain each dimension’s status, scope, case coverage, observations, findings and limitations. Keep absent/unverified evidence distinct from measured failure. Sensitive raw values do not belong in broadly visible finding messages; use scoped references.

Illustrative assessment flow, not executable gate code:
  validate report and artifact provenance/current scope
  → verify every mandatory control and case coverage
  → retain verified failures
  → classify missing evidence as insufficient
  → inspect utility, uncertainty, human work, latency and cost
  → preserve the exact candidate and all unsuccessful trials
  → request independent current release authorization

No number of favorable soft metrics can erase a mandatory failure. Evaluation acceptance is not permission to merge, deploy or perform a new effect.

Release requires independent authority and current controls

Separate development/search data from final evidence. Freeze each candidate before a separately protected release trial and bound repeated submissions/feedback. Prespecify endpoints, acceptable regressions, repeated-trial design, infrastructure-failure handling and task grouping. Report confidence/coverage limits; a perfect observed rate is not proof of zero future risk or monotonic improvement.

A release action needs current owned tenant/audience/purpose authority intersected with every ancestor restriction/revocation; full-action/evidence/immutable-pin stored approval from currently entitled independent principals; native risk/guardrails; actual active containment/credentials/data; provider preconditions; and available shared capacity.

Protect atomic run/operation/approval/budget state and enforce generation fencing on wake/resume and dispatch targets. Reconcile unknown provider effects before unsafe repetition using stable operation identity and verified provider scope/window or authoritative absence. A key, cancellation request or future release rollback cannot prove stop or reversal. Compensation is a new governed effect.

Select exact control profiles and collect current independent useful, denial and fault conformance evidence for the actual workflow, implementation, release and environment. A missing report cannot merge. A current failure remains visible. Source/reference tests alone cannot certify the deployment.

Verification checklist

AreaUseful evidenceDenial/fault evidence
PolicyCurrently permitted exact action completesMissing decision, revoked parent, changed args, forged approval
UtilityDomain postcondition independently verifiedWrong subject, absent source, incomplete outcome, always-deny behavior
LatencyCorrect distribution and accepted-outcome timeMissing clock data, expiry, cold start, partial stream
SafetyPermitted data reaches only approved recipientsLeak before output, stale boundary, unverified transformation
CostExact root/child reserve and settlementDuplicate mutation, exhausted capacity, unknown spend, stale revision
ReleaseFrozen exact candidate with current independent evidenceEmpty report, stale/wrong environment, failed mandatory case, leaked final set

The repository tests exercise these bounded contract primitives, including rejection of vacuous conformance and synthetic runtime evidence. A production adapter must additionally prove actual telemetry collection, store atomicity, target fencing, containment, credentials, processor routes and provider recovery in its own environment.

Found this useful? Share it.

Share:XBSMRedditHNEmail

Continue through the same topic without returning to the index.

View the series