A reliability reviewer checks whether a workflow can handle slow providers, lost acknowledgements, worker replacement and cancellation without pretending an uncertain effect is safe to repeat.
It shares the typed findings and protected change-control pattern from Building a Compliance Reviewer. This is a rubric and test-plan guide, not a deployed reviewer service. The former article-local function checked header and reversal-field presence, accepted non-finite timeouts and looked up capabilities without their adapter identity. Those checks could not establish the reliability claims made for them.
Four useful rules
The R- identifiers are local review rules, not standard control IDs or new adapter schema fields. Map them to actual native risk, execution attachments and the provider contract.
| Rule | Required check | Reject or mark insufficient when |
|---|---|---|
R-001 | Explicit finite positive bounded timeout, deadline and resource limits appropriate to the workflow | Missing, NaN, infinite, fractional or out-of-range configuration; undeclared hidden defaults |
R-002 | Stable exact operation intent, current owned worker/claim and declared provider duplicate/recovery semantics | A header name is the only duplicate-safety evidence; changed arguments reuse a key; IDs collide across adapters |
R-003 | Honest reversibility classification and governed reconciliation/compensation | An irreversible action is represented as reversible because an endpoint/token exists; compensation lacks its own authority |
R-004 | Bounded deterministic retry eligibility, schedule, original shared capacity and current preconditions | Timeout/5xx alone triggers retry, a new key hides an unknown commit, or resume creates fresh budget |
No universal 100–60,000 ms interval fits every workflow. Declare supported bounds and validate them, including non-finite and boundary cases. A backoff array with two entries does not prove safe retry. Match a capability by its complete adapter/capability/version identity, not a name alone.
Separate tool status from effect state
A tool can time out after its provider commits. Preserve effect states such as not-dispatched, confirmed, not-applied, partial or unknown using the canonical durable execution contract; these are distinct from a tool response status.
Keep a stable operation/action digest and provider-specific idempotency/query/recovery configuration. New dispatch after uncertainty requires verified provider idempotency or verified absence under the supported recovery mode. Manual/query/no-retry modes do not become automatic retry because the error is called transient. Late facts can resolve the outcome without restoring a stale worker’s authority.
Compensation is a new governed effect with its own exact action, current mandate/approval, active boundary/data controls, external preconditions and budget. Release rollback selects future code/configuration; it cannot undo a sent message, payment or physical change.
Ownership and cancellation are reviewable behavior
Use protected atomic run, operation, approval-redemption and budget stores with compare-and-swap revisions and generation/lease checks. A trace describes events; it cannot serialize competing workers. Unsupported required target fencing blocks the path rather than falling back to unfenced execution.
A cancellation request is not observed stop. Account for dispatched work, outstanding children and uncertain provider effects before claiming completion or release of reservations. Recheck current owner/ancestor grant and revocation, exact approval, supported contract, active execution/credential/data controls, external preconditions and original shared capacity at scheduled wake or resume.
Fault cases worth testing
These are proposed implementation cases, not a claim that the old pictured fixture files or a reviewer CI service exist in this repository.
| Case | Expected behavior |
|---|---|
| Supported useful action | Completes under current controls without duplicate effect |
| Invalid timeout or missing required configuration | Visible structured rejection, never implicit pass |
| Provider commits and drops acknowledgement | Unknown outcome, stable identity, reconcile before eligible retry |
| Two workers claim the same operation | Only the current atomic owner can dispatch |
| Stale generation resumes or receives late success | Preserve late fact; reject stale dispatch/authority |
| Provider recovery mode is manual or unsupported | No automatic retry or weaker recovery fallback |
| Cancellation with active child or unknown effect | Do not report stopped/completed or release uncertain reservations |
| Duplicate budget settlement after later spending | Return original receipt in current ledger without rewinding spending |
| Compensation changes destination or amount | New exact governed proposal/approval and operation |
| Fresh session after interruption | Original spent/reserved capacity and deadlines remain |
Add actual provider fault evidence and record assumptions, test oracle, environment and expiry. Do not run disruptive tests against live third parties without the appropriate scope. Synthetic reference tests prove validator behavior; they do not establish a live provider, broker, sandbox or store guarantee.
Compose the result with other controls
A reviewer supplies bounded findings tied to the exact proposal/artifact, reviewer and policy versions. The protected release owner validates required report provenance/currentness and current entitled approvals. A required fail blocks dependent release; missing, stale, synthetic or unevaluated evidence is insufficient. A named approver cannot waive an independent prohibition or grant an exception that policy does not permit.
Reviewer work is also governed work. Admit actual data readers/processors and purpose/retention before sending traces; minimize sensitive payloads and track derived/cache/export/delegated copies and deletion receipts. Reserve model/tool/reviewer/retry/child capacity from the original shared pool. A background reviewer needs a current owner mandate and active containment too.
Acceptance evidence
Select exact control profiles for the enabled features. Conformance binds useful and denied cases, artifacts and current independently verified observations to the actual workflow, implementation, release and environment. A four-rule rubric, an idempotency key or a reversal declaration cannot certify production reliability or prevent a stated percentage of incidents.
Start with one bounded provider path and a known recovery contract. Expand scope only after both useful work and the relevant faults have evidence. Reviewer count and lines of code are not assurance metrics.