A typed error improves diagnosis. It does not, by itself, establish whether a provider already acted or whether retry is authorized.
The previous article-local dispatcher contradicted its own prose: it called unknown/5xx a non-retry bucket while its map retried it, retried timeouts without provider guarantees, and treated an idempotency-conflict message as proof of success. Its local ToolResult/status shapes also differed from the canonical envelopes. That implementation is withdrawn; this is an explicit recovery-policy design backed by the existing durable execution contract, not a new dispatcher service.
Separate response status, diagnosis and effect
The canonical tool statuses are success, denied, gate_pending, failed and timeout. Durable effect states are separate: not_dispatched, confirmed, not_applied, partial or unknown. Preserve the actual canonical fields/enums rather than adding local ok, error, COMPLETED or RETRIED states to them.
A malformed response after dispatch can leave the effect unknown. A timeout or 5xx can follow a commit. A conflict can mean a prior result, in-progress work, changed parameters or another provider-specific condition. Parsing a word in an error message cannot distinguish those safely.
A useful diagnostic map
The labels below are application diagnostics, not additional canonical control IDs or a universal FailureVerdict export. An adapter must normalize actual supported provider semantics and preserve unknown evidence.
| Diagnostic | Effect interpretation | Permitted next step |
|---|---|---|
| Transport timeout or lost acknowledgement | Unknown unless verified provider evidence establishes more | Preserve identity/reservation; query, reconcile or await entitled manual resolution under the declared recovery mode |
| Provider 5xx | May be not-applied, partial or unknown; code alone is insufficient | Same provider-specific outcome check before any eligible retry |
| Idempotency conflict | Not proof of success or absence | Resolve exact operation/key/arguments and provider state; changed-body conflict cannot reuse the key |
| Stale evidence or precondition | Changed facts may invalidate proposal/approval | Refresh eligible evidence, recompile/replan and rebind any changed action/required approval; do not resend the old call |
| Policy denial | Dependent effect remains prohibited | Return denial or policy-defined escalation; a human cannot waive an independent prohibition without an explicit permitted exception |
| Schema mismatch or unknown error | Visible invalid/unknown result; possible effect remains unresolved | Block dependent work, investigate adapter/provider facts; never default into a retry bucket |
| Applicable policy-expression error | Compilation fails before dependent work | Structured ContextCompileError; repair/review, never permissive false/else |
Not every denial should be escalated, and escalation is not authorization. Record the current policy-defined route and outcome visibly.
Own the operation before recovery
Persist exact stable operation intent/action digest and provider duplicate/query/recovery configuration before dispatch. Use protected atomic run/operation/approval/budget stores with revision, current lease/generation and required target fencing. A trace or planner note describes what happened; it cannot serialize competing workers.
A late definitive receipt may settle the original operation without restoring the old worker’s dispatch rights. Current owner/tenant/audience/ancestor mandate and revocation, exact approval when required, supported contract, active execution/credential/data controls, external preconditions and original shared capacity remain independent conditions.
Retry eligibility precedes backoff
This is non-executable control flow; the store/provider adapters and trusted observations must be implemented and verified separately:
1. Validate the exact current operation, intent and owner/generation/revision.
2. Preserve the actual provider observation and honest effect state.
3. If confirmed: record/recover the original result; do not dispatch again.
4. If partial or unknown: reconcile under the supported provider recovery mode.
5. New dispatch after uncertainty requires verified provider idempotency or absence.
6. Manual/query/no-retry modes do not become automatic retry on timeout or 5xx.
7. Recheck every current authority/approval/boundary/data/precondition condition.
8. Enforce declared eligible-at schedule, attempts, deadline and reserved capacity.
9. Persist the atomic claim/reservation, then issue the eligible owned attempt.
10. Settle trusted cumulative usage and outcome without rewinding later spending.Backoff must be a deterministic function of explicit pinned configuration, attempt history and supplied evaluation time. No hidden random jitter, ambient-clock decision or undeclared default schedule belongs in the reference contract. A production scheduler can wait until the computed eligible time without changing the rule.
Root, child, reviewer, retry, repair and resumed work share original immutable ceilings and spent/reserved accounting. Unknown effects/usage retain reservations. A new retry/session cannot reset capacity; a duplicate settlement returns its original receipt in the current ledger rather than replaying an old state snapshot.
Compensation is a new effect
A reversal token may be an optional correlation or provider artifact. Its presence does not prove entitlement, single use, invertibility, successful completion or current target state. The base contract does not promise that every destructive call has a gateway-issued token or a reversal endpoint.
Bind the compensation’s own exact action, actor, resources, destination, amount/currency, evidence and immutable pins. Obtain current scoped mandate and exact stored approval/quorum when required, verify active boundaries/data permission and external preconditions, and reserve capacity/claim the new operation under current fencing. Link it to the original operation without deleting that history.
A compensating refund is not a mathematical inverse of every downstream consequence. Keep irreversible or unknown consequences explicit. Pack/plugin rollback changes future execution; cancellation requests stop and child cleanup but cannot prove an external effect was reversed. Report observed stop and unresolved effects separately.
Preserve eligible recovery evidence
Provider queries, manual reviewers and incident exports are actual data recipients/processors. Admit current purpose/readers/tenant scope, residency and retention before disclosure. Collection, inference, tool input/output, telemetry, cache, candidate/memory storage, export and delegation all need their applicable controls. Minimize sensitive payloads and bind actual transformations to source/artifact/action/policy facts.
Track corrections/deletion through derived copies with bound receipts. Pending, failed or externally unknown copies are incomplete; an append-only audit or a final apology cannot prove propagation.
A lost-refund acknowledgement
A supported provider may commit the refund while the gateway loses its reply. The original operation becomes unknown. An automatic retry under a query/manual recovery mode is rejected; a new idempotency key is not a workaround. A later definitive provider receipt can confirm the original refund without another effect.
That is the synthetic lost-acknowledgement path in the repository. Run:
rtk npx vitest run tests/execution-workflows.test.ts tests/durable-execution.test.tsThese tests exercise the real reference validators over synthetic facts. They do not establish the actual provider’s idempotency window, query consistency, broker authenticity, atomic storage or deployed containment. The article makes no percentage-of-incidents prevention or executed refund claim.
Release the playbook as a governed artifact
Version diagnostics, provider recovery settings, schedules and escalation policy with the actual runtime. Test useful completion plus wrong/current action, conflicting receipts, concurrent/stale workers, lost acknowledgement, unsupported recovery, cancellation with active children, exhausted shared capacity and duplicate settlement.
Exact profiles and conformance evidence bind artifacts and current independent observations to the actual workflow, implementation, release and environment. Missing, stale, synthetic-only or unevaluated mandatory evidence remains insufficient and blocks dependent release. Typed labels help make a recovery decision inspectable; they do not make duplicate effects structurally impossible without the real provider/store guarantees.