Skip to content
Back to Blog
Reviewers & improvement
July 25, 2026
·by ·24 min read

What Production Agent Runtimes Actually Teach ContextOS: Twelve Laws of a Governed Harness

Share:XBSMRedditHNEmail

The most important lesson from production agent systems is not that models have become better at writing code, browsing applications, or calling tools.

It is that a capable model is only one nondeterministic component inside a much larger runtime.

The systems that are becoming useful in production—coding agents, long-running workflow agents, tool-using assistants, and multi-agent orchestrators—are converging on the same underlying architecture:

  • context is selected and compiled rather than indiscriminately accumulated;
  • execution state is persisted outside the model;
  • side effects are recorded, deduplicated, and verified;
  • human decisions enter the runtime through explicit interrupts;
  • technical containment, identity, policy, and approval remain separate controls;
  • deterministic checks surround probabilistic reasoning;
  • traces record causal execution, not merely model text;
  • improvements are promoted only after evidence, replay, and governed release.

Production coding agents make these patterns unusually visible because they work in a hostile operational regime. They enter unfamiliar repositories, discover local conventions, select specialised procedures, run commands, edit files, call external systems, request authority, recover from failed attempts, verify outcomes, and sometimes continue the same task hours or days later.

But the lessons are not limited to software engineering. The same runtime problems appear when an agent changes a booking, issues a refund, reconciles an invoice, prepares a regulatory submission, modifies a campaign, updates a customer record, or coordinates work across several specialist agents.

This article extracts twelve laws of a governed agent harness and maps them to ContextOS.

The central claim is simple:

The production unit is not the prompt, the model, or even the agent. It is the governed, resumable run.

Contract status: This article proposes orchestration extensions around the current ContextOS contract. The published RunContext, CompiledContext, ToolEnvelope, DecisionRecord, and ReplayPacket schemas remain authoritative until any candidate type is separately specified, implemented, and released.

The evidence is broader than any one framework

No single runtime should become the blueprint for ContextOS. Each exposes a different part of the production problem.

Runtime familyWhat it makes concreteTransferable lesson
Production coding agentsScoped project instructions, skills, hooks, isolated workspaces, permissions, subagents, verificationThe environment around the model must carry durable method, containment, and deterministic checks
OpenAI Agents SDK, Google ADK, Microsoft Agent FrameworkAgent loops, handoffs, sessions, callbacks, guardrails, workflow graphs, human-in-the-loopModel-directed reasoning must compose with explicit runtime control points
LangGraphCheckpointed state, interrupts, persistence, re-execution semanticsPausing and resuming is a state-machine concern, not a conversational trick
Temporal and Microsoft Durable TaskEvent history, replay, retries, durable timers, idempotent activitiesLong-running agents need distributed-systems semantics, not just longer context windows
MCPTool schemas, host-mediated invocation, authorisation, consent, security boundariesTool discovery is not authority; tool metadata is not automatically trustworthy
A2AStateful tasks, lifecycle states, messages, streaming updates, artifactsAgent interoperability needs task and artifact contracts, not transcript forwarding
OpenTelemetry GenAI conventionsStandardised spans for model calls, tools, tokens, and agent operationsObservability must describe the complete causal run across vendors and frameworks

The convergence matters more than the individual product features. Across these systems, the model increasingly sits inside a runtime kernel that owns state, execution, policy, recovery, and evidence.

That is the architectural direction ContextOS should preserve.


Law 1: Put the model inside a deterministic runtime shell

A prototype agent loop often looks like this:

user request
  -> model
  -> tool call
  -> tool result
  -> model
  -> final answer

That representation hides nearly every production concern.

A real run also needs identity, policy, context selection, budgets, cancellation, retries, tool contracts, approvals, durable state, artifact storage, verification, telemetry, and terminal outcomes.

A more accurate model is:

request
  -> authenticate principal
  -> bind tenant and policy snapshot
  -> compile context pack
  -> instantiate run state
  -> enter controlled agent loop
  -> mediate every external effect
  -> checkpoint transitions
  -> interrupt, retry, resume, or compensate
  -> verify declared outcome
  -> emit artifacts, evidence, and terminal state

Microsoft Agent Framework explicitly distinguishes an agent, whose next step is dynamically chosen by an LLM, from a workflow, whose control flow is defined by code. Microsoft Durable Task similarly distinguishes deterministic workflows from agent-directed loops while allowing both to coexist. Google ADK exposes a runtime event loop around agents, tools, callbacks, state changes, and external services. These are different implementations of the same principle: probabilistic reasoning should be embedded inside a runtime that owns the non-negotiable parts of execution.

For ContextOS, this means the model must not own:

  • the canonical state of the run;
  • the list of authorised capabilities;
  • approval validity;
  • retry semantics;
  • side-effect deduplication;
  • budget enforcement;
  • completion criteria;
  • audit history.

The model may propose a plan, choose among eligible actions, interpret results, and adapt its reasoning. The runtime decides whether the proposed transition is valid and how it is recorded.

ContextOS implication

At the orchestration layer, the durable aggregate should be a Run that contains or references the canonical RunContext; a conversation remains only one projection.

A minimum run record should bind:

run_id: run_...
run_context_ref: run_context_...
compiled_context_hash: sha256:...
policy_snapshot_ref: policy_...
workflow_ref: workflow_...
tool_manifest_hash: sha256:...
state: RUNNING
event_sequence: 187
pending_interrupt_refs: []
artifact_refs: []
effect_refs: []

The transcript is one projection of this run. It is not the source of truth.


Law 2: Compile context; do not accumulate it

The naive response to an unreliable agent is to add more instructions. That produces a larger prompt, not a stronger runtime.

Production coding agents increasingly separate several kinds of context:

  • always-relevant project guidance;
  • directory- or path-scoped rules;
  • task-specific workflow skills;
  • tool definitions;
  • current run state;
  • retrieved references;
  • compacted historical context;
  • specialised subagent context.

Claude Code loads root-level guidance at session start, while instructions in subdirectories can load when work enters those directories. Its documentation warns that large instruction files consume context and reduce adherence. Mature runtimes distinguish lightweight persistent instructions from richer skills that should load only when relevant. They also optimise for stable prompt prefixes and compact long conversations to preserve both context-window capacity and prompt-cache efficiency.

The general rule is:

Context should be compiled for the next decision, not inherited from everything the system has ever seen.

A Context Pack Compiler should construct a bounded, typed context pack from explicit sources:

request + current state
  -> resolve tenant and principal
  -> classify intent and decision stage
  -> select applicable policies
  -> discover eligible skills
  -> retrieve task-relevant facts
  -> load scoped instructions
  -> include only surfaced tools
  -> apply recency, trust, and token budgets
  -> produce context manifest + rendered model context

The compiler should preserve both the rendered content and its lineage:

context_item:
  id: ctx_item_42
  type: scoped_instruction
  source: repo://payments/AGENTS.md
  source_version: git:8fa21d
  trust_class: organisation_managed
  scope: payments/**
  selected_because: current_resource_path
  token_count: 184
  expires_at: null

This makes context selection inspectable and replayable.

Why progressive disclosure matters

Progressive disclosure is often described as a token optimisation. That is only the first-order benefit.

It also reduces:

  • instruction collision;
  • accidental policy leakage across domains;
  • irrelevant tool salience;
  • context poisoning surface;
  • stale guidance exposure;
  • prompt-cache invalidation;
  • cross-agent contamination.

A payment-reconciliation procedure should not influence a customer-support refund unless the runtime deliberately selects it. A hotel-booking skill should not enter a visa workflow simply because both belong to “travel.”

ContextOS implication

ContextOS should treat context compilation as a first-class decision with its own output contract, telemetry, and evaluation. The compiler should be judged on:

  • relevance precision and recall;
  • policy completeness;
  • stale-context rate;
  • trust distribution;
  • token efficiency;
  • contradiction rate;
  • downstream decision quality.

This moves context engineering from prompt craft into runtime architecture.


Law 3: Every correction needs a promotion target

When an agent makes a mistake, teams commonly paste the correction into a global system prompt. This creates prompt sediment: a growing collection of local, temporary, contradictory, and unenforceable instructions.

Production coding agents expose a better pattern because they provide several durable surfaces: project instructions, path-scoped rules, reusable skills, hooks, tests, linters, CI checks, permissions, and environment configuration.

The architectural question is not “how do we remember this?” It is:

Which layer is capable of owning this correction with the narrowest valid scope?

The correction promotion ladder

Correction shapeCorrect targetWhy
“For this run, do not change the API.”Current run constraintsTemporary and task-specific
“For this customer, use the corporate policy profile.”Principal or account policy bindingUser/account-specific and governed
“Every change in this package needs the integration test.”Closest scoped guidance, then validatorLocal recurring rule with a mechanical proof
“Release work always follows these seven steps.”Versioned skill or workflowReusable procedural method
“Never proceed when schema validation fails.”Deterministic gateMachine-checkable invariant
“Refunds above the threshold need finance approval.”Policy and approval ruleAuthority constraint, not model guidance
“This downstream API duplicates requests on retry.”Tool adapter and idempotency strategyIntegration behaviour belongs at the effect boundary
“The model often misclassifies this intent.”Training/evaluation dataset or classifierStatistical behaviour, not a prompt-only defect

Reflection is therefore an evidence source, not an automatic write path.

An agent may notice that a build command is missing, a tool description is misleading, or a workflow repeatedly stalls. Its observation should become a typed improvement proposal with trace references, affected scope, suggested target layer, and expected metric impact. It should not directly rewrite production instructions or policy.

ContextOS implication

The Improvement Loop should implement promotion as a governed pipeline:

observed failure
  -> evidence bundle
  -> recurring-pattern detection
  -> root-cause classification
  -> target-layer selection
  -> candidate change
  -> offline replay
  -> policy and security review
  -> canary release
  -> production monitoring
  -> retain, revise, or rollback

The target layer belongs in the improvement proposal linked to the source DecisionRecord. A correction promoted to the wrong layer is itself a harness defect.


Law 4: Durable execution is a prerequisite for long-running agency

A long context window does not make a run durable.

If a process restarts, a deployment replaces a worker, a tool times out, a human responds tomorrow, or a rate limit pauses execution, the system must know what already happened and what can safely happen next.

Temporal models this through an append-only Event History and workflow replay. LangGraph persists checkpoints and resumes a thread from saved state. Microsoft Durable Task checkpoints state transitions so completed model calls and tool results do not have to be repeated after infrastructure failure. OpenAI Agents SDK serialises RunState across human approval interruptions.

The shared lesson is:

Agent state must live outside the model and survive the process that currently hosts it.

A run requires an explicit lifecycle

A useful ContextOS state machine could be:

CREATED
  -> COMPILING_CONTEXT
  -> READY
  -> RUNNING
      -> WAITING_FOR_INPUT
      -> WAITING_FOR_APPROVAL
      -> WAITING_FOR_EXTERNAL_EVENT
      -> RETRY_SCHEDULED
      -> RESUMING
  -> VERIFYING
      -> COMPENSATING
  -> COMPLETED | FAILED | CANCELLED | EXPIRED

Each transition should be represented as an event with a monotonic sequence number:

event_id: evt_0187
run_id: run_92a
sequence: 187
type: TOOL_CALL_COMPLETED
occurred_at: 2026-07-25T14:02:11Z
actor: tool_gateway
causation_id: evt_0184
correlation_id: effect_31
payload_ref: blob://...
policy_snapshot_ref: policy_77

The event history provides:

  • recovery after failure;
  • time-travel debugging;
  • replay against new runtime versions;
  • human-readable audit;
  • causal evaluation;
  • deterministic reconstruction of run state.

Checkpoints are not memory

This distinction is critical:

  • Checkpoint/state says what the run has done and what it is waiting for.
  • Working memory contains information useful to the current reasoning process.
  • Long-term memory contains durable information potentially useful across runs.
  • Evidence records why a decision or action was taken.

Combining these into one “memory” abstraction creates incorrect retention, poor replay, and serious governance ambiguity.

ContextOS implication

The ContextOS runtime should be able to destroy any worker process and resume the run without repeating committed effects or losing pending approvals. If that cannot be demonstrated in a fault-injection test, the runtime is not yet durable.


Law 5: Side effects must be designed for replay, retries, and ambiguity

Distributed systems have an uncomfortable failure mode: an external action may succeed while the agent runtime fails before recording the success.

Temporal’s documentation uses this exact class of example to explain why retried activities must be idempotent. LangGraph warns that a node may run again from the beginning after a pause or retry, so side effects before the checkpoint must tolerate re-execution.

For agent systems, this is not a corner case. It applies to:

  • charging a card;
  • creating or cancelling a booking;
  • issuing a refund;
  • sending a customer communication;
  • changing inventory;
  • updating a CRM record;
  • opening a pull request;
  • submitting an application;
  • deploying a service.

“Do not call the tool twice” is not a reliable solution. The runtime must assume retries and ambiguous outcomes.

Use an effect ledger

Every side-effecting operation should have a stable identity and an explicit lifecycle:

effect_id: effect_31
run_id: run_92a
tool: booking.change_flight
operation: commit_change
arguments_hash: sha256:...
idempotency_key: run_92a:change_flight:1
risk_class: high
required_scope: booking.write
approval_ref: approval_9
status: COMMITTED
external_receipt:
  provider_request_id: abc-123
  confirmation_id: MMT-9876
compensation:
  tool: booking.reverse_change
  eligibility: conditional

Recommended effect states:

PROPOSED
  -> VALIDATED
  -> AUTHORISED
  -> DISPATCHED
  -> ACKNOWLEDGED
  -> COMMITTED
  -> VERIFIED
 
or
 
DISPATCHED
  -> OUTCOME_UNKNOWN
  -> RECONCILING
  -> COMMITTED | FAILED | COMPENSATING

Separate prepare, authorise, commit, and verify

High-risk actions should not be represented as one opaque tool call. Prefer a transaction-shaped interaction:

prepare action
  -> return immutable preview, price, terms, expiry, and action hash
  -> obtain policy decision and approval against that frozen action
  -> commit using idempotency key
  -> reconcile external receipt
  -> verify business outcome
  -> compensate when policy and provider semantics allow it

This is a more important production pattern than any improvement in tool-call accuracy.

ContextOS implication

The Tool Gateway should own effect identities, idempotency, timeout classes, retry policies, receipts, reconciliation, and compensation metadata. The model should never have to remember whether an external write probably succeeded.


Law 6: Human-in-the-loop is a typed interrupt, not a chat message

Many systems implement approval as a conversational exchange:

Agent: Shall I proceed?
User: Yes.

That is insufficient for consequential actions. What exactly was approved? Did the price, recipient, request body, policy version, or tool arguments change before execution? Who approved it? Was the approval still valid when the action ran?

OpenAI Agents SDK exposes approval-required tool calls as interruptions that serialise and resume through RunState. LangGraph persists state at an interrupt and resumes with a structured value. Microsoft Agent Framework workflows similarly pause on approval-required tools.

The transferable principle is:

An approval is a decision over a frozen action proposal, not permission for the agent to “continue generally.”

A production approval object

approval_id: approval_9
run_id: run_92a
interrupt_id: interrupt_4
action_hash: sha256:...
action_summary:
  operation: change_flight
  traveller_count: 3
  incremental_amount: 4250
  currency: INR
risk_class: high
requested_from:
  authority_role: account_owner
  principal_id: user_123
decision: APPROVED
decided_at: 2026-07-25T14:05:02Z
expires_at: 2026-07-25T14:15:02Z
conditions:
  max_amount: 4250
  exact_itinerary_only: true
policy_snapshot_ref: policy_77

On resume, the runtime must revalidate:

  • the approver still has authority;
  • the approval has not expired or been revoked;
  • the action hash still matches;
  • the policy snapshot permits execution;
  • external preconditions have not materially changed;
  • the budget remains available.

If the fare changes, the approval does not silently stretch. The runtime prepares a new action and raises a new interrupt.

Human input is broader than approval

Typed interrupts should distinguish:

  • APPROVAL_REQUIRED — authority decision;
  • MISSING_INFORMATION — factual input needed;
  • CHOICE_REQUIRED — preference among valid alternatives;
  • POLICY_EXCEPTION_REQUESTED — explicit exception workflow;
  • EXTERNAL_CONFIRMATION_REQUIRED — out-of-band verification;
  • REVIEW_REQUIRED — quality or legal review.

The user interface can still render these conversationally. The runtime contract must remain typed.

ContextOS implication

Approvals belong in the run state and Decision Record. They should be durable, auditable, revocable, expiring, and bound to immutable action proposals.


Law 7: Containment, identity, policy, and approval are independent controls

Production agent security architectures separate sandboxing from approval policy. The sandbox defines what the process can technically access; approval determines when the agent must stop and ask before crossing a boundary. Mature runtimes similarly expose environment, permission, and tool restrictions as explicit runtime configuration.

Enterprise agents need an even stronger separation.

A complete authority decision is an intersection:

technical containment
  ∩ authenticated principal
  ∩ delegated identity scopes
  ∩ tenant policy
  ∩ tool allowlist
  ∩ data classification rules
  ∩ action-specific approval
  ∩ budget and rate limits
  = effective capability for this transition

Each control answers a different question:

ControlQuestion
Sandbox / execution boundaryCan this process technically reach or modify the resource?
IdentityWho is acting?
DelegationOn whose behalf, with which scopes, is it acting?
PolicyIs this class of action allowed under current rules?
Tool manifestIs this capability surfaced to this run?
ApprovalHas the right authority approved this exact action now?
BudgetIs the run still within cost, time, and action limits?

An approval must not:

  • disable the sandbox;
  • mint arbitrary credentials;
  • expose undeclared tools;
  • bypass tenant isolation;
  • override an absolute deny rule;
  • grant broader authority than the frozen action requires.

Automated approval review is instructive here: a separate reviewer can evaluate eligible sandbox-boundary escalations while the primary agent remains inside the same sandbox. The valuable pattern is separation of proposer and reviewer under unchanged containment, not “let another model approve everything.”

ContextOS implication

ContextOS Governance should represent capability expansion as a named, logged state transition. Policy evaluation should produce a decision object, not an invisible prompt fragment:

policy_decision:
  decision: REQUIRE_APPROVAL
  policy_ids: [refund_threshold_v3, customer_authority_v2]
  reasons:
    - amount exceeds autonomous limit
  required_authority: finance_approver
  constraints:
    max_amount: 25000
  evaluated_at: ...
  policy_snapshot_ref: policy_77

This makes authority explainable and testable.


Law 8: Tool discovery is not tool authority

MCP standardises how servers expose tools and their schemas. It also warns that tools can represent arbitrary code execution, that tool metadata should not automatically be trusted, and that hosts must mediate consent and safety. MCP authorisation is transport-level access control; it does not replace application policy or action-specific approval.

This distinction is fundamental:

  • a catalog says a capability exists;
  • a manifest says it is surfaced to this run;
  • authentication proves an identity;
  • authorisation grants scopes;
  • policy decides whether the action is eligible;
  • approval authorises a specific transition;
  • the gateway executes and records the effect.

A model seeing a tool definition must not imply it can call the tool under all circumstances.

Tool contracts need more than JSON Schema

A production tool contract should include:

name: payments.refund
version: 4.2.0
input_schema: ...
output_schema: ...
effect_class: reversible_write
risk_class: high
required_scopes: [payments.refund]
data_classification:
  input: restricted
  output: confidential
idempotency:
  supported: true
  key_field: request_id
retry_policy:
  safe_on: [timeout_before_dispatch, 429, 503]
  unsafe_on: [outcome_unknown]
timeouts:
  connect_ms: 1000
  total_ms: 10000
approval_policy_ref: refund_policy_v3
compensation_tool: payments.reverse_refund
observability:
  redact_fields: [card_token, bank_account]
owner: finance-platform

This contract gives the runtime enough information to govern execution without asking the model to infer operational semantics from prose.

Prevent tool confusion

The runtime should also defend against:

  • duplicate or ambiguous tool names;
  • schema drift after a run starts;
  • tools whose behaviour changes without versioning;
  • untrusted descriptions influencing policy decisions;
  • output injection entering the model as trusted instruction;
  • excessive tool exposure increasing choice entropy;
  • cross-tenant credentials reused by mistake.

Agent-loop engineering shows that dynamically changing tool lists can invalidate prompt caches. For ContextOS, the more important issue is reproducibility: the run should bind a stable tool-manifest hash so replay uses the same capability surface unless the test explicitly asks otherwise.

ContextOS implication

The Tool Gateway should be a policy enforcement and effect-management plane, not a thin RPC proxy.


Law 9: Multi-agent design is primarily an isolation strategy

Multi-agent systems are often presented as digital organisations: planner, researcher, critic, executor, supervisor, and so on. That metaphor can be useful, but it encourages unnecessary agent proliferation.

Production coding systems reveal a more concrete reason to delegate: context isolation and parallel work.

Subagents are useful when independent work can run in parallel or when exploratory logs and intermediate output would pollute the main thread. Production runtimes can give subagents separate prompts, tool restrictions, permission modes, hooks, and skills. Worktrees and ephemeral environments isolate concurrent changes from the primary workspace.

The correct design question is not “which persona should handle this?” It is:

Does this subtask need an independent context, capability set, failure boundary, budget, or execution environment?

Delegate only when a boundary is real

Good reasons to create a subagent or child run include:

  • independent research that can execute in parallel;
  • specialised tool access not required by the parent;
  • large noisy exploration whose details should be distilled;
  • a distinct trust or data boundary;
  • independent verification by a different policy or model;
  • separate workspace or transaction scope;
  • failure isolation;
  • different latency or cost profile.

Weak reasons include:

  • assigning human-like job titles;
  • making a linear workflow look sophisticated;
  • using debate to compensate for missing deterministic checks;
  • asking several agents the same question without a selection rule.

Every delegation needs a contract

delegation_id: delegation_12
parent_run_id: run_92a
child_run_id: run_92a_3
objective: verify_change_penalties
input_refs: [artifact_itinerary, artifact_fare_rules]
allowed_tools: [fare_rules.read, policy.lookup]
forbidden_effects: [write, notify, purchase]
budget:
  max_tokens: 18000
  deadline_ms: 30000
expected_output_schema: FarePenaltyAssessment
merge_policy: parent_validates

The parent should receive a typed artifact and evidence summary, not an unbounded transcript dump.

ContextOS implication

ContextOS should model delegation as child runs with explicit context, tools, budgets, and merge rules. Multi-agent orchestration then becomes a governed execution topology rather than a collection of prompts talking to one another.


Law 10: Promote repeated instructions into deterministic lifecycle checks

Hooks, callbacks, plugins, and setup workflows expose the same architectural seam: deterministic programs can execute before or after model calls, tool calls, file edits, permission requests, or run completion.

The lesson is not to add arbitrary hooks everywhere. It is to identify where a rule has become mechanical.

If a runtime can prove a condition, the model should not remain solely responsible for remembering it.

Examples:

Repeated instructionBetter enforcement point
“Do not send secrets to the model.”Pre-model redaction and data-loss-prevention gate
“Validate the schema before writing.”Pre-tool validator
“Run the test suite after a code change.”Post-edit or pre-completion hook
“Do not refund more than the captured amount.”Deterministic business invariant
“Citations must support the claim.”Post-generation grounding validator
“Do not book an expired fare.”Pre-commit freshness check
“Stop when the cost budget is exhausted.”Runtime budget governor

Lifecycle checks require their own governance

A hook is executable production logic. It therefore needs:

  • a declared owner;
  • versioning;
  • bounded inputs and outputs;
  • timeouts;
  • retry semantics;
  • structured error classes;
  • observability;
  • a fail-open or fail-closed policy;
  • security review;
  • replay tests.

A hook that silently rewrites prompts, bypasses policy, or mutates state without an event is an ungoverned agent by another name.

ContextOS implication

Harness Engineering should define lifecycle extension points as part of the public runtime contract. A policy validator, PII scanner, business-rule check, and completion evaluator should not each invent a different interception mechanism.


Law 11: A trace must reconstruct causality, authority, and artifact lineage

Agent observability cannot stop at recording prompts and completions.

OpenAI Agents SDK traces model generations, tool calls, handoffs, guardrails, and custom events. OpenTelemetry’s GenAI conventions standardise attributes such as model identity, token counts, tool calls, and tool results. A2A distinguishes messages, stateful tasks, streaming status, and artifacts.

These patterns point to a richer trace model.

A production trace should answer:

  1. What did the user ask?
  2. Which principal and tenant were bound to the run?
  3. Which policy snapshot applied?
  4. Which context items were selected, from which sources and versions?
  5. What plan or decision did the model produce?
  6. Which tools were visible?
  7. Which action was proposed?
  8. Why was it allowed, denied, or sent for approval?
  9. What external effect occurred?
  10. What evidence and receipt confirmed the effect?
  11. Which artifacts were produced from which inputs?
  12. What did verification conclude?
  13. How much time, token, and money did each step consume?
  14. Why did the run terminate in its final state?

Model content must be selectively observable

Full prompt and response capture can create privacy, security, and retention risk. Observability therefore needs data classes and capture modes:

metadata only
redacted content
sampled content
encrypted restricted content
no content, hash and lineage only

The trace schema should separate operational metadata from sensitive payloads so retention and access controls can differ.

Artifacts are first-class outputs

A final answer is not always the product. Runs may produce:

  • a booking proposal;
  • a reconciled ledger;
  • a code patch;
  • a report;
  • a policy decision;
  • a generated itinerary;
  • a customer communication;
  • an evaluation result;
  • a structured plan for another agent.

Each artifact should carry lineage:

artifact_id: artifact_55
type: booking_change_proposal
schema_version: 2.0
created_by: run_92a
input_artifacts: [artifact_12, artifact_19]
source_events: [evt_171, evt_174, evt_181]
policy_snapshot_ref: policy_77
content_hash: sha256:...
classification: confidential

ContextOS implication

The Decision Record should be derivable from runtime events and artifact lineage, not generated after the fact as a model-authored explanation. Explanations may be useful, but the audit foundation must be factual execution evidence.


Law 12: Stabilise the method before automating the cadence

Scheduled agent work is attractive because it appears to convert a useful prompt into a persistent capability. In practice, scheduling an unstable method multiplies its ambiguity and failure rate.

A recurring operation should not be considered production-ready merely because it succeeded interactively.

Before scheduling, the method should have:

  • a versioned workflow or skill;
  • explicit input and output schemas;
  • deterministic start conditions;
  • bounded tool and network access;
  • idempotent writes or compensation;
  • durable checkpoints;
  • retry and timeout policies;
  • ownership and service-level objectives;
  • cost and action budgets;
  • replay coverage;
  • monitor and rollback criteria.

The key separation is:

skill or workflow = method
schedule or event trigger = cadence
policy = authority
runtime = execution and recovery

A schedule is not a release credential.

Improvement jobs need an even stricter boundary

A recurring improvement process may:

  • inspect traces;
  • cluster recurring failures;
  • identify stale instructions;
  • propose tool-description changes;
  • generate new evaluation cases;
  • compare candidate prompts or workflows;
  • recommend promotion targets.

It should not directly mutate production policy, prompts, skills, or tools without the normal review and release path.

This prevents “self-improvement” from becoming an unaudited write channel into the harness.

ContextOS implication

The ContextOS Improvement Loop should separate four artifacts:

  1. Finding — what pattern was observed;
  2. Proposal — what should change and at which layer;
  3. Candidate — the exact versioned change;
  4. Release decision — evidence, approvals, rollout, and rollback conditions.

That is controlled adaptation. Anything less is prompt mutation.


Proposed runtime extensions around the canonical contract

The canonical execution contract remains RunContextCompiledContextToolEnvelopeDecisionRecord, with ReplayPacket carrying replay inputs. The list below distinguishes existing primitives from candidate orchestration extensions. Run, Event, Effect, Interrupt, and Artifact are proposals for the surrounding runtime; they are not silently added to the published schemas by this article.

1. Run

Candidate durable orchestration aggregate containing or referencing the canonical RunContext.

2. Event

An append-only record of every state transition and externally relevant occurrence.

3. Context Pack

A versioned manifest plus rendered context compiled for a specific run and decision stage.

4. Skill or Workflow

A reusable, versioned method with declared inputs, outputs, tools, checks, and ownership.

5. Tool Manifest

The stable, hashed capability surface visible to a run.

6. Effect

A potentially side-effecting operation with idempotency, receipts, reconciliation, and compensation semantics.

7. Interrupt

A durable pause requesting typed human or external input.

8. Policy Decision

A structured allow, deny, constrain, or require-approval result bound to a policy snapshot.

9. Artifact

A versioned output with schema, content hash, classification, and lineage.

10. Decision Record

A projection over events, policy decisions, evidence, approvals, and effects.

11. Evaluation Result

A reproducible assessment tied to a run or replay, evaluator version, rubric, and evidence.

12. Improvement Proposal

A governed recommendation to change a specific harness layer.

This model is more useful than a feature checklist because it defines ownership boundaries.


A reference execution flow

The following flow illustrates how the pieces fit together:

1. Request accepted
   - authenticate principal
   - bind tenant and delegation
   - create run
 
2. Context compiled
   - resolve intent
   - select policy, skills, facts, memory, and tools
   - persist context manifest
 
3. Agent proposes plan or next action
   - runtime records model event
   - validator checks schema and budgets
 
4. Tool action prepared
   - Tool Gateway validates contract
   - effect ledger creates stable effect_id and idempotency key
 
5. Policy evaluated
   - allow, deny, constrain, or require approval
 
6. If approval is required
   - freeze action hash
   - create typed interrupt
   - checkpoint and release compute
 
7. Run resumes
   - validate approver, expiry, action hash, and policy
   - dispatch effect
 
8. External result reconciled
   - capture receipt
   - resolve ambiguous outcome
   - retry only under declared semantics
 
9. Outcome verified
   - deterministic checks
   - model-based evaluation where appropriate
   - compensate or escalate on failure
 
10. Run completed
   - emit final artifacts
   - materialise Decision Record
   - close budgets and telemetry
 
11. Improvement Loop observes
   - aggregate failure patterns
   - create proposals, not direct mutations

This is what “agent runtime” should mean in ContextOS.


Anti-patterns these runtimes expose

Anti-patternWhy it failsReplacement
Giant global system promptScope collision, stale rules, poor adherence, high token costCompiled context with scoped instructions and progressive disclosure
Conversation as stateCannot reliably recover, audit, or resumeDurable run state and event history
Memory as policyProbabilistic context cannot enforce authorityDeterministic policy engine and capability gateway
“Yes” as approvalNot bound to exact action, amount, expiry, or authorityTyped interrupt over immutable action hash
Retry the whole agentRepeats expensive reasoning and may duplicate side effectsCheckpointed execution and idempotent effect ledger
Tool schema equals permissionDiscovery is confused with authorityStable manifest plus identity, scopes, policy, and approval
Multi-agent swarm by defaultHigher cost, coordination noise, unclear ownershipChild runs only for real isolation or parallelism boundaries
LLM critic as final gateStatistical judgement cannot replace machine-checkable invariantsDeterministic validators plus calibrated model evaluation
Raw logs as observabilityNo causal model, policy lineage, or artifact traceabilityTyped events, spans, Decision Records, and artifact lineage
Cron-wrapped promptRepeats an unstable, non-idempotent methodVersioned workflow first, cadence second
Agent self-rewrites production promptsNo provenance, replay, review, or rollbackImprovement proposals through governed release
Approval expands all permissionsViolates least privilege and containmentNarrow transition-specific capability grant

What should change in ContextOS

The original five harness patterns remain valid, but they are only part of the required architecture. The production-runtime evidence suggests the following expanded changes.

ContextOS surfaceRequired improvement
Runtime kernelMake Run, lifecycle state, event history, checkpoint, retry, cancellation, and resume first-class
Context Pack CompilerCompile typed, scoped, versioned context; record selection lineage and trust
Harness EngineeringDefine the smallest durable layer for corrections and standard lifecycle extension points
Tool GatewayAdd stable manifests, policy mediation, effect ledger, idempotency, reconciliation, and compensation
GovernanceSeparate sandbox, identity, delegation, policy, approval, and budgets; bind approval to immutable actions
Workflow engineCombine model-directed loops with deterministic graphs and durable waits
Multi-agent orchestrationRepresent delegation as child runs with explicit contracts, budgets, tools, and merge rules
ObservabilityAdopt causal events and spans covering context, model, tool, policy, approval, effect, and artifact lineage
EvaluationEvaluate lifecycle transitions and business outcomes, not only final responses
Improvement LoopPromote corrections through evidence, target-layer selection, replay, canary, and rollback
AutomationSchedule only pinned, replay-tested methods with narrow capabilities and operational ownership

This does not make ContextOS “self-rewriting.” It makes the runtime self-observing, proposal-generating, and safely improvable.

That is a much stronger and more defensible claim.


The broader lesson

Production agent runtimes are gradually rediscovering disciplines that distributed systems, workflow engines, security architecture, and control planes have refined for decades.

Agents do not remove the need for those disciplines. They make them more important because the planner is probabilistic, the execution path is dynamic, and the inputs can be adversarial or incomplete.

The durable design rules are therefore straightforward:

  • One-off constraints belong to the run.
  • Local recurring guidance belongs near the work.
  • Reusable methods belong in versioned skills or workflows.
  • Machine-checkable invariants belong in deterministic gates.
  • External actions belong in an effect ledger.
  • State belongs in durable storage.
  • Authority belongs in identity, policy, and typed approvals.
  • Tools belong behind a capability gateway.
  • Delegation belongs in isolated child runs.
  • Evidence belongs in causal traces and artifact lineage.
  • Learning belongs in a governed promotion pipeline.
  • Cadence belongs around a method that has already proved stable.

The model remains important. It provides interpretation, planning, adaptation, synthesis, and judgement where deterministic software is insufficient.

But the harness determines whether that intelligence becomes a reliable production capability or merely a persuasive sequence of tokens.

The model proposes. The runtime persists. Policy constrains. Humans authorise. Tools effect. Evidence proves. Replay improves.

That is the production architecture ContextOS should keep.

Primary sources consulted

Found this useful? Share it.

Share:XBSMRedditHNEmail

Continue through the same topic without returning to the index.

View the series