The Production Harness Roadmap: Lessons from Claude Code, Copilot Studio, and Thea

The fastest way to misunderstand harness engineering is to reduce it to a prompt wrapper.

A production harness is the machinery that decides what context enters a run, which instructions win, which tools exist, what actions are allowed, how state survives, how failures recover, what evidence is retained, how cost is bounded, and whether a new build may replace the old one.

The August 3–13, 2026 evidence makes those responsibilities unusually visible. Microsoft’s Copilot Studio documentation turns harness selection into a product, channel, and billing choice. Anthropic’s Claude Code changelog reads like a live catalogue of runtime boundary failures and repairs. Thea shows what happens when a coding-agent loop controls a physical system and rollback is no longer cheap.

The roadmap below translates those signals into an engineering sequence. It assumes the organization already has an agent prototype and needs to make it governable. It does not assume one model vendor or one framework.

The six invariants of a production harness

Before choosing features, define what must remain true.

Invariant 1: every behavior-shaping input is versioned

The release is a tuple:

(model, harness, instructions, tools, policy, memory schema, evaluators)

If any element changes, the runtime changed. A model alias that silently advances, a project instruction added below the repository root, a cloud skill that shadows a local command, or a hook delivered by a server can alter behavior without an application-code diff.

Invariant 2: policy is enforced outside probabilistic reasoning

The model may propose an action. It must not be the authority that decides whether the action is allowed.

Authorization needs a deterministic input: actor, task, resource, action, arguments, environment, risk class, and prior approvals. Ambiguous parsing must fail closed. Hooks can narrow or enrich a request but must not bypass the tool gateway.

Invariant 3: success requires postcondition evidence

Tool transport success is not business success. A file write may have hit the wrong path. A payment API may return a successful request while the transfer remains pending. A robot gripper may move while failing to hold the object.

The harness needs an evaluator appropriate to the consequence of a false success.

Invariant 4: partial execution has a defined recovery state

Streams end early. Workers disappear. compaction fails. Messages expire. Tools time out after committing side effects. Cleanup races with durable memory.

Every side-effecting step needs an idempotency key or a reconciliation strategy. Every run state needs an owner, a durable checkpoint, and a legal next transition.

Invariant 5: evidence is durable enough to explain the decision

A transcript is useful but incomplete. An audit-grade trace includes instruction provenance, compiled context identity, tool requests and responses, policy decisions, approvals, state transitions, evaluator verdicts, artifacts, and release versions.

Invariant 6: improvement is gated

The harness may propose changes to prompts, skills, routing, context, or tools. It must not promote them because one score improved. Promotion requires a representative evaluation set, critical-rule gates, cost and latency checks, canary exposure, and rollback.

These invariants turn a vague agent platform into a testable runtime contract.

What the latest implementations teach

Copilot Studio: capability is a workload class

Microsoft distinguishes a structured standard harness, a reasoning-heavy GitHub Copilot harness, and a Microsoft 365 Copilot Chat extension harness. The selection affects capabilities, authoring, consumption, and publishing surfaces.

That suggests a better platform abstraction than one universal “agent”:

Do not expose every capability to every task. Route work to the smallest runtime class that can complete it.

Claude Code: the attack surface is in the seams

Anthropic’s August releases repaired or hardened seams across the runtime:

• command normalization and approval classification;
• invisible Unicode, tabs, trailing slashes, IPv6 forms, and dangerous flags;
• worktree isolation for destructive Git operations;
• credential masking and permission-bypass policy;
• hook behavior before tool execution;
• plugin archives, SHA pinning, marketplace precedence, and cloud-skill shadowing;
• cross-session messages, inbox delivery, expiry, offline workers, and agent listing;
• project-memory cleanup, compaction failure, partial streams, retries, and OAuth redirects;
• self-hosted runner directories and server-supplied hooks;
• staggered fan-out for prompt-cache reuse.

The common failure is not weak reasoning. It is a disagreement between two layers about what an action means, where state belongs, which extension has authority, or whether a partially completed operation is safe to repeat.

The roadmap must therefore prioritize boundary tests before agent cleverness.

Thea: advice and authority are different things

Thea applies coding-agent patterns to robots. It maintains a persistent scene graph, refreshes a compact state brief, allows one physical action per turn, runs deterministic pre- and post-hooks, and invokes an evaluator after action execution.

Its sharpest design rule is that skills provide advice while tools provide enforceable contracts. A skill can explain how to grasp an object. The physical tool must still validate that the object exists, the target is reachable, the action is within limits, and the safety system permits it.

Software agents need the same separation:

The model is inside the loop. It is not the perimeter.

Days 0–15: enumerate the runtime

The first milestone is not a feature. It is a bill of materials.

Build the runtime manifest

For every environment, record:

• exact model provider, model, revision, and service tier;
• harness or framework version;
• system and developer prompt hashes;
• discovered project-instruction files and their scopes;
• skill sources, versions, descriptions, and bodies;
• tool and MCP-server versions, schemas, and credentials;
• hooks and where their code originates;
• policy bundle and approval rules;
• memory stores, schemas, retention, and cleanup ownership;
• evaluator bundle and judge versions;
• sandbox filesystem, process, network, and secret configuration.

Do not serialize secrets into the manifest. Record secret identity, scope, owner, and rotation version.

Define runtime classes

Map task families to the smallest required capability envelope. A documentation edit does not need production database credentials. A read-only research fan-out does not need a writable workspace. A refund execution agent should not inherit the exploratory shell access of a coding agent.

Exit gate

Pick any production trace. The team must be able to name the exact runtime tuple and reconstruct which instructions, extensions, and policies could have influenced it. If it cannot, stop adding autonomy.

Days 16–30: enforce action boundaries

Create one tool gateway

All consequential actions should cross a shared admission layer. The gateway should normalize arguments before policy evaluation, resolve paths, reject ambiguous encodings, bind credentials at execution time, and produce a durable decision record.

type ActionRequest = {
  runId: string
  actor: { userId: string; agentId: string; tenantId: string }
  task: { intentId: string; riskClass: string }
  tool: { name: string; version: string; publisher: string }
  arguments: unknown
  environment: { workspaceId: string; sandboxProfile: string }
}
 
type AdmissionDecision =
  | { status: "allow"; normalizedArguments: unknown; policyVersion: string }
  | { status: "deny"; reasonCode: string; policyVersion: string }
  | { status: "approval-required"; packetId: string; policyVersion: string }

Avoid policies that parse only the first token of a shell command or inspect raw strings without normalization. Paths, flags, quotes, environment expansion, Unicode, redirects, and command composition all change semantics.

Separate hook power from policy power

Hooks can add telemetry, inject environment context, or veto a request. A hook should not be able to turn a denied action into an allowed one. Server-supplied hooks and dynamic plugin sources need signature, origin, version, and tenancy checks.

Add postconditions

For every side-effecting tool, define:

• expected state transition;
• observation source independent of the command response where possible;
• success, incomplete, failed, and unknown states;
• retry safety and idempotency behavior;
• escalation route for uncertainty.

Exit gate

Run adversarial admission tests covering whitespace, control characters, invisible Unicode, path traversal, symlinks, alternate IP forms, shell composition, dangerous flags, hook rewriting, and credential leakage. Critical ambiguity must deny or request approval.

Days 31–45: make state and failure explicit

Model the run as a state machine

At minimum:

created → context-compiled → planning → admission → executing
        → observing → evaluating → completed
                               ↘ approval-pending
                               ↘ repair-pending
                               ↘ reconciliation-required
                               ↘ failed-terminal

Do not use one generic failed state. A retryable provider timeout, a committed tool call with a lost response, a rejected approval, and invalid postcondition evidence require different next actions.

Design distributed messaging as a protocol

Cross-session agents require message IDs, sender and recipient identities, task lineage, sequence or causal metadata, expiry, acknowledgement, and replay protection. “Message sent” and “message delivered” are separate states. Offline workers need an inbox policy; expired workers must not regain stale authority.

Protect memory from cleanup and compaction

Separate resident runtime state, refreshed context, accumulated run evidence, durable project memory, and temporary files. Each class needs a namespace, retention rule, and cleanup owner. A session cleanup routine should never infer ownership from a broad directory pattern.

Compaction must emit a receipt: source range, summary artifact, retained critical facts, dropped material, model and prompt version, and recovery pointer to the original trace.

Exit gate

Interrupt the runtime during planning, approval, tool execution, response streaming, compaction, memory promotion, and worker messaging. Demonstrate that each run either resumes safely or enters reconciliation without duplicating a consequential action.

Days 46–60: evaluate rules and trajectories

Compile an instruction registry

The Harness-IF study shows why a single final score is insufficient. Store each rule’s surface, authority, owner, applicability, conflict group, evaluator, and criticality. Include against-prior cases that prove the agent read and obeyed the instruction rather than coincidentally matching it.

Normalize traces across candidates

Borrow the protocol separation from A2E: task input, harness binding, runner, and task trace. Compare the current build and candidate build on paired tasks. Score:

• task outcome;
• instruction compliance;
• policy and approval behavior;
• tool selection and arguments;
• postcondition accuracy;
• recovery and duplicate suppression;
• latency, tokens, tool cost, and human effort.

Calibrate judges

Use deterministic evaluation whenever the trace contains an objective event. For semantic judges, retain model, prompt, votes, evidence, and uncertainty. Periodically compare judge verdicts with human review. Do not permit a judge’s high average to override a failed critical deterministic rule.

Exit gate

Every release packet contains paired candidate-versus-current results, rule-level verdicts, critical failures, judge disagreement, representative traces, and a written decision.

Days 61–75: secure the extension supply chain

Skills, plugins, MCP servers, hooks, and remote workers form a software supply chain even when their payload is prose.

Require provenance and non-shadowing rules

• Pin package and archive content by immutable digest.
• Record publisher and installation source.
• Prevent cloud-synced skills from silently shadowing local commands or MCP servers.
• Sanitize discovery metadata before it enters trusted prompts.
• Do not execute embedded shell directives or file expansions while merely loading a skill.
• Make marketplace and workspace precedence explicit.
• Support revocation and forced disable independent of the package source.

Claude Code’s August changes around SHA-pinned archives, synced skill isolation, sanitized descriptions, and plugin precedence are concrete examples of why prose-bearing extensions need code-grade controls.

Constrain self-hosted execution

Validate base directories, sandbox profiles, tenancy, credential mounts, callback addresses, and hook sources. The worker should receive task-scoped capability grants, not long-lived platform credentials. Heartbeats do not prove correct execution; traces and postconditions do.

Exit gate

Run a malicious-extension drill: shadow a trusted name, inject hostile discovery text, request an undeclared credential, alter a hook origin, revoke a package mid-run, and deliver a stale worker message. Verify deterministic containment and an auditable failure.

Days 76–90: optimize and improve safely

Measure cost per accepted outcome

Raw tokens reward incomplete systems. Use:

total run cost + tool cost + human-review cost + retry cost
-----------------------------------------------------------
                  accepted outcomes

Segment by task class and risk. Include cache-hit rate, compaction overhead, fan-out width, duplicate work, and postcondition repair loops.

Anthropic’s staggered dynamic fan-out to reuse prompt cache is a good example of harness-level economics: orchestration order changes cost even when the model and task do not.

Gate automated improvement

An improvement loop may propose a prompt edit, skill update, routing change, context policy, or tool description. It should produce a candidate artifact—not mutate production.

promotion_gate:
  required:
    critical_rule_failures: 0
    permission_ambiguities: 0
    unreconciled_side_effects: 0
    outcome_floor: "not below current lower confidence bound"
    against_prior_floor: "team-defined by risk class"
    cost_ceiling: "team-defined by workload"
  stages:
    - offline_replay
    - adversarial_suite
    - shadow
    - limited_canary
    - reviewed_promotion
  rollback:
    automatic_on_critical_failure: true
    owner: agent-platform-oncall

The quoted values are policy placeholders, not universal thresholds. Derive numbers from baseline distributions and consequence of failure.

Expand side effects last

Increase authority only after the narrower class has stable outcome, compliance, recovery, and evidence metrics. Moving from read to write, reversible to irreversible, or sandbox to production is a new release class—not a feature flag flip.

Exit gate

The team can improve latency or cost without weakening critical compliance, and it can roll back the complete runtime tuple within the declared recovery objective.

The operating scorecard

Track one scorecard per workload class:

Avoid rolling the table into one composite score. A cost improvement must not hide a permission regression.

Architecture decisions teams should make explicitly

One harness or several?

Prefer several bounded runtime classes when task topology and authority differ materially. Share trace, policy, identity, and release contracts so the classes remain governable.

One global instruction file or scoped instructions?

Prefer scoped rules with explicit authority and discovery. Duplicated global prose creates conflict and token overhead. Test precedence rather than assuming it.

Model-generated plans or deterministic workflows?

Use deterministic workflows where the process is known and variance has no value. Use model planning where the environment is open-ended, then constrain execution through the same tool and policy boundary.

Autonomous retries or human escalation?

Retry only when the failure class is known to be safe and idempotent. Escalate when an action may have committed, the observation is uncertain, or authority changed.

Skills or tool constraints?

Use skills to teach strategy and tools to enforce capability. Never rely on prose alone for a safety property.

What the roadmap deliberately does not promise

Thea’s experiments are promising but limited to three robot platforms and a modest number of trials. Its evaluator produced false-success classifications, and the project explicitly disclaims safety certification. Its architecture is an insight, not proof that language-model evaluators are safe controllers.

Claude Code’s changelog demonstrates active implementation work, not an independently verified residual-risk level. Rapid patch cadence also creates rollout risk. Test the exact version and configuration you plan to operate.

Copilot Studio’s harness categories are product choices inside Microsoft’s ecosystem. They do not establish a universal taxonomy or prove one harness is best.

Harness-IF and A2E are recent preprints with material limitations in item selection, judge dependence, task count, and instrumentation. They should reshape the evaluation design, not become procurement certificates.

The 2027 horizon

If this direction holds, mature agent platforms will make five things ordinary:

1. Harness manifests will sit beside model cards, describing instruction surfaces, tools, permissions, memory, traces, and evaluators.
2. Runtime classes will be chosen per workload instead of exposing one maximally capable agent.
3. Rule-level regression suites will test instruction conflicts and omissions across harness versions.
4. Postcondition protocols will distinguish transport success from verified world-state change.
5. Harness release engineering will promote whole evaluated bundles with canaries and rollback, not swap model aliases in place.

The future roadmap is therefore less about making the loop longer. It is about making every boundary explicit: advice versus authority, request versus effect, message sent versus delivered, temporary state versus durable memory, candidate improvement versus promoted release.

That is how a probabilistic agent becomes a production system.

What to read next

• Harness Engineering, August 3–13, 2026: The Runtime Became the Product
• Your Agent Followed the Rule—or Did It? Harness-IF and A2E Change Agent Evaluation
• The Agent Harness Whitepaper
• Build the Tool Gateway: Deterministic Control for Agent Actions
• Autotune: The Harness Improvement Loop