Harness Engineering in August 2026: The Control Plane Gets Measured

Harness engineering crossed an important threshold in August 2026. It is no longer only a useful name for the code around a model. It is becoming a measurable engineering discipline with runtime products, portable trace formats, lifecycle security benchmarks, skill-level release gates, and an emerging standards stack.

The change is easy to miss because no single launch created it. The evidence arrived in pieces:

• OpenAI made the harness visible as the shared execution layer behind Codex surfaces, then described how repository design, structural tests, observability, and review loops let agents produce a large codebase.
• Microsoft exposed harness choice as a product decision in Copilot Studio and documented an opinionated Agent Framework harness for long-running research, coding, and analysis.
• DeepSeek released a composable harness whose runtime is built from plugins, alongside Cordis experiments in self-referential adaptation.
• Harness-IF and A2E moved evaluation from final answers toward instruction surfaces, trajectories, and framework effects.
• HarnessRisk tested attacks across configuration, extensions, runtime, persistent state, action, and recovery.
• ACES and NVIDIA SkillEvaluator treated skills as executable release artifacts whose marginal value can be measured with paired live trials.
• The Agent Trajectory Interchange Format, or ATIF, gave evaluators a cross-harness representation for messages, tool calls, observations, metrics, and subagent trajectories.
• Agent2Agent joined the Agentic AI Foundation, placing agent-to-agent communication beside other open agent infrastructure—but leaving authorization and end-to-end accountability to the runtime.

The combined conclusion is stronger than “the harness matters.” The deployed system—not the model—is the release unit. That system includes instructions, skills, tools, identities, policy, memory, sandboxing, orchestration, trace semantics, evaluators, and recovery behavior.

This report is current through August 24, 2026. It distinguishes shipped products, public implementations, preprints, standards work, and vendor claims. A linked source is evidence that something was documented or released; it is not automatic proof that the system is safe or superior.

What harness engineering means now

A useful definition has to be broad enough to describe a real runtime and narrow enough to remain an engineering category.

Harness engineering is the discipline of designing, versioning, evaluating, and operating the control system through which a model observes context, invokes capabilities, changes state, produces evidence, and recovers from failure.

That control system has at least eight responsibilities:

1. compile task-relevant context;
2. expose tools and skills;
3. carry identity and bounded authority;
4. orchestrate turns, workers, and handoffs;
5. isolate execution and constrain side effects;
6. record decisions, actions, and observations;
7. evaluate outcomes, rules, safety, cost, and recovery;
8. promote improvements through a reversible release process.

This is more precise than calling every prompt template a harness. A prompt is one behavior-shaping input. A framework is one implementation substrate. A sandbox is one containment boundary. A harness is the governed composition that decides how those parts participate in a run.

The definition also exposes a common category error. Model capability and harness quality are related but not interchangeable. A stronger model can improve planning while the same deployment remains unsafe because the gateway accepts ambiguous commands, memory can be poisoned, or an approval is not bound to the exact effect.

The evidence ledger: how the field arrived here

The August 3–13 evidence is analyzed in the earlier field report. The important delta since then is not another general framework. It is measurement at two previously weak layers: lifecycle safety and skill utility.

Shift 1: the release unit became a tuple

Most release processes still name the model and perhaps the application version. That is not enough to reconstruct behavior.

A production agent should be identified by a tuple such as:

release = (
  model,
  harness_build,
  instruction_manifest,
  skill_bundle,
  tool_catalog,
  policy_bundle,
  memory_schema,
  sandbox_profile,
  evaluator_set
)

This is not bookkeeping for its own sake. Harness-IF showed that the surface carrying a rule—system prompt, project file, skill description, tool description, or user instruction—can affect compliance. The Scaffold Effect showed that different harnesses can produce different cost and failure patterns around the same model. HarnessRisk showed that a model’s safety result can change sharply across harnesses. ACES showed that the value of a skill must be estimated under a fixed model, harness, workspace, task, and scorer.

The model-only release label erases every one of those variables.

Version the tuple as an immutable manifest and bind it to the run record:

release_id: support-agent-2026-08-24.3
model: provider/model-version
harness:
  name: support-runtime
  version: 4.8.1
instructions_sha256: "..."
skills:
  - id: refund-investigation
    version: 2.3.0
tools_sha256: "..."
policy_sha256: "..."
memory_schema: 7
sandbox_profile: network-deny-default-v4
evaluators:
  outcome: 12
  instruction: 8
  lifecycle_security: 3
  recovery: 5

If a production run cannot be mapped back to this level of specificity, regression analysis will eventually collapse into anecdotes.

Shift 2: safety became a lifecycle property

Traditional agent-security tests concentrate on a malicious webpage, email, document, or tool response encountered during execution. HarnessRisk expands the attack surface in both directions.

Its 128 sandboxed cases cover six phases:

1. Harness configuration: credentials, connectors, permissions, and policy settings.
2. Capability extension: plugin or skill selection, installation, update, and permission boundaries.
3. Runtime operation: untrusted messages, pages, documents, and tool outputs.
4. State persistence: memory, identity, preferences, policies, and durable triggers.
5. Action control: payments, deployments, deletions, grants, and communications.
6. Incident recovery: investigation, rollback, credential rotation, state repair, and evidence preservation.

Across 14 model–harness configurations, the paper reports attack-success rates from 12.6% to 80.9% while benign utility remains between 75.0% and 97.6%. Configuration was the most vulnerable phase on all three evaluated harnesses. For one model, changing the harness moved attack success from 54.7% to 12.6%.

The most operationally important result is that detection is not control. Some configurations recognized risk in more than 90% of runs while still allowing substantial attack success. An agent can say that content looks malicious and then continue to use it, preserve contaminated state, or perform the requested effect.

This requires four separate verdicts:

A single “passed” flag hides the useful-but-unsafe quadrant. HarnessRisk reports that 38% to 59% of trajectories, depending on harness, completed the legitimate task while an attack also succeeded.

The practical rule is simple: a risk-detection model may create evidence, but only a deterministic policy or tool boundary can deny an effect.

Shift 3: skills became executable release artifacts

Skills are increasingly the application layer of agent systems: procedural instructions, scripts, references, examples, and metadata that a runtime loads when a task appears to match. Until recently, most skill gates were static. They checked schema, style, security smells, or whether a document seemed well written.

ACES asks the missing question: does the skill help a live agent under controlled conditions?

For each task, ACES compares two runs:

• a treatment run with the target skill available;
• a baseline run with the target skill withheld while the model, harness, workspace, task, and scorer remain fixed.

The paired difference is Skill Lift.

The study analyzed 145 skills and reports 947 scored paired cases from 58 of 64 production skills across four primary harnesses. Mean composite Skill Lift was 0.2134, with a 95% paired-case confidence interval of 0.1967 to 0.2301. Composite lift was positive in 72.8% of paired cases. The largest gains appeared in process measures such as skill execution, behavior checks, and skill efficiency.

Those numbers do not mean “skills add 21% performance” in general. Lift is conditional on the task distribution, model, harness, competing skills, environment, and scoring contract. More than a quarter of paired cases did not show positive composite lift. A skill can also improve task completion while weakening security or adding unacceptable cost.

The corresponding implementation, NVIDIA’s SkillEvaluator, combines:

• deterministic schema, PII, license, Unicode, and lint checks;
• semantic overlap and context-quality checks;
• synthetic evaluation-dataset generation;
• live agent trials in local, Docker, or cloud environments;
• optional blocking on agent-evaluation results.

The project labels itself experimental and says Tier 3 live evaluation is advisory by default. That is the correct level of maturity to report. It is usable evidence infrastructure, not a production certification authority.

Shift 4: traces became portable evaluation inputs

Cross-harness evaluation fails when every runtime emits a different transcript shape. ACES normalizes trajectories into the Agent Trajectory Interchange Format, or ATIF.

ATIF v1.7 represents:

• agent and model metadata;
• ordered system, user, and agent steps;
• tool calls and linked observations;
• token, cache, cost, and call metrics;
• continuation references;
• embedded or externally referenced subagent trajectories.

This matters because the same evidence can support debugging, visualization, evaluation, supervised fine-tuning, and reinforcement-learning pipelines. A standardized trace also lets a benchmark compare multiple harnesses without letting one harness’s bespoke logging determine what can be scored.

But trace interoperability is not replay equivalence. Two harnesses may interpret permissions, retries, compaction, parallel calls, or cancellation differently even when their traces share a schema. ATIF records a trajectory; it does not by itself guarantee that another runtime can reproduce the same decision or external effect.

Use a portable trace alongside a stronger decision record that binds:

• run identity and release manifest;
• principal, tenant, and business purpose;
• approval mode and authority scope;
• policy verdicts;
• tool request and normalized effect;
• precondition and postcondition evidence;
• durable-state mutations;
• recovery and compensation status.

Shift 5: interoperability widened the trust boundary

On August 17, Agent2Agent became a hosted project of the Agentic AI Foundation. The A2A specification and SDKs give agents a common way to advertise capabilities, exchange messages and artifacts, stream results, and manage long-running tasks without exposing internal memory or tools.

This is meaningful infrastructure. It is not an end-to-end trust runtime.

Consider a delegated task:

A2A can standardize delegation. MCP can standardize tool and resource access. Neither automatically answers:

• whether authority narrowed when the task was delegated;
• whether the specialist’s identity is bound to the originating principal;
• whether a tool call still serves the original business purpose;
• who owns a failed or partial effect;
• how evidence from both protocols becomes one auditable run;
• how cancellation, timeout, or compensation propagates across the chain.

The roadmap in the A2A repository includes richer authorization data in Agent Cards and improved streaming and push reliability. Those will help. The harness still has to enforce continuity across the protocols.

What is genuinely implemented today

The field now has concrete implementations at several layers:

This table also clarifies what has not been solved. There is no universal harness leaderboard, portable approval semantics, end-to-end delegated identity standard, or accepted certification for an adaptive harness.

What practitioners are actually debating

The public conversation often collapses into slogans: “the model is a commodity,” “agents should write all code,” “everything is a plugin,” or “the harness is the moat.” The evidence supports narrower, more useful positions.

Is the model becoming irrelevant?

No. Model capability still affects reasoning, tool use, instruction following, latency, and cost. The newer result is that model quality does not dominate every system outcome. Harness design can change safety, failure patterns, and economics enough that the deployed pair must be evaluated together.

Should rules live in instructions or code?

Both, according to consequence. Instructions communicate goals, workflow, and judgment criteria. Deterministic code should enforce permissions, schema validity, budget limits, idempotency, isolation, and high-risk invariants. A rule that merely tells an agent not to exfiltrate a secret is not a security boundary.

Are skills the new application package?

Often, yes—but a skill is not just documentation. It is a behavior-changing dependency with routing, provenance, conflict, security, cost, and regression properties. It needs a manifest, task set, baseline, trace, and promotion gate.

Will one open protocol unify the agent stack?

Unlikely in the near term. A2A, MCP, ATIF, OpenTelemetry, and runtime-specific protocols solve different problems. The valuable goal is not one universal wire format. It is continuity of identity, authority, evidence, and lifecycle state across the formats a workflow actually uses.

Can the harness improve itself?

It can propose changes. It should not silently grant those proposals authority. Adaptive systems need an immutable outer loop that keeps held-out evaluation, policy, approval, canarying, rollback, and audit outside the component being optimized.

The architecture implied by the newest evidence

The newest work converges on a three-loop architecture:

The separation matters:

• The execution loop completes one bounded task.
• The evaluation loop judges the task and trajectory using evidence not controlled solely by the executor.
• The improvement loop creates a new candidate and sends it back through evaluation.

When these loops are fused, the system can optimize the metric, rewrite the rule, discard inconvenient evidence, or promote a change that would have failed a stable gate.

A practical 30-day response

Teams do not need to adopt every new project. They should absorb the design lessons in a controlled order.

Week 1: enumerate the release

• Write the model–harness release tuple.
• Hash instructions, skills, tools, policy, and evaluator configuration.
• Record the sandbox and identity profile.
• Make every run point to the immutable manifest.

Week 2: normalize the evidence

• Export ordered messages, tool calls, observations, metrics, and subagent links.
• Add principal, purpose, approval, policy, effect, and postcondition fields that a generic trace format does not carry.
• Preserve partial and failed runs, not only successful transcripts.

Week 3: add two missing evaluations

• Build paired with-skill and without-skill trials for one consequential skill.
• Add lifecycle attacks at configuration, extension, persistence, and recovery—not only malicious runtime content.
• Score utility and attack success independently.

Week 4: create a real promotion gate

• Block critical permission, persistence, and duplicate-effect failures.
• Require a minimum Skill Lift on the target task slice without a security, latency, or cost regression.
• Canary the complete release tuple.
• Bind rollback to the same manifest and evidence packet.

What the evidence does not yet prove

The newest results are valuable and preliminary.

• HarnessRisk is a v1 preprint covering 128 sandboxed cases, three open harnesses, six models, and controlled mock services. It does not establish real-world compromise rates for commercial runtimes.
• Its process backend confines intended fixtures but is not an operating-system sandbox; the repository explicitly recommends stronger isolation for untrusted code.
• ACES uses a mix of deterministic, security, LLM-judge, and optional domain metrics. Its mean lift is not portable to a different task distribution.
• SkillEvaluator is experimental and community-supported. Static checks and live trials reduce uncertainty; neither proves the absence of malicious behavior.
• ATIF is an active project format, not a universal standard adopted by every major runtime.
• A2A joining AAIF is a governance milestone, not evidence that cross-vendor agent workflows are safe, reliable, or semantically uniform.
• Vendor engineering reports reveal useful implementation patterns but do not replace independent replication.

The right response is neither dismissal nor hype. Use the artifacts, reproduce the experiments on your workload, and keep the evidence class attached to every claim.

The roadmap ahead

The next phase of harness engineering will be defined less by adding another planner and more by closing six system gaps:

1. Release manifests: a standard way to identify the full model–harness–policy–skill build.
2. Proof-carrying actions: tool effects returned with verifiable postconditions, authorization receipts, and compensation state.
3. Lifecycle security: repeatable configuration, extension, memory, action, and recovery attacks in every release suite.
4. Skill supply-chain evidence: provenance, static scanning, paired lift, conflict tests, and revocation for procedural packages.
5. Delegated authority: identity and purpose that attenuate across A2A handoffs and remain bound to downstream tool calls.
6. Causal improvement: offline candidates, held-out evaluation, canaries, and rollback rather than online self-mutation.

The detailed sequence is published in The Harness Engineering Roadmap for 2026–27. The implementation guides go deeper on lifecycle security testing and paired Skill Lift in CI.

The durable conclusion

Harness engineering began as a practical observation: capable models still need good environments, tools, feedback, and constraints. By August 2026, that observation had become a system discipline.

The strongest teams will not ask only which model scored highest. They will ask:

• Which complete release tuple produced the outcome?
• Which rules were actually followed?
• Which unsafe effects remained possible?
• Which durable state changed?
• What marginal value did each skill add?
• Can the trajectory be inspected across harnesses?
• Can the action be proven, compensated, and replayed?
• Can the next improvement be rejected safely?

That is the frontier. The model reasons inside the system. The harness determines whether the system deserves authority.

What to read next

• HarnessRisk in Practice: Turn the Agent Lifecycle Into a Security Test Matrix
• Skill Lift: How to Gate Agent Skills With Paired Live Evals
• The Harness Engineering Roadmap for 2026–27
• Harness Engineering: The Runtime Became the Product
• Agent Harness Whitepaper