The most important agent-engineering work published between August 3 and August 13, 2026 did not come from one model launch. It came from five different directions that converged on the same conclusion: the harness around the model is becoming the real product boundary.
Microsoft exposed harness choice as an explicit Copilot Studio product decision. Anthropic shipped a dense run of Claude Code changes around sandbox boundaries, distributed sessions, plugin trust, recovery, and prompt-cache economics. Two new research efforts—Harness-IF and A2E—made instruction placement and framework behavior measurable. Thea carried coding-agent harness patterns into embodied systems, where a mistaken action cannot be reverted with git restore.
Taken separately, these look like documentation, releases, benchmarks, and a robotics prototype. Taken together, they describe a maturing field:
- The model is only one versioned dependency. System prompts, project files, skills, tool descriptions, hooks, frameworks, sandboxes, and evaluators all change behavior.
- Final-answer quality is not enough. A system can finish the task while disobeying a workflow, formatting, safety, or tool-use rule.
- Reliability work is boundary work. Permission bypasses, credential exposure, stale memory, partial messages, plugin shadowing, and recovery failures are harness defects.
- The harness has an economic design. Cache reuse, token accounting, retries, fan-out, and framework overhead decide cost per accepted outcome.
- Side effects make contracts more important than advice. A skill may suggest how to act; the tool boundary must determine what can actually happen.
This report is a dated evidence ledger, not a leaderboard. It distinguishes shipped implementation, public code, experimental results, and vendor positioning. Where the evidence is preliminary, the text says so.
The 10-day ledger
| Date | Evidence | What changed | Why harness engineers should care |
|---|---|---|---|
| August 3 | Microsoft documentation | The current Copilot Studio harness overview was updated, distinguishing standard, GitHub Copilot, and Microsoft 365 Copilot Chat harnesses | Harness selection now changes authoring, tools, billing, and publishing—not merely an internal framework |
| August 3 | Thea preprint and public preview | A coding-agent-style harness was applied to long-horizon robot tasks using a persistent scene graph, one action per turn, deterministic hooks, and post-action evaluation | The harness pattern survives outside software, but the cost of a false success becomes physical |
| August 3–7 | Claude Code 2.1.221–2.1.225 | Sandbox, worktree, hook, command-approval, credential-masking, plugin, and cross-session controls were hardened | The frontier is increasingly a stream of runtime boundary repairs rather than isolated features |
| August 7 / 10 | A2E paper and code | A protocol and auditing engine compared 9 agent harnesses across 23 benchmarks; the full codebase followed on August 10 | Teams can separate benchmark adapters, harness execution, telemetry, and lifecycle-aligned evaluation |
| August 8–13 | Claude Code 2.1.226–2.1.231 | Memory cleanup, cloud-skill isolation, distributed inboxes, workflow fan-out, self-hosted hooks, network parsing, retries, and OAuth handling changed | Session durability, extension trust, and failure recovery are now release-engineering concerns |
| August 12 | Harness-IF preprint | 12 models were tested on rules placed across six instruction surfaces in 2,160 agent-item runs | Instruction compliance is a harness property, and conflicting rules reveal failures hidden by ordinary tasks |
The dated product claims above are traceable to Microsoft’s harness overview, Anthropic’s Claude Code changelog, and the public npm Claude Code package history. The research claims come from Harness-IF, A2E, and Thea.
Shift 1: harness choice became a product decision
Microsoft’s documentation describes three Copilot Studio harnesses:
- the standard harness for structured, topic- and rule-driven experiences;
- the GitHub Copilot harness for reasoning-heavy, multi-step work with tools, connectors, knowledge, MCP, connected agents, file editing, skills, memory, and a secure sandbox;
- the Microsoft 365 Copilot Chat harness for extending Copilot Chat with organizational knowledge and capabilities.
Microsoft positions the GitHub Copilot harness as its most capable option. That is a vendor claim, not a comparative evaluation. The more durable observation is that selecting a harness affects the authoring model, available capabilities, billing mechanism, and supported publishing channels.
This is a category change. A harness is no longer hidden application plumbing. It is closer to a workload profile or runtime class. A product manager can choose one, and that choice determines both experience and economics.
The practical implication is to record harness selection in the same artifact as the product intent:
| Decision | Required evidence |
|---|---|
| Why this harness? | Task topology, required tools, side-effect class, latency target |
| What does it authorize? | Identities, data scopes, tools, approval modes, sandbox policy |
| What does it cost? | Metering unit, expected turns, tool costs, cache behavior, retry budget |
| Where can it run? | Channels, tenancy, data region, publishing constraints |
| How can it change? | Migration path, configuration ownership, rollback plan, regression suite |
Teams that choose the “most capable” harness by default risk buying unnecessary autonomy and a larger attack surface. Teams that choose the most deterministic harness by default may turn a genuinely exploratory task into brittle topic logic. The choice should follow the work.
Shift 2: evaluation moved from answers to instructions and trajectories
Most agent benchmarks ask whether the final state or final answer is correct. Harness-IF asks a different question: did the agent follow the rule, and did the surface carrying that rule matter?
The study built a library of 642 rules and selected 60 realistic multi-turn coding items. It placed 302 rules across five configurable instruction surfaces—system prompt, tool description, skill description, project file, and user instruction—while also evaluating fixed harness defaults. Of those placements, 256 were scored. Twelve frontier models were run for three rounds, producing 2,160 agent-item runs and 40,104 rule-level verdict rows.
The headline accuracy range was 72.1% to 85.9%. More important, every model did worse when a rule opposed a likely prior behavior. Against-prior accuracy ranged from 66.1% to 78.6%, with a mean gap of 5.81 percentage points from overall accuracy. The paper’s top point estimate belonged to Claude Opus 4.7, but the authors warn that common-support intervals do not resolve adjacent ranks. Treating the table as a precise model leaderboard would discard the paper’s most useful lesson.
The failure distribution is more actionable:
- 77.1% of observed failures were shortfalls—the agent omitted a required behavior—rather than oversteps.
- Output-control rules had the lowest family accuracy at 70.9%.
- Output-control and workflow rules together produced 53.9% of all failures.
- A pilot conflict experiment found a pooled tendency for system, project-file, and user instructions to outrank tool and skill descriptions, but only six of nine individual harness builds reproduced the exact ordering.
The correct conclusion is not “always put rules in the system prompt.” It is: instruction precedence is empirical, build-specific, and testable. Project files, skill metadata, and tool descriptions are not equivalent channels.
A2E attacks the adjacent problem. Its Agent Task Protocol separates task input, harness binding, runner, and trace. The open-source implementation instruments spans using OpenTelemetry and OpenInference, then evaluates reasoning, actions, final answers, and runtime quality. The paper reports 9 harnesses × 23 benchmarks × 5 sampled tasks, or 1,035 scored runs under a shared model configuration.
Single-turn tasks frequently failed to separate frameworks. Multi-turn tasks did. On the instrumented subset, correctness ranged from 0.568 to 0.663 while token cost varied by roughly 3.5×. No harness dominated every benchmark. That is exactly what a useful harness evaluation should reveal: architecture changes the trade space even when the backbone model stays fixed.
But A2E is not a procurement table. Five tasks per benchmark–harness cell produce coarse 0.20 score increments and high variance. Four sandbox-dependent benchmarks can change the overall order. One harness lacked token counts, and several instrumentation-dependent metrics should not be mistaken for direct observations of behavior. The paper explicitly rejects a universal framework ranking.
Together, Harness-IF and A2E produce a stronger evaluation unit:
The release candidate is not “model X.” It is the tuple (model, harness build, instruction manifest, tool catalog, policy bundle, evaluator set).
Shift 3: operational hardening became the frontier
Claude Code’s August release sequence is useful precisely because it is not one spectacular feature. It is a map of failure modes discovered in a fast-moving agent runtime.
Across versions 2.1.221 through 2.1.231, Anthropic’s changelog reports changes including:
- masking selected credentials in the sandbox;
- isolating destructive Git operations in worktrees;
- preventing pre-tool hooks from bypassing restrictions;
- repairing approval bypasses involving crafted commands, tabs, invisible Unicode, trailing slashes, and dangerous Git/GitHub flags;
- ensuring organizational policy can disable permission bypass for subagents;
- pinning archive plugins by SHA and separating synced cloud skills from local commands and MCP servers;
- adding cross-session messaging, agent listing, inbox delivery, and message expiry;
- stopping cleanup from deleting project memory;
- preserving clearer failure states for compaction, partial messages, retries, and stream interruptions;
- staggering dynamic workflow fan-out to improve prompt-cache reuse;
- making ambiguous sandbox network addresses fail closed;
- fixing MCP OAuth redirect handling.
This list should not be read as proof that the runtime is now invulnerable. A changelog is a record of shipped fixes, not an independent security audit. It does reveal what production harness engineering actually contains: path normalization, parser ambiguity, extension precedence, stale state, credential boundaries, distributed messaging, partial delivery, cleanup semantics, and resource accounting.
It also makes version pinning unavoidable. At the time of this research, npm’s latest line had advanced through releases that the stable tag had not yet absorbed. Teams adopting every latest build inherit rapid fixes and rapid behavioral change. Teams pinning an older stable build inherit a growing delta. Neither policy is safe without a canary suite and an explicit promotion rule.
Use a release evidence packet:
harness_release:
runtime: claude-code
candidate_version: 2.1.231
previous_version: 2.1.223
instruction_manifest: sha256:...
plugin_lock: sha256:...
policy_bundle: sha256:...
evaluations:
task_outcomes: artifacts/outcomes.json
instruction_compliance: artifacts/rules.json
permission_adversarial: artifacts/permissions.json
recovery_drills: artifacts/recovery.json
rollback:
owner: agent-platform
max_minutes: 15The version number alone does not describe the system that ran.
Shift 4: coding-agent patterns crossed into the physical world
Thea describes itself as a coding-agent harness for embodied agents. That translation is more than a metaphor.
The system keeps a persistent symbolic scene graph, gives the model a compact refreshed brief, and lets it query deeper state by reference. It allows one tool call per turn because a physical action changes the world and makes a long speculative action list stale. Deterministic pre- and post-hooks wrap tool execution. A post-action evaluator classifies the task as succeeded, failed, or still in progress and returns evidence for the next turn.
The design makes a crucial distinction: skills are advice; tools are enforced contracts. A skill can tell the model how to approach a task. It cannot be the safety boundary. Actions with physical consequences must cross a validated tool interface governed outside the model.
The paper reports that Thea had the highest task success among tested systems across three task-complexity levels on an Astribot S1, and that its evaluator averaged 93.3% across 90 checkpoints from three robots. The caveat matters more than the headline: some failed and in-progress states were classified as success. In physical systems, a false success can leave an unsafe or incomplete world state.
The public Thea repository labels the project a preview and says it is not a certified robot-safety system. Emergency stops, collision avoidance, actuator limits, and other low-level controls remain independent. That is the right architectural boundary. A language-model evaluator may guide task progress; it should not replace a hard real-time interlock.
Thea’s lesson applies back to software. A 200 OK is not proof that the requested business outcome occurred. Every consequential action needs postcondition evidence strong enough for the risk of a false success.
What is genuinely new—and what is not yet proven
| Claim | Evidence class | Confidence | Important limitation |
|---|---|---|---|
| Harness selection is exposed as a first-class Copilot Studio decision | Shipped product documentation | High | Microsoft’s capability language is vendor positioning; suitability is workload-specific |
| Claude Code hardened multiple runtime boundaries during the window | Dated vendor changelog and package history | High that changes shipped | The changelog is not an independent audit and does not establish residual risk |
| Instruction surface and prior conflict affect coding-agent compliance | Controlled preprint experiment | Medium | Only 60 items and 256 scored rules; English/coding centered; most verdicts use an LLM judge |
| Harnesses differ in multi-turn outcomes and token use under a shared model | Preprint plus public code | Medium | Five tasks per cell, benchmark sensitivity, incomplete instrumentation |
| Coding-harness patterns can improve embodied task execution | Prototype, public code, and physical evaluation | Medium-low for generalization | Three robots, limited trials, perception/evaluator errors, not a safety certification |
| One universal harness is best | No supporting evidence | Low | Both product segmentation and benchmarks point toward workload-specific trade-offs |
Harness-IF itself deserves additional caution. Only 13.3% of its rule verdicts were deterministic; 86.8% involved a three-vote GPT-5.2 judge or a hybrid evaluator. A reported judge-swap check had 62.1% agreement and low kappa on a small paired subset. The study is a valuable measurement design and planned public benchmark—not a certification instrument.
A 90-day roadmap for harness teams
Days 0–30: make the runtime enumerable
- Define the release tuple: model, harness, instructions, tools, policy, memory schema, and evaluators.
- Build an instruction registry with rule owner, surface, priority, applicability, and deterministic or judged evaluator.
- Inventory every extension source: local plugins, cloud skills, MCP servers, hooks, project files, and environment defaults.
- Record allowed identities, credentials, filesystems, networks, and side-effect classes.
- Add outcome, rule-compliance, runtime-quality, and cost views to the same trace.
Exit condition: any production run can be traced to immutable versions of every behavior-shaping artifact.
Days 31–60: test conflict, failure, and recovery
- Add against-prior rules and deliberate cross-surface conflicts to the regression set.
- Exercise permission bypass shapes: whitespace, Unicode, path normalization, flags, shell composition, symlinks, and hook rewriting.
- Interrupt streams, compaction, tool responses, remote workers, and session messaging.
- Verify that cleanup preserves durable memory and removes only scoped temporary state.
- Measure task outcome and compliance separately; log evaluator disagreement.
Exit condition: a release blocks on critical rule failure, unsafe ambiguity, unrecoverable partial state, or missing evidence—not only on an average benchmark score.
Days 61–90: optimize within a governed envelope
- Compare harness candidates on cost per accepted outcome, not raw token count.
- Optimize cache-aware fan-out, retries, context loading, and tool payloads without weakening policy.
- Canary framework or runtime upgrades against the exact workload distribution.
- Add postcondition evaluators for consequential tools and a human route for uncertain results.
- Permit automated harness improvement only through proposal, offline evaluation, canary, and rollback gates.
Exit condition: cost and capability can improve while instruction compliance, permissions, recovery, and evidence remain within declared thresholds.
The research agenda after August 13
The next useful work is not another broad “best agent framework” table. The field needs:
- Longitudinal harness evals. Run the same instruction and trajectory suite across runtime versions to quantify regressions and fixes.
- Surface-interaction experiments. Move the same rule between system, project, skill, tool, and user surfaces while holding the rest of the build fixed.
- Judge calibration for process claims. Publish human-reviewed disagreement sets and separate objective traces from model-judged interpretations.
- Failure-recovery benchmarks. Score duplicate side effects, partial tool results, lost messages, expired workers, stale memory, and compaction recovery.
- Supply-chain benchmarks. Test skill shadowing, malicious descriptions, hook injection, marketplace precedence, revocation, and provenance.
- Economic frontiers. Report accepted outcomes, compliance, latency, and cost together instead of optimizing one number.
- Embodied postconditions. Measure false-success risk and require independent safety systems when actions can harm people or equipment.
The center of gravity has moved. Model quality still matters, but the hard engineering question is now: what system turns probabilistic reasoning into bounded, observable, recoverable work?
In these ten days, the answer became more concrete. It is a versioned instruction hierarchy, an explicit runtime class, a policy and tool boundary, a durable trace, a set of evaluators, a recovery protocol, and a release process. That is harness engineering.