Adaptive Agent Harnesses: Learn From Experience Without Letting Production Rewrite Itself

The most interesting agent paper of July 2026 is not about a new model. It is about the software around one.

MemoHarness, submitted July 14, treats the agent harness as an adaptive control layer. Instead of deploying one prompt, one tool policy, and one orchestration pattern for every task, it learns from prior executions and specializes the harness to each new case.

That is the right optimization target. An agent’s behavior is shaped by far more than model weights:

• which evidence enters context;
• which tools are available and how they are called;
• how many model calls are orchestrated;
• what state persists across calls;
• how generation is configured;
• how the final output is parsed and validated.

MemoHarness reports gains across shell tasks, code generation, and financial reasoning while holding the base model fixed in its main comparisons. The result strengthens a claim ContextOS has made for some time: the harness is part of the model’s effective behavior.

But there is a crucial production distinction the paper does not attempt to solve:

A harness may adapt how work is done. It must not be able to adapt what the system is authorized to do.

The best production design is therefore not a self-rewriting agent. It is a two-loop system: a learning loop that proposes and selects performance changes, wrapped inside a deterministic safety envelope that learning cannot weaken.

This is a research review and an operating position. The featured work is a recent preprint, not deployment proof. Its results should be read under its published evaluation protocol and limitations.

What MemoHarness actually changes

MemoHarness decomposes a harness into six editable dimensions:

This decomposition matters. “Prompt optimization” treats the application as one string. Harness optimization recognizes a coupled software system: changing retrieval can alter the useful context format; changing orchestration can alter memory requirements; changing output validation can expose failures that were previously scored as success.

The paper adds two layers of experience:

1. Per-case entries record the harness configuration, configuration delta, execution trace, reward, token cost, and a diagnosis assigning failures to one or more dimensions.
2. Global patterns distill recurring failure clusters and the harness changes expected to address them.

Training-time search uses labeled cases to select a global harness. Test-time adaptation then retrieves related successes, failures, and global patterns to produce a case-specific harness without test labels or an additional search loop.

That separation is one of the paper’s strongest choices. It avoids using the current test answer as feedback while still allowing easy and difficult cases to receive different amounts of machinery.

The reported gains—and the correct level of confidence

Under the paper’s protocol, the final selected harness improved over its base harness on all three primary task families:

On Terminal-Bench, MemoHarness also exceeded the strongest released baseline in the comparison by 0.084. Cross-model transfer was positive on average, while cross-dataset transfer was explicitly selective rather than universal.

That last result is more valuable than a claim of universal transfer. A good research paper should reveal where a mechanism stops generalizing. A harness learned from long-horizon shell work can contain reusable execution patterns, but a harness learned near a code-generation ceiling may have little to teach a different domain.

The limitations are material:

• the primary Terminal-Bench held-out split contains 18 tasks;
• the study does not fully ablate the experience bank, distilled patterns, and case-specific adaptation in every setting;
• some baseline comparisons are closest-available product configurations, not pure scaffold swaps around an identical generator;
• the favorable cost result depends on most retrieved experience being cacheable;
• broader statistical robustness and finer component attribution remain open questions.

So the defensible conclusion is not “adaptive harnessing is solved.” It is:

Execution experience is a credible substrate for improving a harness, and the harness has enough leverage to move end-to-end outcomes without changing model weights.

That is already a consequential result.

The production gap: not every harness dimension is equally safe to edit

The paper’s six dimensions are useful for optimization, but production governance should not expose them as six unrestricted text fields.

Changing context formatting is not equivalent to changing tool access. Increasing a token limit is not equivalent to making memory persistent. Adding a formatter is not equivalent to changing whether an external action requires approval.

Classify every candidate change by authority:

This is the central operating rule:

adaptive harness
  = approved change surface
  + bounded candidate values
  + versioned experience
  + evaluation and rollout
 
immutable safety envelope
  = identity and tenant boundary
  + maximum capability set
  + policy deny rules
  + approval-mode floor
  + data-classification rules
  + budget ceilings
  + revocation and kill switch

The optimizer may choose a smaller surfaced-tool subset. It may not admit a new tool.

It may raise an action from automatic to approval-required. It may not lower a destructive action to automatic.

It may retrieve less sensitive evidence. It may not relax tenant, role, consent, or classification filters.

It may reduce a budget. It may not exceed the signed maximum.

Learning operates inside the feasible set. Deterministic policy defines the feasible set.

Experience must remain evidence, not authority

MemoHarness’s experience bank stores prior traces and diagnoses. In a production agent, that bank becomes both valuable and dangerous.

An old run can be:

• wrong but scored as successful;
• produced under a retired policy;
• based on stale tools or evidence;
• contaminated by prompt injection;
• specific to one tenant or data classification;
• valid for one model and harmful for another;
• a rare success whose strategy fails on the surrounding distribution.

Do not retrieve raw “successful experiences” into a future harness as if success made them trustworthy. Store an experience envelope with at least:

The experience bank should be append-only at the evidence layer. Corrections create a new version or tombstone; they do not erase the run that taught the wrong lesson.

This is the same discipline required for promotion-aware memory. Capture broadly. Promote narrowly. Retrieve only what the current run is eligible to see.

Use two loops, not online self-mutation

The safest architecture separates global learning from per-run selection.

Loop 1: offline candidate learning

1. Replay a versioned dataset against the current harness.
2. Cluster failures by surface: context, tool, generation, orchestration, memory, output.
3. Generate a typed candidate diff.
4. Reject any diff outside the approved change surface.
5. Evaluate utility, safety, cost, latency, and stability.
6. Require reviewer approval for behavior, capability, or policy-adjacent changes.
7. Release the candidate behind a feature flag.

Loop 2: online bounded adaptation

1. Classify the case using fields available before execution.
2. Retrieve approved patterns eligible for this tenant, intent, model, and policy version.
3. Select from released harness variants or bounded parameter ranges.
4. Compile the chosen configuration into the run manifest.
5. Record why that variant was selected.
6. Execute without modifying the released candidate set.

Online adaptation is selection, not invention.

If a production trace suggests a new strategy, it becomes a proposal for the offline loop. It does not rewrite the harness serving the next user.

Evaluate the candidate as a system

MemoHarness uses correctness-first selection with token usage as a tiebreaker. That is reasonable for a research search objective. It is too narrow for production promotion.

Use a release scorecard:

Report slice results, not only a global mean. A candidate that improves common read-only tasks while regressing rare destructive tasks is not a Pareto improvement. The low-volume destructive slice is the one that controls release.

For stochastic stages, run repeated trials and retain the distribution. For stateful stages, start from pinned state and inspect the state diff. For external actions, replay recorded tool results and never repeat the side effect.

The best-practice contract

A production adaptive harness should satisfy ten rules:

1. Type the editable surfaces. Do not search over an opaque system prompt.
2. Freeze the safety envelope. Learning cannot lower policy, identity, approval, or data boundaries.
3. Treat experiences as untrusted candidates. Success does not confer authority.
4. Pin provenance. Every lesson names the model, harness, policy, tools, environment, and dataset slice that produced it.
5. Separate offline learning from online selection. Production runs select released variants; they do not invent new ones.
6. Optimize a vector, not one reward. Utility gains cannot silently purchase safety or reliability regressions.
7. Replay before promotion. A candidate must explain which failures it fixes and which new failures it creates.
8. Roll out progressively. Shadow, canary, expand, and retain an immediate rollback path.
9. Expire learned patterns. Tool, policy, model, and environment changes can invalidate experience.
10. Keep final authority outside the model. The model proposes; deterministic boundaries decide.

MemoHarness is important because it demonstrates that a harness can learn from execution experience and adapt beyond a single fixed configuration. The production opportunity is real.

The production mistake would be to interpret that result as permission for an agent to rewrite its own control plane.

Let the harness learn how to work better. Make it prove every change. Never let learning redefine what “allowed” means.

Research base

• MemoHarness: Agent Harnesses That Learn from Experience, submitted July 14, 2026.
• Meta-Harness: End-to-End Optimization of Model Harnesses, for treating the surrounding harness as a search target.
• HARBOR: Exploring Agentic Harness Optimization, for constrained search over harness flags.
• Agentic Harness Engineering, for trace-grounded harness evolution.
• ContextOS: Autotune the Harness, Harness Candidates Are Model Checkpoints, and Harness Improvement Loops Need Replayable Environments.

What to read next

• Harness Candidates Are Model Checkpoints
• Autotune the Harness
• Harness Improvement Loops Need Replayable Environments
• The Agent Harness Audit