The next useful harness engineering articles should help a reader decide what to build, remove, measure, or refuse to deploy. A list of new framework names cannot do that on its own. A paper summary is more useful when it identifies a testable claim and the experiment needed to apply it.
This roadmap pairs a practitioner agenda with a publishing agenda. First published September 10, extended September 15 and revised for contract alignment October 16, 2026, it is a proposal, not a vendor commitment, a scheduled automation, or an announcement that the experiments below have already been run. It complements the broader 2026–27 field roadmap with concrete editorial deliverables and acceptance gates.
The first four articles establish the research baseline. Subsequent articles should earn stronger claims through released evidence. If an experiment is inconclusive, publish the limitation and the method; do not fill the gap with a performance narrative.
The initial series
| Article | Reader decision | Contribution |
|---|---|---|
| September research update | Which developments deserve attention? | Evidence map distinguishing research, implementation reports, and public artifacts |
| FrontierHarness cost audit | What do the benchmark numbers actually mean? | Executed calculations, source pin, missing-cost accounting, and exploratory paired analysis |
| Self-improving harnesses | How should optimization and long-running work be governed? | Critical comparison and an explicitly proposed operating architecture |
| This roadmap | What should be researched and published next? | Priorities, experimental controls, deliverables, and stop conditions |
Only the benchmark data audit contains new executed numerical analysis in this series. Research-paper results are attributed. Architectural examples and future experiments are proposals. This distinction should remain visible as the series grows.
September 15 extension and revised priorities
Three new research articles extend the baseline for practicing AI engineers and technical founders:
| Article | Decision it helps make | Evidence class |
|---|---|---|
| Updated practitioner field guide | Which developments change what we should build or measure? | Primary-source synthesis with explicit uncertainty |
| Context inheritance and independent review | When should a child inherit context, and how should a reviewer be informed? | Implementation review and proposed experiment |
| Managed harness control boundaries | Which responsibilities remain in the application? | Architecture analysis and proposed acceptance tests |
The new source ledger accompanies this extension. These articles do not add new agent performance measurements to the earlier numerical audit.
The next publication queue should favor focused studies over another broad roundup. The following order reflects editorial judgment about decision value and experimental feasibility, not measured topic popularity.
| Order | Proposed blog | Required artifact | Publication gate |
|---|---|---|---|
| 1 | Fork or Isolate? Measuring Context Reuse and Review Quality | Pinned packet builder, task manifest, per-run records | Include packet-construction cost, seeded defects, false findings, and uncertainty |
| 2 | Does the Harness Winner Change with the Workload? | Paired model–harness matrix and resource manifests | Prespecify workload groups; report incomplete runs and missing usage |
| 3 | What Happens When a Managed Agent Resumes After a Write? | Simulated provider, fault triggers, operation ledger | Distinguish recovered sessions from reconciled external effects |
| 4 | Can an Evolved Safety Harness Survive a New Domain? | Frozen candidate, unseen tasks, benign controls | Preserve critical invariants; report utility and residual attack outcomes |
| 5 | Which Harness Components Can We Delete After a Model Upgrade? | Ablation diffs and original failure fixtures | Report both recovered efficiency and reopened failures |
| 6 | Who Pays for Agent Recovery? | Full workflow cost ledger and human-time records | Include failures, provider reconciliation, and operator work |
The same-model comparison motivates a careful replication, while EvoSafeHarness motivates testing safety transfer. Neither supplies results for these proposed studies.
A concrete article acceptance package
Each experimental post should ship a question, frozen protocol, executable entry point, environment and version manifest, eligible raw outcomes with authorized transformations and retention, analysis code, and failed cases. A research review instead needs claim-level citations and an implementation-availability check. Do not market a literature synthesis as a reproduction.
Lead with the decision, explain the mechanism with one concrete failure, present evidence and alternatives, then show what to implement and what remains uncertain. Use a stable canonical URL for each distinct reader question. Update an existing review when new evidence refines its answer; create a new article when the experimental question changes.
Technical review should verify that code examples match their declared scope and that proposed fixture formats cannot be mistaken for normative ContextOS types. Editorial review should verify every number against its exact source and version. Release review should check rendered tables, mobile reading, internal links, metadata, sitemap, feed, and search inclusion.
What to watch through 2027
These are conditional research directions, not predicted product release dates:
- Near term: measure context delegation, accounting coverage, and managed-runtime interruption behavior. Publish reproducible methods even when outcomes are inconclusive.
- Next quarter: test component removal and safety transfer across intended model and workload changes. A transferable improvement needs protected evidence, not repeated tuning on the final set.
- 2027 agenda: study whether acceptance criteria and effect evidence remain portable across providers, and whether long-running work preserves correctness across repeated policy, tool, and model changes.
Advance an item only when an owner, evaluation environment, and bounded budget exist. If access to a hosted service or private task suite blocks a comparison, publish the boundary and complete the independent parts. This roadmap does not schedule unattended execution or promise future results.
Select topics by the decision they change
Prioritize a topic when failure is costly, an experiment can isolate the cause, and the result would change an engineering decision. Deprioritize topics that merely restate the existing harness contract or rename an established practice.
For each article, write the decision before the headline. “Should we keep this recovery middleware after a model upgrade?” is a sharper research question than “the future of autonomous agents.” It implies a baseline, an intervention, and a measurable consequence.
Use three publication classes. A research review synthesizes attributable evidence. An implementation study supplies code and explains its tested boundary. An experimental report supplies outcomes from a declared protocol. A post may contain more than one class, but each claim should make its evidence level clear.
Contract controls for every proposed study
These are proposed experiments, not missing canonical runtime modules or executed enterprise qualification. Use exact profiles and conformance for the selected workflow, implementation, release and environment. Source inspections and synthetic provider tests cannot establish actual deployed containment or provider guarantees. Required useful positive, denial and fault evidence must be current and independently verified. Mandatory control failures and policy-expression errors cannot become an optimization tradeoff.
Authorize collection before recording traces or publishing cleaned outcomes: actual purposes, readers, recipients, processors, tenant/residency, transformations and retention apply across collection, inference, tools, telemetry, cache, memory, export and delegation. Maintain a copy inventory with correction/deletion receipts; removing sensitive text from the final paper does not repair an earlier disclosure.
Give each researcher, proposer and reviewer a current owned finite scope. Child permissions intersect every ancestor; current exact full-action approval and active sandbox/credential controls precede effects. Protected stores and generation fencing govern interrupted workers, with original shared spent/reserved capacity across search, reviewers, children and final evaluation. Unknown provider outcomes retain reservations and stable operation identity until reconciliation; key-only retries, cancellation requests and future version rollback cannot prove reversal. The execution guide defines these obligations.
Priority 1: model–harness compatibility
Working title: What Survives a Model Upgrade? A Harness Ablation Study.
Compare a baseline harness and a candidate across at least two intended executors. Change one component at a time: completion checks, compaction, tool descriptions, or retrieval. Keep task identities, environment, authority, and scoring fixed. Include repeated trials and a protected final partition.
The motivating evidence is executor-dependent behavior in HarnessDev and the distinction between update quality and update use in Harness Updating Is Not Harness Benefit. Those studies motivate the question; they do not predict the answer for this workload.
Publish candidate diffs, pinned configurations, per-task outcomes, and development cost. The article passes its evidence gate when it can state which component helped which executor, what regressed, and what remains uncertain. If the supposed improvement vanishes under a second executor, that is the result.
Priority 2: the economics of recovery
Working title: The Cheapest Successful Agent Can Still Be the Most Expensive Workflow.
Choose a bounded workflow with observable external state, such as a simulated support refund. Inject a timeout before the effect, a timeout after the effect, a duplicate delivery, and a stale precondition. Pair each fault with a clean control using the same initial state.
Measure accepted outcomes, prohibited or duplicate effects, unresolved states, elapsed time, and operator minutes. Include failed work in the bill and retain missing-cost coverage. The FrontierHarness audit provides a concrete example of why denominators and missing values matter; it does not supply the business measurements for this experiment.
Publish the fault trigger, initial and final state, operation identity, and recovery evidence. The article passes only when a reader can distinguish a successful retry from a repeated external effect. A simulated payment backend must be labeled as simulation, with its differences from a real service stated explicitly.
Priority 3: skills that help, distract, or interfere
Working title: When Two Useful Agent Skills Make a Worse System.
Start with paired runs with a skill present and withheld. Then evaluate two skills individually and together. A combined bundle can alter selection, consume context, or introduce conflicting instructions even if each skill helps in isolation.
ACES supplies a research basis for paired live skill evaluation, and NVIDIA SkillEvaluator is a public implementation to inspect. Adoption should begin with the exact version and supported environment, rather than assuming its defaults constitute a production release policy.
Measure task outcome, skill activation, instruction compliance, cost, and security. Keep the no-skill baseline. Test unrelated tasks to detect interference. The publication gate requires an interaction analysis and failed cases, not merely an attractive average lift.
Priority 4: language interfaces over typed tools
Working title: Who Verifies the Arguments Inside an Agent-Native Tool?
Use the tool abstraction proposed by HEART as a research lead. Compare explicit typed calls with a natural-language wrapper on a small catalog where the intended entity and effect are independently known.
Vary ambiguity, catalog size, stale identifiers, and incompatible output shapes. Include a task where the right behavior is clarification. Keep authorization outside both interfaces so the experiment measures usability and argument formation without removing the safety boundary.
Publish normalized requests and verifier outcomes, with sensitive data replaced by fixtures. The article should answer whether reduced caller complexity moves hidden work into the wrapper and whether that work is reliably checked. Tool-discovery accuracy alone is insufficient.
Priority 5: multi-day progress and regression
Working title: Can a Coding Agent Resume Without Losing What Already Works?
Compare an outer planning-and-evaluation loop with ordinary continuation under both matched passes and a separately matched resource budget. Reuse several project families. Interrupt at session boundaries and during tool execution. Assess previously accepted requirements after every iteration.
HoH motivates the workflow comparison. Its public repository’s implementation status must be checked before selecting a reproduction target; a local implementation inspired by the paper should be labeled as such. Do not call an independently constructed approximation a reproduction of unreleased code.
Publish artifact revisions, requirement histories, evaluation evidence, cumulative cost, and human intervention. The gate is sustained verified progress across repeated projects, with regression and failure accounting. A single multi-day demo supports a case study, not an operational reliability estimate.
Priority 6: improve the harness under an immutable evaluator
Working title: Can an Agent Improve Its Runtime Without Improving Its Own Grade?
Allow a proposer to edit a bounded candidate directory. Separate evaluator code, hidden tests, release authority, and retained evidence. Give every search method the same target-evaluation budget and compare agent-guided editing with simple alternatives, including a static baseline and a predefined candidate search.
HarnessOpt-Bench motivates protected evaluation; HarnessCompass motivates component-level constraints. This experiment should measure whether the added optimization machinery earns its cost in the selected environment.
The article passes when multiple independent searches produce assessable results, eligible candidate metadata and failed attempts are retained under their retention policy, and the final choice is frozen before protected scoring. Include a failed attempt to modify the evaluator as a boundary test. A prompt-level prohibition is not equivalent to an enforced write boundary.
A twelve-week research sequence
These are proposed work windows beginning after this series, contingent on owners, infrastructure, and evaluation budget. They are not automatic publication dates.
| Window | Deliverable | Gate before stronger claims |
|---|---|---|
| Weeks 1–2 | Task inventory, acceptance criteria, and environment manifest | Independent review of labels, authority, and sampling scope |
| Weeks 3–4 | Compatibility and component-removal experiments | Paired results, repeated trials, and preserved regressions |
| Weeks 5–6 | Recovery economics and skill-interaction pilots | External-state checks and explicit cost coverage |
| Weeks 7–8 | Tool-interface comparison | Argument-level evidence and clarification cases |
| Weeks 9–10 | Long-running workflow study | Requirement history, interruption tests, cumulative cost |
| Weeks 11–12 | Bounded optimization comparison and synthesis | Frozen candidate selection and protected final evaluation |
Each study needs a technical owner, an evaluation reviewer, and someone accountable for the decision it informs. A small team may combine roles only where its selected profile permits it. Required separation of duties, independent verification and quorum cannot be satisfied by relabeling the same person. Permissions and records must separate candidate development from final scoring.
Estimate resource use from a small smoke suite before authorizing a full model run. Reserve budget for final evaluation instead of spending everything on candidate search. If prerequisites are unavailable, publish a design review with that limitation rather than relabeling it as an empirical result.
The longer horizon through 2027
The following directions are analytical forecasts. Confidence describes the usefulness of investigating them, not the probability of a particular product launch.
| Direction | Confidence | Evidence that would strengthen it | Evidence that would weaken it |
|---|---|---|---|
| More explicit model–harness release manifests | High | Teams reproduce regressions from pinned complete configurations | Maintenance overhead exceeds diagnostic benefit in measured deployments |
| Routine component removal after model upgrades | Medium | Repeated ablations preserve quality while reducing cost | Removed components repeatedly protect rare but important cases |
| Evaluation of interruption and recovery as release criteria | High for agents with external effects | Fault tests predict operational incidents | Benchmarks fail to represent real recovery failures |
| Automated search over harness code | Medium | Gains persist across tasks and exceed equally funded simple baselines | Search overfits or costs more than its deployment savings |
| Portable semantics for delegated authority and evidence | Medium | Interoperable implementations preserve end-to-end effect constraints | Shared formats mask materially different enforcement behavior |
No forecast requires rewriting the ContextOS specification ahead of working evidence. New proposals should identify the existing contract they relate to, the implementation that exercised them, and the migration implications before becoming normative.
The quality bar for every article
A strong technical post contains a concrete failure, an explicit claim, the artifact or source that supports it, and the conditions under which it might not hold. Put citations next to sourced claims and retain publication dates and versions. Use primary papers, repositories, and engineering reports as the evidentiary core.
For experiments, publish the protocol before interpreting results. Keep eligible failed-attempt metadata, missing values, compute use, and manual interventions visible; bound repeated held-out submissions and feedback after freezing the selected candidate. Separate measured results from illustrative numbers. Avoid a universal “best” label when the experiment only identifies a candidate under one model and task distribution.
For implementation guides, ship runnable examples only after testing their stated behavior. For research reviews, do not imply personal operating experience that the evidence does not support. For forecasts, state what would change the conclusion.
Update an article when its central evidence changes: an implementation is released, a paper is revised materially, a benchmark corrects a field, or a reproduction changes the conclusion. Keep a dated correction note so readers can understand both the earlier claim and the new evidence.
Sources and status
This roadmap synthesizes the primary evidence cited inline. Key research foundations are HarnessDev (September 1), HarnessOpt-Bench (August 6), HarnessCompass (August 3), ACES (August 20), HEART (September 1), and HoH (September 1), all 2026 preprints. The complete citation map is available in the series research ledger.
The future experiments are not yet executed. The benchmark audit artifacts are the executed analysis delivered with the initial series. This roadmap schedules no unattended work and promises no vendor release date.