The next useful harness engineering articles should help a reader decide what to build, remove, measure, or refuse to deploy. A list of new framework names cannot do that on its own. A paper summary is more useful when it identifies a testable claim and the experiment needed to apply it.
This roadmap pairs a practitioner agenda with a publishing agenda. It is a proposal dated September 10, 2026, not a vendor commitment, a scheduled automation, or an announcement that the experiments below have already been run. It complements the broader 2026–27 field roadmap with concrete editorial deliverables and acceptance gates.
The first four articles establish the research baseline. Subsequent articles should earn stronger claims through released evidence. If an experiment is inconclusive, publish the limitation and the method; do not fill the gap with a performance narrative.
The initial series
| Article | Reader decision | Contribution |
|---|---|---|
| September research update | Which developments deserve attention? | Evidence map distinguishing research, implementation reports, and public artifacts |
| FrontierHarness cost audit | What do the benchmark numbers actually mean? | Executed calculations, source pin, missing-cost accounting, and exploratory paired analysis |
| Self-improving harnesses | How should optimization and long-running work be governed? | Critical comparison and an explicitly proposed operating architecture |
| This roadmap | What should be researched and published next? | Priorities, experimental controls, deliverables, and stop conditions |
Only the benchmark data audit contains new executed numerical analysis in this series. Research-paper results are attributed. Architectural examples and future experiments are proposals. This distinction should remain visible as the series grows.
Select topics by the decision they change
Prioritize a topic when failure is costly, an experiment can isolate the cause, and the result would change an engineering decision. Deprioritize topics that merely restate the existing harness contract or rename an established practice.
For each article, write the decision before the headline. “Should we keep this recovery middleware after a model upgrade?” is a sharper research question than “the future of autonomous agents.” It implies a baseline, an intervention, and a measurable consequence.
Use three publication classes. A research review synthesizes attributable evidence. An implementation study supplies code and explains its tested boundary. An experimental report supplies outcomes from a declared protocol. A post may contain more than one class, but each claim should make its evidence level clear.
Priority 1: model–harness compatibility
Working title: What Survives a Model Upgrade? A Harness Ablation Study.
Compare a baseline harness and a candidate across at least two intended executors. Change one component at a time: completion checks, compaction, tool descriptions, or retrieval. Keep task identities, environment, authority, and scoring fixed. Include repeated trials and a protected final partition.
The motivating evidence is executor-dependent behavior in HarnessDev and the distinction between update quality and update use in Harness Updating Is Not Harness Benefit. Those studies motivate the question; they do not predict the answer for this workload.
Publish candidate diffs, pinned configurations, per-task outcomes, and development cost. The article passes its evidence gate when it can state which component helped which executor, what regressed, and what remains uncertain. If the supposed improvement vanishes under a second executor, that is the result.
Priority 2: the economics of recovery
Working title: The Cheapest Successful Agent Can Still Be the Most Expensive Workflow.
Choose a bounded workflow with observable external state, such as a simulated support refund. Inject a timeout before the effect, a timeout after the effect, a duplicate delivery, and a stale precondition. Pair each fault with a clean control using the same initial state.
Measure accepted outcomes, prohibited or duplicate effects, unresolved states, elapsed time, and operator minutes. Include failed work in the bill and retain missing-cost coverage. The FrontierHarness audit provides a concrete example of why denominators and missing values matter; it does not supply the business measurements for this experiment.
Publish the fault trigger, initial and final state, operation identity, and recovery evidence. The article passes only when a reader can distinguish a successful retry from a repeated external effect. A simulated payment backend must be labeled as simulation, with its differences from a real service stated explicitly.
Priority 3: skills that help, distract, or interfere
Working title: When Two Useful Agent Skills Make a Worse System.
Start with paired runs with a skill present and withheld. Then evaluate two skills individually and together. A combined bundle can alter selection, consume context, or introduce conflicting instructions even if each skill helps in isolation.
ACES supplies a research basis for paired live skill evaluation, and NVIDIA SkillEvaluator is a public implementation to inspect. Adoption should begin with the exact version and supported environment, rather than assuming its defaults constitute a production release policy.
Measure task outcome, skill activation, instruction compliance, cost, and security. Keep the no-skill baseline. Test unrelated tasks to detect interference. The publication gate requires an interaction analysis and failed cases, not merely an attractive average lift.
Priority 4: language interfaces over typed tools
Working title: Who Verifies the Arguments Inside an Agent-Native Tool?
Use the tool abstraction proposed by HEART as a research lead. Compare explicit typed calls with a natural-language wrapper on a small catalog where the intended entity and effect are independently known.
Vary ambiguity, catalog size, stale identifiers, and incompatible output shapes. Include a task where the right behavior is clarification. Keep authorization outside both interfaces so the experiment measures usability and argument formation without removing the safety boundary.
Publish normalized requests and verifier outcomes, with sensitive data replaced by fixtures. The article should answer whether reduced caller complexity moves hidden work into the wrapper and whether that work is reliably checked. Tool-discovery accuracy alone is insufficient.
Priority 5: multi-day progress and regression
Working title: Can a Coding Agent Resume Without Losing What Already Works?
Compare an outer planning-and-evaluation loop with ordinary continuation under both matched passes and a separately matched resource budget. Reuse several project families. Interrupt at session boundaries and during tool execution. Assess previously accepted requirements after every iteration.
HoH motivates the workflow comparison. Its public repository’s implementation status must be checked before selecting a reproduction target; a local implementation inspired by the paper should be labeled as such. Do not call an independently constructed approximation a reproduction of unreleased code.
Publish artifact revisions, requirement histories, evaluation evidence, cumulative cost, and human intervention. The gate is sustained verified progress across repeated projects, with regression and failure accounting. A single multi-day demo supports a case study, not an operational reliability estimate.
Priority 6: improve the harness under an immutable evaluator
Working title: Can an Agent Improve Its Runtime Without Improving Its Own Grade?
Allow a proposer to edit a bounded candidate directory. Separate evaluator code, hidden tests, release authority, and retained evidence. Give every search method the same target-evaluation budget and compare agent-guided editing with simple alternatives, including a static baseline and a predefined candidate search.
HarnessOpt-Bench motivates protected evaluation; HarnessCompass motivates component-level constraints. This experiment should measure whether the added optimization machinery earns its cost in the selected environment.
The article passes when multiple independent searches produce assessable results, all candidates are preserved, and the final choice is frozen before protected scoring. Include a failed attempt to modify the evaluator as a boundary test. A prompt-level prohibition is not equivalent to an enforced write boundary.
A twelve-week research sequence
These are proposed work windows beginning after this series, contingent on owners, infrastructure, and evaluation budget. They are not automatic publication dates.
| Window | Deliverable | Gate before stronger claims |
|---|---|---|
| Weeks 1–2 | Task inventory, acceptance criteria, and environment manifest | Independent review of labels, authority, and sampling scope |
| Weeks 3–4 | Compatibility and component-removal experiments | Paired results, repeated trials, and preserved regressions |
| Weeks 5–6 | Recovery economics and skill-interaction pilots | External-state checks and explicit cost coverage |
| Weeks 7–8 | Tool-interface comparison | Argument-level evidence and clarification cases |
| Weeks 9–10 | Long-running workflow study | Requirement history, interruption tests, cumulative cost |
| Weeks 11–12 | Bounded optimization comparison and synthesis | Frozen candidate selection and protected final evaluation |
Each study needs a technical owner, an evaluation reviewer, and someone accountable for the decision it informs. One person may fill multiple roles in a small team, but permissions and records should still separate candidate development from final scoring.
Estimate resource use from a small smoke suite before authorizing a full model run. Reserve budget for final evaluation instead of spending everything on candidate search. If prerequisites are unavailable, publish a design review with that limitation rather than relabeling it as an empirical result.
The longer horizon through 2027
The following directions are analytical forecasts. Confidence describes the usefulness of investigating them, not the probability of a particular product launch.
| Direction | Confidence | Evidence that would strengthen it | Evidence that would weaken it |
|---|---|---|---|
| More explicit model–harness release manifests | High | Teams reproduce regressions from pinned complete configurations | Maintenance overhead exceeds diagnostic benefit in measured deployments |
| Routine component removal after model upgrades | Medium | Repeated ablations preserve quality while reducing cost | Removed components repeatedly protect rare but important cases |
| Evaluation of interruption and recovery as release criteria | High for agents with external effects | Fault tests predict operational incidents | Benchmarks fail to represent real recovery failures |
| Automated search over harness code | Medium | Gains persist across tasks and exceed equally funded simple baselines | Search overfits or costs more than its deployment savings |
| Portable semantics for delegated authority and evidence | Medium | Interoperable implementations preserve end-to-end effect constraints | Shared formats mask materially different enforcement behavior |
No forecast requires rewriting the ContextOS specification ahead of working evidence. New proposals should identify the existing contract they relate to, the implementation that exercised them, and the migration implications before becoming normative.
The quality bar for every article
A strong technical post contains a concrete failure, an explicit claim, the artifact or source that supports it, and the conditions under which it might not hold. Put citations next to sourced claims and retain publication dates and versions. Use primary papers, repositories, and engineering reports as the evidentiary core.
For experiments, publish the protocol before interpreting results. Keep failed attempts, missing values, compute use, and manual interventions visible. Separate measured results from illustrative numbers. Avoid a universal “best” label when the experiment only identifies a candidate under one model and task distribution.
For implementation guides, ship runnable examples only after testing their stated behavior. For research reviews, do not imply personal operating experience that the evidence does not support. For forecasts, state what would change the conclusion.
Update an article when its central evidence changes: an implementation is released, a paper is revised materially, a benchmark corrects a field, or a reproduction changes the conclusion. Keep a dated correction note so readers can understand both the earlier claim and the new evidence.
Sources and status
This roadmap synthesizes the primary evidence cited inline. Key research foundations are HarnessDev (September 1), HarnessOpt-Bench (August 6), HarnessCompass (August 3), ACES (August 20), HEART (September 1), and HoH (September 1), all 2026 preprints. The complete citation map is available in the series research ledger.
The future experiments are not yet executed. The benchmark audit artifacts are the executed analysis delivered with the initial series. This roadmap schedules no unattended work and promises no vendor release date.