Harness Engineering
55 essays tagged with Harness Engineering.
AI Agents as Insider Risks: What Nadella's Trust Architecture Requires
A research-backed analysis of Satya Nadella's AI insider-risk argument: external authorization, reasoning transparency, agent-to-agent provenance, independent evidence, and the limits of emergency stops.
NVIDIA OpenShell and Sentry: Containment Needs an Independent Boundary
A research assessment of NVIDIA's September 28 Open Agent Safety Platform: software enforcement, an out-of-band watchdog, assurance boundaries, and falsifiable containment tests.
Qwen-Planner-Agent: What Model–Harness Co-Evolution Can Actually Prove
A critical assessment of Alibaba's September 24 Qwen-Planner-Agent preprint: hybrid environments, development-set feedback, attribution, and a stronger transfer experiment.
After Meta Muse: A Research and Publishing Roadmap for Personal-Agent Harnesses
A falsifiable research roadmap for personal-agent harnesses: authorization binding, ambiguous-effect recovery, credential misuse, governed memory, counterparty permission, interfaces, deployment, and reasoning architecture.
Harness Engineering on September 26, 2026: From Agent Loops to Personal-Agent Operating Systems
A current practitioner field report on harness engineering through September 26, 2026: Meta Muse, JAZ’s minimalist harness, event-sourced coding agents, agentic-commerce access disputes, embodied agents, and the next research questions.
Muse Spark, Muse Code, and Glimmer: Meta’s Full Agent Stack
A practitioner map of Meta’s 2026 agent stack: Muse Spark 1.3, Muse Code’s event-sourced multi-agent harness, Glimmer’s local open weights, Model API, voice, avatars, connectors, and AI-glasses distribution.
Meta Muse Under the Hood: A Harness Engineering and Security Review
A practitioner’s review of Meta Muse’s personal-agent architecture: Secure VM isolation, Sentinel authorization, credential surrogation, connectors, browser controls, memory, prompt-injection defenses, and the tests still missing.
Harness Engineering: The September 2026 Practitioner’s Field Guide
A primary-source review through September 15, 2026: controlled harness comparisons, evolving safety defenses, context inheritance, managed runtimes, implementation availability, and the next experiments.
Managed Agent Harnesses: The Control Boundaries You Still Own
A research-backed operating guide to managed agent harnesses: durable sessions, caller credentials, authorization, interrupted effects, version drift, migration tests, and application-owned evidence.
Subagent Context Inheritance and Independent Review
A practitioner’s design and evaluation guide for forked and isolated subagents: context selection, reviewer independence, cache economics, information boundaries, and a reproducible experiment plan.
FrontierHarness Eval: Auditing the 17.5× Cost Gap
A reproducible audit of FrontierHarness Eval: cost per success versus median cost, paired task outcomes, small-sample uncertainty, and what to measure before choosing a production harness.
A Research and Publishing Roadmap for Harness Engineering
A prioritized roadmap for original harness engineering research and technical publishing, with experiments, baselines, artifacts, decision gates, and falsifiable directions through 2027.
Harness Engineering in September 2026: Measurement, Evolution, and Operational Reality
A critical research map of harness engineering through September 10, 2026: new benchmarks, automatic optimization, reusable tools, long-running agents, public implementations, and unresolved questions.
Self-Improving Harnesses: HarnessDev, HoH, and the Promotion Boundary
A practitioner analysis of HarnessDev, Harness-of-Harness, HarnessOpt-Bench, and HarnessCompass, with concrete boundaries for evaluation, durable state, recovery, and harness promotion.
Agent Harness Benchmark: An Experimental Protocol Beyond Model Scores
A proposed benchmark protocol for isolating harness effects, injecting operational faults, measuring repeated reliability, and publishing auditable scorecards.
Proof-Carrying Agent Runtime: A Bounded Architecture and Specification
A research-grounded runtime proposal with explicit proof obligations, an effect-boundary state machine, a conditional safety argument, and failure tests.
Proof-Carrying Context: Why AI Agents Need More Than a Context Window
A research-grounded revision of context engineering: from relevant tokens to a replayable view of evidence, conflicts, omissions, and decision sufficiency.
The Harness Engineering Roadmap for 2026–27: From Agent Loops to Proof-Carrying Runtimes
A source-grounded 18-month roadmap for harness engineering across release manifests, portable trajectories, lifecycle security, skill evaluation, delegated authority, recovery, economics, and governed adaptation.
Skill Lift: How to Gate Agent Skills With Paired Live Evals
An implementation guide to evaluating agent skills as executable release artifacts using paired baselines, Skill Lift, ATIF trajectories, security checks, domain graders, and CI gates.
HarnessRisk in Practice: Turn the Agent Lifecycle Into a Security Test Matrix
A deep implementation guide to lifecycle security testing for agent harnesses: configuration, extensions, runtime, persistence, actions, recovery, evidence oracles, and release gates.
Harness Engineering in August 2026: The Control Plane Gets Measured
A source-backed state of the field on harness engineering: what shipped, what the newest research measured, what practitioners are debating, and the roadmap from agent loops to proof-carrying runtimes.
DeepSeek Harness: Everything Is a Plugin—Except Trust
A source-code critique of DeepSeek Harness and Cordis: why an everything-is-a-plugin runtime matters, where reversibility stops, and why self-evolution still needs an immutable control plane.
Agentic Recommender Systems: How Harness Engineering Changes Personalization
Recommenders are evolving from rankers into tool-using agents. Here is the hybrid architecture, evaluation harness, and rollout plan needed to build them safely.
The Production Harness Roadmap: Lessons from Claude Code, Copilot Studio, and Thea
A concrete 90-day roadmap for building a governed agent harness, derived from the latest Claude Code, Copilot Studio, Thea, Harness-IF, and A2E evidence.
Your Agent Followed the Rule—or Did It? Harness-IF and A2E Change Agent Evaluation
A deep implementation guide to rule-level, surface-aware, trajectory-based agent evaluation using the new Harness-IF and A2E research.
Harness Engineering: The Runtime Became the Product
A source-backed 10-day field report on Harness-IF, A2E, Thea, Microsoft Copilot Studio, and Claude Code—and what their combined evidence changes for agent engineering.
Agent Evaluation After the Leaderboard: The KDD 2026 Production Assurance Playbook
KDD 2026 shows why agent evaluation must cover protocols, trajectories, external state, evaluator operating profiles, causal rollout, and drift.
Agent Memory Should Be Compiled at Recall, Not Replayed From Storage
What MemHarness teaches us about reconstruction, negative transfer, source state, and compiling production agent memory against current evidence, policy, and authority.
Passive Awareness Is the Missing Multi-Agent Primitive—But It Needs a Control Plane
AgentRadio shows that agents improve when discoveries can cross lane boundaries during execution. Production teams should keep the asynchronous channel, then add typed evidence, bounded delivery, authority isolation, and replay.
What Production Agent Runtimes Actually Teach ContextOS: Twelve Laws of a Governed Harness
The strongest production agent runtimes converge on twelve architectural laws: compile context, persist state, separate authority from containment, make side effects resumable, treat approvals as typed interrupts, and promote learning only through evidence and replay.
Adaptive Agent Harnesses: Learn From Experience Without Letting Production Rewrite Itself
MemoHarness shows that execution experience can improve the control layer around an LLM. Production systems need a stricter pattern: adaptive performance inside an immutable safety envelope.
Agent Memory Is a Systems Workload: What SelfMem Changes—and What It Does Not
SelfMem improves long-horizon recall by letting an agent optimize its memory strategy. New systems and security research shows the production contract must also cover cost, freshness, provenance, and poisoning.
Your Agent Benchmark Is Measuring the Harness
Two new evaluation systems separate benchmark, harness, model, and environment. Their results show why one leaderboard score cannot tell you which agent to ship.
Agent Harness: An Architectural Framework for Production AI Agents
A whitepaper on typed contracts, policy gates, traces, verification loops, and release control for production AI agents.
Harness Improvement Loops Need Replayable Environments
Why harness improvement needs replayable episodes, bounded mutations, scorecards, source closure, and promotion gates.
Product Managers: How to Think About and Build Complex Agentic Systems
A practical PM guide to building agentic systems with workflow maps, intents, context packs, tools, records, evals, and rollout gates.
How to Develop an Agent with an Agent Harness, End to End
An end-to-end field guide for building agents as measurable harnesses: context, planning, tools, records, evals, rollout, and learning.
Autotune the Harness: Baking the Improvement Loop into ContextOS
How ContextOS treats autotune as a gated loop over traces, scorecards, replay sets, bounded candidates, approval, and rollout.
How Great AI Engineers Build Agents: Datasets, Scores, and Harnesses That Improve
Why strong AI engineers build datasets, scorecards, traces, and improvement loops instead of treating agents as prompts plus tools.
Harness Candidates Are Model Checkpoints: How to Improve Agents Without Silent Mutation
How to treat every prompt, retrieval, tool, policy, and evaluator change as a scored, reviewed, reversible harness candidate.
AGENTS.md Done Right: The Navigation File That Actually Helps Coding Agents
How to write AGENTS.md as a short, scoped, testable navigation file for coding agents instead of a bloated prompt dump.
The Agent Harness Audit: A Production Readiness Checklist for Governed AI Agents
The original 44-control agent harness checklist, with a note on the current release-specific assurance method and evidence requirements.
Build an AI Agent Replay Test with the Context Pack Compiler
Run four TypeScript tests for repeatable context compilation, tool permissions, approval thresholds, and missing evidence. Includes code and replay limitations.
End-to-End Refund: How 12 Primitives Compose in One Production Run
A single refund run traced through 12 ContextOS primitives, from invokeAgent envelope to byte-equal replay.
Failure Playbooks: The Typed Verdict Map
How to replace generic retry loops with typed failure verdicts, compensations, escalation paths, and reversal-token checks.
Approval Gates in Code: The Destructive-Mode Handshake
A build-along for approval gates: frozen evidence, human signatures, gateway redemption, and replayable destructive-action handshakes.
Build the Tool Gateway: The Boundary That Actually Stops a Bad Action
A build-along for the Tool Gateway: adapter manifests, typed envelopes, resolver checks, dispatch, and destructive-action boundaries.
The Critic: verify, score, consolidate — in 80 Lines
A compact Critic implementation that verifies plans, scores outcomes, consolidates results, and records caveats.
Promotion-Aware Memory: Capture, Review, Promote, Recall in Code
A build-along for agent memory: capture, review, promote, recall, contradiction checks, and governed memory writes.
Build the Context Pack Compiler: Eight Stages, Eight Files
A build-along for the Context Pack compiler: eight deterministic stages that turn runtime inputs into a typed compiled context.
From Operator Correction to Released StrategyRule: The Improvement Loop, Coded
How one operator correction becomes a reviewed, replayed, versioned StrategyRule that prevents repeat agent failures.
Pack Rollout in Five Stages: Shipping a Context Pack Without Blowing Up Production
A five-stage rollout model for Context Packs: shadow, internal, low-risk, monitored expansion, full release, and rollback.
Wiring the Five Evaluators: Policy, Utility, Latency, Safety, Cost
A build-along for wiring policy, utility, latency, safety, and cost evaluators into a release-gated scorecard.
Building a Reliability Reviewer Agent: 70 Lines Past the Compliance One
How to extend the reviewer pattern for reliability: timeouts, retries, idempotency, fallback behavior, and rollback declarations.
Building a Compliance Reviewer Agent in 60 Lines and a Golden Set
How to build a compliance reviewer agent with a typed verdict envelope, rubric, golden set, and change-control queue.