Agentic Context Engineering
The Context-plane discipline of compiling governed, bounded, observable context for each agent run.
Context as a proof-carrying materialized view: declared in packs, resolved under policy, checked for sufficiency, budgeted, traced, and improved from replay.
- RunContext with tenant, actor, intent, safety mode, budget, and trace
- Versioned Context Pack declarations
- Policy, tool, evidence, memory, and session state
- Pressure signals from prior evaluated runs
- CompiledContext
- Policy, tool, and admitted-evidence manifests
- Provenance-bearing context blocks and structured omissions
- Context-admission controls, evidence gates, and budget report
- Diagnostics for the Improvement Loop
- ContextPack
- CompiledContext
- ContextProvenance
- EvidenceConflictMarker
- EvidenceGate
- BudgetReport
- ContextLedger
Agentic Context Engineering is the discipline of deciding what the model is allowed to see for a specific run, then making that decision reproducible.
It is not prompt writing. A prompt is one output of the Context plane. The real artifact is the CompiledContext: a proof-carrying materialized view containing the bounded prompt plus the manifests, controls, conflicts, omissions, source versions, and budget report that explain how it was produced.
Definition
Agentic Context Engineering turns a versioned Context Pack, a RunContext, and runtime evidence into a CompiledContext envelope.
The envelope must answer four questions:
| Question | Required artifact |
|---|---|
| What source material was eligible? | pack version, registry refs, policy refs, memory refs, evidence candidates |
| What did the compiler include? | compiled prompt, provenance-bearing blocks, tool manifest, admitted-evidence manifest |
| What did it exclude, supersede, or compress? | structured omissions, rejected evidence, resolved and unresolved conflicts, truncation reasons, budget report |
| Which controls were active? | context admission, sufficiency-aware evidence gates, approval gates, redaction rules, loop guards |
If those artifacts are missing, the system may still be using good prompts, but it is not doing ContextOS-grade context engineering.
The stronger model: a materialized epistemic view
The compiler does not merely select relevant tokens. It materializes the run’s current epistemic state: which claims are supported, which requirements remain open, which sources disagree, which evidence was superseded by an explicit resolution, and which material was omitted. “Epistemic” here means evidence and uncertainty state; it does not mean hidden chain-of-thought.
This framing adds three obligations to relevance:
- Sufficiency: the admitted context contains enough current, verified evidence to close every required decision obligation.
- Consistency: an unresolved conflict on required evidence keeps the obligation open even when both conflicting items are individually current and verified.
- Lineage: the compiled view preserves the source and resolution references needed to invalidate, replay, or challenge it later.
The view is materialized because the same pinned inputs deterministically produce the same envelope. It is proof-carrying because commit_allowed is derived from explicit gate evidence rather than from the model’s confidence. It is still bounded and lossy: omissions and unresolved state must therefore remain part of the artifact, not disappear during packing.
Why It Exists
Enterprise agents fail when context assembly is informal:
- Policy is copied into prompts and quietly goes stale.
- Tool availability is implied rather than bound to identity and approval mode.
- Retrieval injects evidence without provenance, freshness, access scope, or conflict state.
- Memory writes return as future context before consent or promotion.
- Token pressure silently drops the facts needed for an auditable decision.
The Context plane makes context assembly explicit. It lets the Decision plane reason over a known input envelope and lets the Trust plane replay the run later.
Compiler Contract
The Context Pack Compiler follows the same eight-stage shape used in How It Works:
| Stage | Purpose | Output |
|---|---|---|
| Intent classification | Resolve the requested intent and risk class. | intent binding, confidence, unresolved entities |
| Policy resolution | Evaluate relevant policy bundles against the run. | policy manifest, active gates, prohibitions |
| Tool surfacing | Intersect registry, permissions, policy, and safety mode. | tool manifest |
| Evidence normalization | Label retrieved candidates with source, integrity, freshness, hash, covered requirements, and retrieval-layer conflict markers. | typed evidence candidates and conflicts |
| Memory recall | Reconstruct promoted memory against current source state, evidence, policy, authority, freshness, and contradictions. | typed admitted, omitted, and uncertain memory candidates |
| Token budget allocation | Allocate per-bucket limits from the run budget. | budget plan |
| Bucket admission and assembly | Assign instruction treatment, reject ineligible evidence, rank, redact, and pack blocks. | compiled prompt, admitted-evidence manifest, structured omissions |
| Runtime controls | Evaluate context sufficiency and emit controls for the Decision and Action planes. | context admission, conflict-aware evidence gates, approval gates, refusal/escalation rules |
The compiler may use model assistance for classification, summarization, or compression, but the final envelope must still be typed, versioned, and traceable.
Context Buckets
ContextOS uses buckets so teams can reason about pressure and governance consistently.
| Bucket | Typical content | Common failure |
|---|---|---|
business | goals, tone, operating rules, domain constraints | generic behavior that ignores business reality |
policy | active rules, gates, refusals, escalation conditions | model acts from stale or informal policy |
tool | eligible capabilities, schemas, approval modes | planner calls tools it cannot execute |
evidence | retrieved records, documents, graph facts, receipts | unsupported claims or wrong joins |
memory | promoted episodic, semantic, and durable memory | contaminated or unapproved recall |
session | recent conversation and run state | loss of continuity or repeated work |
Working state that only belongs to the current turn should remain session state. Durable recall belongs in memory only after the Memory Model promotion rules allow it.
Context integrity model
Every material ContextBlock carries ContextProvenance. This is an execution boundary, not decorative metadata:
| Field | Contract |
|---|---|
source_refs and content_hash | identify the exact source material and detect caller/source mismatches |
origin | distinguishes pack, policy, tool registry, retrieval, memory, and runtime material |
integrity | says whether the block may justify a governed decision |
freshness | says whether the evidence is inside its validity window |
instruction_treatment | says whether the model may treat the block as control text or only as data |
inclusion_reason | records why the compiler admitted the block |
Integrity and confidentiality answer different questions. Data classification governs who may see a block; integrity governs whether the block may justify an action.
| Integrity | Typical source | Decision use |
|---|---|---|
authoritative | pinned pack, evaluated policy, surfaced tool registry | may carry control instructions |
verified | validated retrieval or promoted memory with lineage | may support facts, but remains data_only |
untrusted | unknown, legacy, or externally supplied text without verification | may be shown as data_only; never closes an evidence gate |
Retrieved evidence is eligible to satisfy a DecisionSpec only when it is admitted, verified, current, and not blocked by an unresolved conflict. stale, unknown, untrusted, budget-omitted, conflict-blocked, and resolution-superseded candidates remain visible in the gate diagnostics but cannot make commit_allowed true. Legacy string-only evidence therefore fails closed until a caller supplies the structured fields.
An EvidenceConflictMarker names the conflicting evidence refs and affected requirements. An unresolved marker makes the gate insufficient; the model cannot settle it by choosing whichever passage sounds more plausible. A resolved marker must carry a resolution_ref and the selected evidence refs, leaving the rejected branch visible for audit and replay.
Evaluated prompt contract
The prompt material inside a CompiledContext is released as an evaluated contract, not edited as free-form prose. A production prompt contract declares:
- objective — the task and terminal success condition;
- input trust labels — which blocks are control, verified data, or untrusted data;
- authority — what the model may decide versus only propose or escalate;
- decision procedure — required checks, ordering, and evidence gates;
- tool contract — surfaced capabilities, argument rules, effect boundaries, and required receipts;
- output schema — the response or decision envelope that must validate; and
- failure and stopping behavior — what to do when evidence, authority, budget, or progress is insufficient.
The corresponding prompt release record pins the model route, prompt and skill refs, tool/schema versions, stable prefix and cache topology, evaluation datasets and graders, release thresholds, and rollback target. This record contributes to the harness release manifest; it does not replace the canonical CompiledContext or DecisionRecord.
Prompt structure is also a latency control. Stable authoritative prefixes should be separated from volatile evidence and session blocks so caches remain valid without freezing stale state. Routing, prefix-cache hit rate, time-to-first-token, compilation latency, and required-evidence retention are evaluated together; a cheaper prompt that drops a gate-closing fact is a regression.
CompiledContext Shape
The implementation guide owns the full schema. This abridged example shows the integrity-critical fields:
{
"compiled_prompt": {
"context_blocks": [
{
"block_id": "ev_order",
"bucket": "evidence",
"content": "{\"order_id\":\"ord_881\",\"status\":\"delivered\"}",
"priority": 60,
"provenance": {
"source_refs": ["kg:order:ord_881#snapshot_2026_07_31"],
"origin": "retrieval",
"integrity": "verified",
"freshness": "current",
"instruction_treatment": "data_only",
"inclusion_reason": "retrieved evidence admitted for support.refund",
"content_hash": "sha256:..."
}
}
]
},
"manifests": {
"policy_manifest": [
{ "bundle_id": "POLICY_RETURNS_V4", "rule_ids": ["R_HIGH_VALUE_REQUIRES_APPROVAL"] }
],
"tool_manifest": [
{ "adapter_id": "adp_orders", "capabilities": ["lookup"] }
],
"evidence_manifest": [
{
"evidence_ref": "kg:order:ord_881#snapshot_2026_07_31",
"source_id": "orders-db",
"hash": "sha256:...",
"integrity": "verified",
"freshness": "current",
"instruction_treatment": "data_only",
"satisfies": ["order_lookup"]
}
]
},
"runtime_controls": {
"approval_gates_active": ["GATE_FINANCE_APPROVAL"],
"must_refuse": [],
"must_escalate": ["fraud_signal_high"],
"redaction_rules_active": [],
"context_admission": {
"data_only_block_ids": ["ev_order"],
"untrusted_block_ids": [],
"privileged_sink_policy": "verified_or_user_intent"
},
"evidence_gates": [
{
"decision_key": "support.refund.execute",
"required": ["identity_verified", "order_lookup", "policy.eval"],
"satisfied": ["identity_verified", "order_lookup", "policy.eval"],
"unresolved": [],
"rejected_evidence": [],
"conflicts": [],
"sufficiency": "sufficient",
"commit_allowed": true
}
]
},
"budget_report": {
"tokens_used_at_compile": 620,
"omissions": [],
"warnings": []
},
"context_ledger": {
"pack_ref": "ctxpack.support@1.0.0",
"evidence_refs": ["kg:order:ord_881#snapshot_2026_07_31"],
"omissions": [],
"compiled_context_hash": "sha256:..."
}
}Quality Signals
Context quality is not a vibe. The compiler should emit signals that can be scored later:
| Signal | Meaning |
|---|---|
| Relevance | The block matched the current intent and entities. |
| Integrity | The source is authoritative, verified, or untrusted for this decision type. |
| Freshness | The block is inside its validity window or snapshot pin. |
| Density | The block carries useful information per token. |
| Coverage | Required evidence is admitted, current, verified, and mapped to the requirement. |
| Sufficiency | Coverage is complete and no required obligation has an unresolved conflict. |
| Conflict closure | Every detected disagreement is either blocking or linked to an explicit resolution. |
| Pressure | Budget pressure did not remove required material. |
These signals feed Evaluation and Observability and the Improvement Loop. They should be stored as diagnostics, not hidden in logs.
Autotune surfaces
The Context plane is the safest first place to apply autotune because many changes are bounded, replayable, and reversible. A Context Pack may declare tunable surfaces, but the compiler must refuse proposals outside that declaration.
| Surface | Candidate examples | Guardrail |
|---|---|---|
| Retrieval | top_k, max_hops, source priority, freshness window | Evidence-gate closure and integrity cannot regress. |
| Bucket budgets | evidence_tokens, memory_tokens, tool_tokens, compression threshold | Required blocks cannot be omitted while a gate appears satisfied. |
| Prompt fragments | Small instruction-block changes with token and forbidden-term limits | Policy, redaction, and tool manifests remain outside model discretion. |
| Memory recall | promoted-memory class filters, recency window, contradiction handling | Unpromoted or consent-missing memory cannot enter CompiledContext. |
| Runtime controls | loop guards, escalation hints, refusal messages | Approval gates and must-refuse rules cannot be weakened by a cost target. |
Every candidate should produce a TuningProposal that names the target intent, target metric, baseline pack, replay sets, expected scorecard delta, and rollback target. The Context plane does not promote the candidate; the Trust plane gates it through replay, review, and staged rollout.
Boundary With Adjacent Pages
| Page | Owns |
|---|---|
| Cognitive Core | where the compiler sits in the runtime loop |
| Context Pack | the concrete pack schema and lifecycle |
| API Contracts | invocation and runtime envelope examples |
| Memory Model | what can be recalled or written back |
| Governance | policy and approval-mode taxonomy |
Agentic Context Engineering is the discipline. The Context Pack Compiler is the component. The Context Pack is the source artifact. CompiledContext is the runtime artifact.
Failure Modes
| Failure | Runtime response |
|---|---|
| Required evidence is missing | return a typed missing_evidence verdict before planning destructive action |
| Evidence is untrusted, stale, or has unknown freshness | retain a rejection reason; do not use it to close an evidence gate |
| Current verified sources conflict on required evidence | keep both in the compiled view, mark the gate insufficient, and require an explicit resolution or escalation |
| Caller-provided hash disagrees with the content | reject the compile visibly instead of accepting ambiguous provenance |
| Legacy string evidence has no integrity metadata | label it untrusted with unknown freshness and emit a migration warning |
| Policy bundle fails to evaluate | fail closed for enforced policy; do not treat as “not fired” |
| Tool manifest conflicts with safety mode | exclude the tool and record the exclusion |
| Memory candidate has no promotion or consent record | omit from compiled context |
| Token budget drops a required evidence block | record a structured omission, reject that candidate for the gate, and block or escalate |
| Source snapshot cannot be pinned | mark replay as incomplete and block high-risk decisions |
Operational Metrics
- Context compile latency by stage.
- Required evidence coverage.
- Evidence-gate closure rate by decision key.
- Untrusted-evidence admission rate (target: zero for satisfying evidence).
- Stale and unknown-freshness rejection rate.
- Required-evidence conflict rate, resolution rate, and time to resolution.
- False-closure rate: decisions marked sufficient despite a known unresolved conflict (target: zero).
- Legacy evidence migration warnings and content-hash mismatch failures.
- Omission rate by bucket and reason.
- Tool manifest eligibility mismatch rate.
- Policy manifest evaluation error rate.
- Replay match rate for
CompiledContextreconstruction. - Prompt-contract schema pass rate and stable-prefix cache hit rate.
- Recall reconstruction correctness across source, memory, evidence, policy, and authority changes.
- Budget pressure by intent and pack version.
- Decision quality delta after context-pack changes.
Example
For support.refund, the compiler should include:
- the active refund policy bundle,
- the eligible read and refund tools,
- order and customer evidence refs,
- the
support.refund.executedecision binding, GATE_FINANCE_APPROVALwhen the amount exceeds INR 3000,- a budget report showing whether any evidence or session state was compressed.
The Planner can then propose work against a known envelope. The Critic can reject a plan if required context is missing before the refund tool is ever called.
Common Misconceptions
- “Better prompts solve context.” Better prompts help, but governance comes from typed inputs, manifests, and replay.
- “Retrieval is the Context plane.” Retrieval is one input. The Context plane also resolves policy, tools, memory, budgets, and controls.
- “Two verified sources mean the context is stronger.” If they disagree on a required claim, the context is insufficient until the conflict is explicitly resolved.
- “Summaries are always safe compression.” Summaries must carry source refs, transformation metadata, and quality signals.
- “The model can decide which policy matters.” Policy selection and enforcement happen outside model discretion.