Memory Fabric

The Memory Fabric is an implementing service for the memory primitive in the Intelligence plane. It owns the capture log, candidate extractor, review queue, promotion pipeline, tier storage and data controls. This repository supplies deterministic lifecycle validation, not a production memory backend.

Definition

A storage and policy layer that materializes the promotion pipeline (capture → candidate → review queue → promoted → compiled context). Reads expose only promoted memory; writes are typed by class, evidence-bound, contradiction-checked, and consent-gated.

Why it exists

The Memory model defines the contract. The Fabric is what stores it, indexes it, governs it, and exposes it to the Compiler. Without a concrete fabric, every team’s memory diverges and the runtime cannot guarantee that “remembered” facts are governed.

How it works

1. Capture — authorized, redacted observations and correlation receipts enter the event log under explicit retention; raw captures are never queried by the Compiler directly.
2. Extract — a consolidator transforms captures into typed candidate facts (entity, predicate, value, evidence_refs, confidence, class).
3. Queue — candidates above a confidence floor enter the review queue; auto-approval policies may admit specific low-risk classes.
4. Promote — approved candidates are tier-routed (working / episodic / semantic / durable) with a contradiction check.
5. Recall — the Fabric resolves eligible promoted memory under current tenant, owner/namespace, readers, classification, purpose and intent policy; the reference Compiler consumes the supplied records rather than querying a store.

Storage tiers

Backing stores and retention ranges below are illustrative deployment choices, not universal defaults or a requirement to deploy every tier. Actual retention ceilings and all derived-copy obligations come from current lifecycle policy.

All tiers carry tenant, classification, and provenance on every record.

Review queue

The review queue is a first-class store with operator UI hooks:

• proposal_id (stable), candidate, class, tier_target, evidence_refs, confidence.
• consent_check and contradiction_check results computed at queue time.
• auto_promote_eligible flag based on the pack’s promotion_thresholds.
• Operator actions: approve, edit, reject. Rejections are remembered with reason so the same noise is filtered next time.

Consolidation pipeline

capture log
   ↓
candidate extractor (typed transforms per source)
   ↓
review queue
   ↓
contradiction checker  ←→ existing promoted records (same entity + predicate)
   ↓
tier router (per memory_layer.memory_policy)
   ↓
write-side redactor (PII, classification)
   ↓
tier storage + index

Read-side context assembly

When the Compiler recalls memory, the Fabric:

• filters by tenant_id, role, and data_classification from the Run Context,
• applies freshness windows and confidence thresholds from the pack,
• ranks by relevance to the active intent,
• returns each record with its provenance, originating capture event, and confidence.

Expose excluded/low-confidence recall diagnostics through a declared service protocol so the runtime can ask for clarification when appropriate. low_confidence is not a canonical promoted-memory freshness value; do not inject excluded records into the admitted manifest. The reference compiler separately reports its integrity, freshness and context-budget omissions.

Write-side governance

Writes are typed by class:

Each class has its own retention policy, classification rules, and consent gating. correction_writer is an illustrative deployment role whose current entitlement the service verifies. Model-emitted consent_check, confidence and auto_promote_eligible fields cannot authenticate consent or grant write authority. Correction writes emit a governed learning signal to the Insight Synthesizer.

Contradiction handling

When a promotion targets the same (entity, predicate) as an existing record:

• if fresh, higher-integrity evidence wins under an explicit deterministic source-resolution policy, the prior record is superseded with audit;
• otherwise, both records persist with contradiction: true markers and the planner is required to handle the conflict on the next recall.

Redaction

Apply required minimization before capture and candidate extraction, with current data-flow policy checked again at recall, inference, tool use and export. Candidate-time masking alone cannot authorize a later recipient or prevent source data leaking into telemetry/cache. Pack redaction rule names and classified attributes guide the implementing enforcer; the reference compiler does not execute redaction. Where a transformation permits a flow, retain the approved transformation’s exact source/result/action/recipient binding and verified proof.

Governed lifecycle implementation

Bind every memory operation to an authenticated (tenant, owner, namespace, purpose) and its exact action/operation. Keep personal and employer namespaces, credentials, readers and indexes separately scoped. The data-flow policy applies to capture, inference, telemetry, cache, memory and export; a permitted tool origin does not authorize storing its complete response. Broker references remain opaque and token material stays outside captures, memory and logs.

The reference validateMemoryLifecycle(record, context, evaluationTime) validates an additive lifecycle attachment against current namespace access, the pinned data policy, permitted readers/operations, retention ceiling and required review. Production obtains those normalized observations from trusted brokers and stores. An approver or verifier name in a record cannot establish its own authenticity. The validator does not perform a database write, OS confinement, broker verification or deletion.

Use a complete derived-copy inventory for correction and deletion, covering tier stores, search/vector indexes, caches and controlled exports. Each propagation receipt names its artifact, lifecycle record, stable operation and intent digest. Keep request authorization separate from outcome: an allowed deletion may remain pending or fail. Only current verified receipts for every target and a verified complete inventory can establish completion_verified for the scoped stores. Unknown external copies remain external_unknown; do not report them erased.

Review hashes the domain-separated complete intent, excluding the review itself and later propagation entries/status. This avoids a self-referential hash while keeping target, provenance, readers, policy, expiry and correlation bound. A changed intent requires new review when policy requires it. Status changes require independently verified propagation receipts, rather than silently refreshing the old review digest.

Expired retention blocks new promotion/correction, while authorized cleanup can proceed unless an explicit retention exception requires escalation. Keep a minimal authorized audit lineage without retaining forbidden payloads indefinitely. External backups, processors, exported copies and model-training effects need explicit deployment policy and separately evidenced limits; this reference does not erase them.

Implementation mapping

• Memory model — the spec narrative.
• Knowledge Graph — the substrate that promoted memory writes against.
• Identity Layer — CEIDs that key memory records.
• Governance — write classes, consent rules, redaction.

Interfaces

Inputs

• Capture stream from every plane
• Memory write policy and consent records
• Run Context (tenant_id, role, classification scope)
• Operator queue actions

Outputs

• Promoted records consumable by the Compiler
• Review queue items with consent + contradiction checks
• Write proposals for the next turn
• Contradiction events for operator review

Failure modes

• Capture log unbounded (mitigated by tier-by-tier compaction).
• Candidate extractor produces low-quality candidates that swamp the queue (mitigated by per-source confidence calibration).
• Auto-promotion policy too permissive — drift; mitigated by sample audit of auto-promoted records.
• Recall returns records past TTL because freshness window not honored — mitigated by index TTLs.
• Cross-tenant leakage through a shared semantic record (mitigated by tenant scoping at storage).

Operational concerns

• Storage and index cost budgets per tier.
• Operator throughput on the review queue; SLA per class.
• Contradiction backlog monitoring.
• Per-tier TTL alerts when records approach deletion.
• Backfill and compaction jobs run against pinned ontology versions and an explicit effective time, never ambient wall-clock time.

Evaluation metrics

• Recall precision/recall against golden memory queries.
• PII write-back violation rate (target: zero).
• Review queue throughput (items/operator/day).
• Auto-promotion accuracy (sample-audited).
• Correction adoption rate (operator corrections that change downstream decisions).

Example

A typed memory write proposal for review:

{
  "proposal_id": "mwp_2026_05_04_0931_a17",
  "candidate": {
    "entity_ceid": "customer:cus_77",
    "predicate": "prefers_communication_channel",
    "value": "email",
    "evidence_refs": ["tool:crm.lookup:tc_212", "session:sess_42f1"],
    "confidence": 0.93
  },
  "class": "preference",
  "tier_target": "semantic",
  "consent_check": "passed",
  "contradiction_check": { "existing": null, "verdict": "no_conflict" },
  "auto_promote_eligible": true
}

Common misconceptions

• The Fabric is not the Knowledge Graph. Memory is what survived promotion; the graph is the broader substrate.
• Capture is not memory. Capture is the audit log; memory is the governed surface.
• Recall is not retrieval. Retrieval queries the graph; recall queries promoted memory only.
• Auto-promotion is not automatic. It is an explicit pack policy for specific low-risk classes.