Knowledge Substrate

Reference Architecture

The Knowledge Substrate is the reference component profile that backs the Knowledge Graph spec. A deployment adopting this profile serves typed evidence bundles to the Compiler under hop budgets, snapshot pinning, and tenant scoping. Direct evidence sources remain valid when they provide equivalent provenance and replay controls.

This repository consumes supplied evidence and validates context admission; it does not implement this storage/retrieval service. Retrieval adapters enforce query/hop limits and actual source access before the pure compiler runs. Confidence and a self-declared verified label cannot authenticate evidence, identities or data rights.

Definition

A read-side service for the LPG (typed labeled-property graph). Owns vector / BM25 / graph indices kept in sync against a single snapshot_version; serves GraphRAG retrieval; enforces tenant isolation at the storage layer.

Why it exists

Naive RAG returns opaque chunks. Decisions need typed evidence the Critic can validate and the audit can reproduce. This component is the substrate that produces evidence bundles (hops[], evidence_refs, conflicts, snapshot pin) instead.

Inputs

• Ontology + Identity Layer references from the active Context Pack
• Retrieval parameters from the pack (mode, seed_strategy, max_hops, top_k, freshness_window, evidence_required)
• Run Context (tenant_id, role, classification scope)

Outputs

• EvidenceBundle:
  • snapshot_version, ontology_version
  • seed, hops[] with evidence_ref per hop
  • conflicts[] (subject + predicate + values)
  • truncated flag (always explicit; never silent)
• Aggregate confidence and per-hop confidence

How it works

1. Resolves snapshot_pin_rule → fixed snapshot_version for replay.
2. Seeds expansion from the intent + extracted entities.
3. Walks permitted relationships under max_hops and top_k constraints.
4. Filters by tenant_id, role, and data_classification from the Run Context.
5. Surfaces conflicting facts as EvidenceConflictMarker values with stable evidence refs and affected requirements—does not silently resolve.
6. Returns the typed bundle with truncated set explicitly if the budget capped expansion.

Apply data-flow policy to the actual purpose, verified source labels, recipient/processor and residency at collection, inference, tools, telemetry, cache, memory, export and delegation. Graph-edge filtering alone does not authorize a vector service, reviewer model or exported bundle. Namespace/readers and correction/deletion propagation cover derived indexes/caches as applicable; approved transformations need exact bound proof rather than a redaction flag.

Failure modes

• Edge missing evidence_ref — write rejected at the boundary.
• Cross-tenant traversal — denied at the storage layer; emitted as a security event.
• Snapshot outside the workflow freshness window — reject current-decision use or explicitly pin a newly admitted snapshot and reevaluate dependent evidence/approvals. Do not silently advance a historical replay pin because live data changed.
• Hop budget truncates a relevant hop — mitigated by truncated: true flag the planner must handle.
• Vector index drift vs. structured edges — alerting on disagreement; reindex on drift.

Operational concerns

• Per-environment snapshot pinning; promotion deliberate.
• Reindex cost when ontology version advances.
• Eviction of edges past valid_to requires audit trail.
• Backfill jobs replay against ontology versions and an explicit effective time, not ambient wall-clock time.

Evaluation metrics

• Evidence coverage (fraction of decisions whose evidence_refs resolve).
• Retrieval precision/recall on golden sets per intent.
• Conflict rate by (entity_type, relationship_type).
• Unauthorized cross-tenant successful access rate (target: zero); separately track denied attempts and coverage of denial tests.
• Deterministic retrieval artifact agreement under complete pinned query/index/embedding inputs; record model-assisted query artifacts and compare declared properties where applicable.