Deployment Blueprint

Reference Architecture

This non-normative blueprint maps the five planes to one deployable topology with tenant isolation, data residency, mixed replay conformance, and SLO-driven reliability. Implementations may combine or distribute components differently while preserving the contracts.

Executive summary

The runtime decomposes into stateless Decision-plane services, regional AI Gateway / LLM Router pods, regional stateful Intelligence-plane stores, a single Action-plane Tool Gateway per region, and a globally-replicated Trust-plane control plane. Replay is preserved by content-addressed pack, model profile, route decision, and snapshot pinning; audit is preserved by append-only WORM-capable storage.

That topology is illustrative and unbenchmarked. Pin exact control profiles, actual workflow features and ownership before deployment; the repository supplies validators and synthetic references rather than these services. Versioned artifacts and WORM capability support evidence handling but do not alone prove correct execution, audit completeness or conformance. Define workflow-specific threats, forbidden effects, failure assumptions, escalation/stop owners, availability and recovery objectives, then collect current deployment evidence.

Deployment topology (reference)

Service placement by plane

Multi-region strategy

• Active/active for serving APIs (invokeAgent, retrieval, recall).
• Regional AI Gateway / LLM Router for model calls; fallback cannot cross residency, classification, or capability boundaries.
• Active/passive for batch processing and embedding pipelines.
• Cross-region replication for KG snapshots, audit, pack registry, Decision Record store.
• Region pinning for regulated tenants and data-residency compliance.

Active serving does not permit concurrent unfenced ownership of one run or effect. Atomic run/approval/operation/budget stores must enforce expected revisions and current generation across failover. Define the consistency, quorum, lease and split-brain behavior; a lagging replica cannot reset reservations or replay a possible committed write. Before resumed dispatch, verify checkpoint bytes/pins, current authority/approval/boundary/external preconditions/budget and provider target-fencing support. If a required control is unavailable, suspend/escalate rather than silently routing around it.

Tenancy partitions

• Per-tenant namespaces for KG and memory indices.
• Row / column security for source-of-truth stores.
• Tenant-scoped model profile allowlists, routing budgets, and provider data-retention policy at the AI Gateway.
• Scoped credentials and per-tenant rate budgets at the Tool Gateway.
• Hard multi-tenancy (isolated clusters) for regulated tenants.

Caching

• L1 compiled-context cache keyed by the complete declared compile inputs: exact pack, request, Run Context/limits, supplied evidence/conflicts, memory and relevant policy/registry pins, within tenant/owner/namespace boundaries. Pack/snapshot/request alone can reuse the wrong authority-sensitive context.
• L2 regional cache with tenant-keyed namespaces.
• Cache invalidation tied to: pack promotion, snapshot promotion, policy bundle promotion, model-profile promotion, and routing-policy promotion.
• Embedding cache keyed by (model_version, text_hash, tenant_id) when the pack permits.
• Prompt-response caching is disabled by default at the AI Gateway and enabled only when tenant policy, classification, and profile allow it.

Cache eligibility and TTL do not replace current dispatch authorization or lifecycle checks. Track derived copies, readers, actual processors and correction/deletion propagation; keep credentials out of cache keys and payloads. A cached response is not proof that an uncertain provider effect did not occur.

SLO wiring

• Per-tenant Run Budget enforcement at the Conversation Manager boundary.
• Model-call latency, token, and cost budgets enforced at the AI Gateway.
• Step timeouts enforced by the Tool Gateway, not by the adapter.
• SLO dashboards: p95 compile, p95 router decision, p95 gateway overhead, p95 retrieval, p95 recall, p99 end-to-end Run Context wall-clock.
• Error budgets banded by native risk dimensions and material effect, with v1 risk_class retained for historical slices.

Root, child, evaluator, retry/repair/fallback and replan work share original aggregate limits and deadline. Reserve before work, use exact declared cost units, settle verified cumulative usage and retain uncertain capacity/charges. A restart, new region or restored checkpoint cannot replenish that budget.

Replay conformance

• Pack registry stores immutable, content-identified versions; signatures are required only under an adopted supply-chain profile.
• Model profiles, routing rules, and candidate routing-lineage records are pinned. The documented RoutingDecision profile is not yet a canonical schema.
• KG snapshots are pinned per environment; promotions deliberate.
• Audit store retains tool transcripts, policy decisions, approval records.
• Replay substitutes recorded tool transcripts, compares deterministic artifacts exactly, and evaluates model-dependent verdicts with declared properties or scorecards without silently changing the model route.

Stateful stores

• KG primary: regional, sharded by tenant; cross-region replicas for read-only failover.
• Vector / BM25 indices: rebuilt from KG snapshot; do not lead the snapshot.
• Memory tier storage: regional primary with WAL replication; per-tenant TTL.
• Routing-lineage store: append-only; joins model profile, routing rule, provider adapter, and trace ID using the candidate profile until a canonical schema is promoted.
• Decision Record store: append-only; cross-region replicated; WORM-capable for audit class.
• Audit store: append-only, WORM-capable, replicated.
• Run/approval/operation/budget stores: durable atomic revisions, single current executor generation, exact approval redemption, stable operation identity and cumulative reservations/settlement; replication semantics must preserve those invariants.

Scaling model

• Stateless services: HPA by p95 latency or in-flight count.
• Stateful services: vertical + sharded by tenant or entity type.
• Queues: backpressure with dead-letter queues for tool execution and batch jobs.

Data residency and isolation

• Data classified at ingestion (PUBLIC / INTERNAL / CONFIDENTIAL / RESTRICTED).
• Residency constraints enforced at the Conversation Manager and storage layers.
• Cross-region replication excluded for RESTRICTED classes.

The last exclusion is this blueprint’s illustrative policy, not a universal regulatory rule. Actual source labels, purpose, recipients, processors and residency must be verified across collection, inference, tool I/O, telemetry, cache, memory, exports and delegation. Region labels or dedicated clusters alone do not demonstrate those controls. Require observed sandbox/process/filesystem/resource restrictions, mediated DNS/redirect/egress and brokered credentials appropriate to the workload. Separate personal/employer and tenant trust domains; govern backups and external copies explicitly.

Append-only/WORM audit storage must coexist with authorized retention, holds and deletion duties. Minimize payloads, protect opaque lineage and distinguish retained minimal receipts from prohibited personal data. Do not claim external backups, exports or model-training effects erased without separate evidence.

Runbooks (high level)

• Provider outage: AI Gateway routes to an eligible alternate profile; degrade quality tier only within residency, capability, and budget constraints; log scorecard delta.
• Latency spike: reduce context and graph hop budgets within policy, lower model route tier if allowed, and enable cache; do not silently drop required evidence.
• Cost spike: enforce Run Budget caps, select lower-cost model profiles when policy allows, and use cached read-only aliases where valid; do not weaken native risk or the conservative v1 projection.
• KG index corruption: promote replica; replay from audit logs.
• Audit store unavailability: fail closed for effects whose policy requires durable evidence; any degraded read path must be explicitly declared and must not rely only on a v1 tier.
• Lost provider acknowledgement: retain unknown with stable operation and reservations; reconcile using verified provider-specific facts before any permitted retry. Compensation requires a new governed operation and may be partial.
• Region loss or stop request: stop new dispatch, propagate child cancellation and fence stale workers; verify remaining effects and ownership before recovery. A re-pin or cancellation acknowledgement cannot undo a committed effect.

Reference SLOs (example)

These are illustrative planning targets, not observed performance or universal enterprise requirements. Establish accepted SLOs and measured recovery time/data loss for the actual workflow/dependencies. Authorized drills must cover authority/approval/credential outages, policy errors, sandbox/egress denial, provider commit with lost response, store failover, stale workers, fan-out exhaustion and cancellation races. Record residual limitations and the accountable recovery owner.

Sizing assumptions (example)

The numbers below are unvalidated capacity assumptions, distinct from control-profile revisions. Do not infer that a workload volume determines regulatory status or the needed isolation. Load-test accepted and denied paths, saturation/backpressure, data-route restrictions and durable-store consistency under representative faults before relying on them.

Workload tiers

• Tier 1 (Dev/PoC): 50 RPS, p95 2s, single region, soft multi-tenancy.
• Tier 2 (Prod mid): 300 RPS, p95 1.5s, active/active regions, partial isolation.
• Tier 3 (Regulated): 1000 RPS, p95 1.2s, hard tenant isolation, residency pinned.

Service baseline (per region)

• Conversation Manager / Compiler / Orchestrator: 6–12 pods, 2 vCPU / 4 GB; autoscale by p95 latency.
• AI Gateway / LLM Router: 4–8 pods, 1–2 vCPU / 2–4 GB; autoscale by in-flight model calls and router latency.
• Policy Engine: 4–8 pods, 1 vCPU / 2 GB; CPU-bound.
• Knowledge Substrate: 6–10 pods, 4 vCPU / 8 GB; memory-heavy.
• Memory Service: 4–8 pods, 2 vCPU / 4 GB; latency-sensitive.
• Vector index: 3–6 shards, 8 vCPU / 32 GB; shard by tenant.
• Graph store: 3 nodes, 8 vCPU / 32 GB; replicated.
• Audit store: append-only; WORM-capable storage with daily compaction.