Agentic Incident Command Center
Governed IT and security incident response with bounded remediation, exact approval, verified effects and owned recovery.
Purpose
Run an incident from signal to verified remediation without turning the agent into an unbounded admin. A conforming runtime can coordinate detector, diagnosis, change-planning, communications, and remediation agents against the same evidence snapshot, then evaluate every action against native risk, authority, evidence, and policy-selected obligations. Network transport alone does not imply a human gate.
Admit exact contextos.enterprise@1 and contextos.durable-delegated@1 requirements for this delegated/durable response design, plus actual network/code/data/effect features; require high-assurance evidence where the service’s risk demands it. The named agents can be logical roles. Assign service-specific stop/escalation/reconciliation owners, recovery objectives and residual risks. The repository supplies reference validators, not an incident-control service or proof of production restoration.
Why this is agentic-first
Classic automation works when the incident matches a runbook. Real incidents often cross APM, SIEM, cloud, ticketing, IAM, and customer communications. Gartner uses cybersecurity threat response as an example of task-specific agents that scan logs and behavior, assess the issue, and initiate response. ServiceNow describes agent orchestration across network management, SIEM, APM, and human network approval for incident resolution.
This is a ContextOS-native use case because the problem is not “summarize logs.” It is “decide what can safely be changed, who must approve it, and how to replay the decision later.”
Context Pack
| Layer | Required entries |
|---|---|
decision_layer.decision_specs[] | incident.triage.classify, incident.remediation.execute, incident.customer_update.publish |
policy_layer.policy_bundles[] | Severity policy, change-freeze policy, service ownership policy, data-disclosure policy |
policy_layer.approval_gates[] | GATE_SRE_APPROVAL, GATE_SECURITY_APPROVAL, GATE_CHANGE_MANAGER |
tooling_layer.adapter_registry[] | adp_apm.query, adp_siem.search, adp_cloud.scale, adp_iam.revoke_session, adp_statuspage.post, adp_ticket.update |
memory_layer.memory_policy.write_classes_allowed | evidence_link, correction, decision_outcome; incident-pattern names are domain content, not new canonical write classes |
evaluation_layer.eval_targets[] | P/U/L/S/E targets; false positives, remediation recovery, approval and diagnosis times are linked domain-suite measures |
Agent roles
| Agent | Responsibility | Boundary |
|---|---|---|
| Signal Agent | Correlates alerts, traces, deploys, and user-impact signals. | Read-only across observability and ticketing. |
| Diagnosis Agent | Proposes likely root cause and confidence. | Cannot change systems. |
| Remediation Planner | Produces candidate actions with blast radius and rollback plan. | Cannot execute without policy decision. |
| Communications Agent | Drafts internal and external status updates. | Publication is gated. |
| Guardian Agent | Verifies policy, scope, evidence, and rollback readiness. | Can block or escalate any step. |
Execution flow
invokeAgentarrives withintent=incident.resolve, service, symptoms, and active alert IDs.- Trusted admission resolves current service/owner/change-calendar/policy/snapshot pins and exact controls; the compiler consumes supplied inputs and stops on applicable policy-expression errors.
- Signal Agent correlates APM, logs, traces, deploy history, and open incidents.
- Diagnosis Agent emits candidate causes with evidence references.
- Remediation Planner emits one or more plans, each with native action-risk dimensions, a conservative v1 projection, recovery semantics, and expected user impact.
- Guardian Agent checks blast radius and policy constraints.
- Eligible read-only checks may execute inline. Scaling, IAM, config, database, and network actions route through the approval or escalation obligation selected by policy; some are denied outright.
- The Gateway checks complete stored action/approval binding, current eligible roles/quorum/redemption, scoped authority and ancestor revocation, actual boundary/data routes and original aggregate reservation; then claims the stable operation under current revision/generation before dispatch.
- Communications Agent drafts updates; publication is gated when customer-visible.
- The implementing runtime emits a
DecisionRecordwith evidence, applicable approvals, controls, trace, and post-action verification.
Provider timeout or lost acknowledgement preserves an unknown effect and reserved capacity. Reconcile the real target before any retry/replan effect; provider-specific idempotency and current-state preconditions have finite scope/retention. Rollback or compensation needs a new governed operation and may fail or be impossible. A release re-pin, stop click or cancelled run does not prove restoration.
Propagation of stop and grant revocation must reach child work and actual dispatch enforcers. Recovery requires current ownership/generation and required target fencing, exact checkpoint pins, current authority/approval/boundary/external checks and the original shared budget/deadline. A region failover or restarted worker cannot mint fresh capacity. Emergency handling uses explicitly scoped, expiring authority and independent review; it cannot silently lower mandatory controls or a protected tenant/data boundary.
APM/SIEM logs, forensics, code workers, model inference, telemetry and status messages are actual data routes. Verify source labels, purpose, recipients/processors/residency and retention; redact/minimize with bound transformation evidence where required. An externally visible status update is its own governed effect. Exercise authorized lost-response, stale-worker, revocation, outage and cancellation races and record observed recovery results for the exact release/environment. See Execution Contract and Conformance.
Decision gates
| Gate | Trigger | Required evidence |
|---|---|---|
GATE_SRE_APPROVAL | Any production remediation with customer impact. | service owner, active incident, rollback plan, blast radius |
GATE_SECURITY_APPROVAL | IAM, token revocation, firewall, containment, or forensic action. | security severity, affected identities, containment rationale |
GATE_CHANGE_MANAGER | Change-freeze window, regulated service, or irreversible migration. | change calendar, exception reason, business owner |
Failure modes
- Alert storm - evidence pack must deduplicate alerts by service, symptom, deploy, and user impact before planning.
- Wrong root cause - verify postconditions and reconcile existing effects before proposing another bounded action; agent confidence or consensus does not establish the cause.
- Unsafe remediation - block actions without required current authority/approval, tenant/data boundaries and explicitly reviewed recovery limits; irreversible effects cannot invent rollback evidence.
- Tool side-effect race - all privileged tools require idempotency keys and current-state preconditions.
- Incident communication leak - external updates must pass data-disclosure policy before publishing.
Metrics
- Mean time to classify, mean time to propose, mean time to approve, mean time to restore.
- Remediation acceptance rate by severity.
- Rollback rate and failed-change rate.
- False-correlation rate between alert and root cause.
- Evidence completeness at approval.
- Human override and Guardian block rate.
Research signals
- Gartner predicts task-specific agents in 40% of enterprise apps by 2026, with cybersecurity threat response as a named example.
- ServiceNow describes agent orchestration for network security incidents, including coordination across SIEM, APM, and human approval.
- McKinsey’s agentic AI security playbook emphasizes ownership, access rights, agent-to-agent controls, and traceability.