CyberSec Agent

A security-operations agent prototype built on LangGraph, with a deliberate boundary between untrusted model output and authoritative planning, policy, and audit behavior.

Overview

Security analysts spend much of their time turning raw telemetry into a defensible conclusion: is this indicator malicious, how serious is it, what should be done about it, and who approved that action. CyberSec Agent is a prototype that answers those questions over structured security events and threat-intelligence records — and that makes the answer auditable.

The design premise is that the language model is an untrusted component. The model contributes intent, tool selection, and prose. The risk level, the response plan, the approval requirement, and the audit trail are produced by deterministic rule engines that never read the model’s output. The evaluation harness is built to check that this separation actually holds.

This is a research and engineering prototype. It is not production-ready and not an enterprise SOC product. It is not autonomous: it plans and requests approval, and it does not execute remediation actions against real infrastructure. All security data is synthetically generated by deterministic seed scripts.

Architecture

                    HTTP client (curl / Swagger UI)
                                 │
                     FastAPI — app/api/main.py
          POST /chat     POST /triage     POST /resume     GET /audit/events
                                 │
              Authorization: Bearer  on all four; docs routes stay public
                                 │
            ┌────────────────────┴────────────────────┐
            │                                         │
        /chat path                          /triage + /resume path
      SecurityAgent                            TriageService
      └─ ReAct graph                           └─ HITL graph, server-issued
         (agent ⇄ tools)                          thread handle, resume gate

  LangGraph StateGraph — app/core/graph.py
    agent ── LLM.bind_tools().ainvoke() ──►  ⚠ untrusted model output
      │
      ├─ has tool_calls ──► tools    name-routed execution; every failure becomes
      │         │                    a structured, retryable ToolMessage
      │         └────────► back to agent        (ReAct loop, capped iterations)
      │
      └─ no tool_calls ──► [ HITL graph only ]
                              │
                              plan           deterministic: collect evidence →
                              │              analyze_risk → plan_response
                              │              (does not read the message history)
                              │
                              policy_gate    deterministic: evaluate_policy(plan)
                              │              (does not read the message history)
                              │
                    ┌─────────┴─────────┐
                 allow               require_approval
                    │                   │
                   END            human_approval  ── interrupt() ⇒ graph pauses
                                                        │  resume with a decision
                                                        ▼

  Tools — app/tools/  (pure functions + thin LangChain wrappers)
    query_security_logs    structured event query over the seed log file
    query_threat_intel     exact-match IOC lookup (IP / domain / SHA256)
    analyze_risk           rule-based risk scoring          (pure function)
    plan_response          rule-based response planning     (pure function)

  Security layer — app/security/  (cross-cutting)
    policy.py           evaluate_policy(ResponsePlan) → allow | require_approval
    audit.py            plan digest + audit-record construction     (pure)
    auth.py             static API-key authentication → Principal(subject, role);
                        route-level role admission (viewer / analyst / approver)
    store_protocol.py   AuditStore — the ten-method contract that the graph,
                        the triage service and the API depend on
    store.py            SQLite backend (default): incidents / action_requests /
                        audit_logs / thread_owners — append-only, enforced by
                        eight database triggers
    store_postgres.py   PostgreSQL backend (explicit opt-in): same contract, same
                        append-only controls, twelve database triggers
                        read path: list_audit() — bounded, ordered SELECT only

  Data
    data/security_events.jsonl    144 synthetic events   (generated)
    data/threat_intel.jsonl        29 IOC records        (generated)
    data/audit.db                 created at runtime     (default SQLite backend)
    PostgreSQL                    opt-in; see "PostgreSQL audit backend" below

Two compiled graphs exist in one process: a plain ReAct graph for /chat, and a graph with the human-in-the-loop safety layer for /triage and /resume. They share one model client and one audit store.

Why LangGraph

The orchestration layer is a StateGraph rather than a hand-written loop because the properties this prototype cares about are control-flow properties, and a graph makes them explicit and testable:

• The safety path is structural. plan → policy_gate → human_approval is wired into the graph, so the model cannot skip, reorder, or conditionally bypass it.
• Tool execution is a named boundary. The agent node’s output is routed by tool name into a single tools node; model-produced code is never evaluated.
• Pausing is a state, not a blocking call. interrupt() suspends the graph at human_approval; a later request resumes it with a decision.
• Control flow is testable offline. The graph is compiled from plain functions, so nodes and edges are asserted with a scripted fake model and no network.

LangGraph supplies the state machine and the interrupt/resume mechanism — not the security properties above, which come from the graph’s shape and from the policy gate being a node rather than a tool.

Authentication and Authorization

Every business route (/chat, /triage, /resume, /audit/events) requires an Authorization: Bearer header. Authentication is always on: there is no AUTH_ENABLED=false switch, and a missing or invalid AUTH_API_KEYS configuration makes the process refuse to start rather than serve an open API.

Configuration holds only SHA-256 digests of the keys, never the keys themselves:

# 1. generate a key and store it now — it is shown only here
KEY=$(python -c "import secrets; print(secrets.token_urlsafe(32))")
echo "store this API key: $KEY"

# 2. put only its digest into AUTH_API_KEYS
printf '%s' "$KEY" | sha256sum

AUTH_API_KEYS=[{"sha256":"","subject":"alice","role":"analyst"},{"sha256":"","subject":"bob","role":"approver"},{"sha256":"","subject":"carol","role":"viewer"}]

subject is the identity (used for ownership and assignment); role is the authorization (used for route admission). They are deliberately separate fields: two people can hold the same approver role, which is what makes “the submitter and the approver are different people” expressible. Subjects must be unique, and the digest must be 64 lowercase hex characters.

Role matrix

Two layers of authorization

Role admission answers “does this authenticated principal have the right role for this route” — a viewer calling /triage gets 403.

Object authorization answers “may this principal act on this specific thread” — an approver who was never assigned to a thread gets 404 from /resume, byte-identical to the response for a thread that does not exist.

Approver assignment and the self-approval prohibition

POST /triage requires an explicit approvers list. Before anything is written or executed, the server checks that the list is non-empty, has no duplicates, contains no unknown or non-approver subjects, and does not contain the caller — a submitter can never approve their own run. Any violation returns 422 before any graph call or ownership write. The owner is taken from the authenticated subject, never from the request body.

Error expectations

The 401 body is byte-identical for missing, malformed, and unknown credentials — the response is deliberately not an oracle for “does this key exist”. The 404 body is likewise identical for “unrelated thread” and “unknown thread”.

Audit visibility

GET /audit/events is scope-filtered before any audit content is fetched — the server never loads a thread the caller may not see:

• viewer — full read, including historical threads that have no ownership row (records written before per-thread ownership existed). thread_id is optional.
• analyst — must supply a thread_id, and may read only threads it owns.
• approver — must supply a thread_id, and may read only threads it owns or was explicitly assigned to approve.

An analyst / approver that omits thread_id gets 403; one that supplies a thread outside its scope (including an ownerless or unknown thread) gets 404.

Audit Query API (read-only)

The append-only audit trail is also readable over HTTP through a read-only audit query endpoint. It is an additional read interface over the existing authoritative audit store — not a second audit system and not a second audit writer. It is subject to the same authentication and per-thread scope rules as the rest of the API (see Authentication and Authorization).

# a viewer reads the full trail; analyst / approver must add &thread_id=
curl "http://127.0.0.1:8000/audit/events?limit=50&order=desc" \
  -H "Authorization: Bearer $VIEWER_KEY"

The response is a bare JSON array of AuditRecord records — there is no envelope. For an authorized scope, an empty result returns 200 []. A thread_id outside the caller’s scope does not return 200 []: it returns 404 for analyst / approver, because scope is enforced before any content is fetched (see Audit visibility). A viewer may read any thread — including a historical thread with no ownership row — and receives 200 [] for a thread that has no events.

Boundary: the endpoint reads the existing authoritative audit trail through the configured AuditStore’s list_audit (parameterized SELECT only) — the same contract whether the backend is the default SQLite file or the opt-in PostgreSQL one. It does not mutate audit state and does not alter policy, approval, HITL, or checkpoint state. The client cannot choose the audit database path or connection string — the server keeps using the configured trusted store from settings — and no raw SQL is exposed.

This endpoint is read-only and authenticated: it requires a valid Authorization: Bearer credential, and the read scope is filtered to the caller’s role before any audit content is fetched. Read-only is still not the same as tamper-proof — the audit trail is append-only by construction, not notarised (see Known Limitations).

PostgreSQL audit backend (opt-in)

SQLite remains the default audit backend. Setting no AUDIT_* variable gives the v0.1.0 behaviour exactly: a local SQLite file at data/audit.db.

PostgreSQL is an explicit opt-in. It is selected in one place — the application composition root — and nowhere else. No graph node, tool, or route branches on the backend; they all depend on the same AuditStore contract.

No automatic fallback, and no dual-write. If AUDIT_BACKEND=postgres and the DSN is missing or blank, the application refuses to start — it never quietly writes to SQLite instead. There is exactly one audit store per process, and nothing is written to two places.

If PostgreSQL is unavailable. The store connects lazily (constructing it performs no I/O), so an unreachable server surfaces at the first audit operation rather than at process start. From there the two paths behave differently — this is the current behaviour, not a design goal:

That asymmetry is a known limitation, not a guarantee; it is recorded under Known Limitations.

Running the PostgreSQL integration tests

The repository ships an opt-in, isolated test service (compose.postgres.yaml) that shares no namespace, no volume, and no port with the default compose.yaml topology. compose.yaml itself is not modified.

# 1. start the disposable test server (loopback only, restart policy "no")
docker compose -f compose.postgres.yaml up -d --wait

# 2. provision the two separated roles
#    (a FRESH data directory runs this automatically; an existing volume does not)
docker exec -i cybersec-pg-test-postgres-1 psql -v ON_ERROR_STOP=1 \
  -U cybersec_test -d cybersec_test -f - < scripts/pg/bootstrap_roles.sql

# 3. apply the schema as the migration owner
CYBERSEC_PG_MIGRATION_DSN="postgresql://cybersec_migrator:[email protected]:55432/cybersec_test" \
  uv run --no-sync alembic upgrade head

# 4. run the integration suite (explicitly — the default run deselects it)
uv run --no-sync pytest -m postgres -q

# 5. stop the service WITHOUT deleting the data volume
docker compose -f compose.postgres.yaml down

Two roles, deliberately separated:

The credentials in compose.postgres.yaml, scripts/pg/bootstrap_roles.sql, and the CI workflow are disposable local/CI test credentials. They are not secrets, must never be reused, and must never front real data.

docker compose -f compose.postgres.yaml down stops the service and keeps the pg-test-data volume; only down -v deletes it. That is also the only supported way to reach a genuinely empty database, because the tables are append-only at the database level.

The offline suite does not collect these tests (addopts = -m 'not postgres' in pyproject.toml), so a machine without Docker still runs the whole default suite green. CI runs them in a separate job against a throwaway PostgreSQL service container.

Scope and limits

• Audit, incident, action-request and ownership data only. The PostgreSQL backend stores the same four tables as the SQLite backend (incidents, action_requests, audit_logs, thread_owners). It does not store LangGraph checkpoints. Since v0.4.0 the checkpoints live in a separate local SQLite file (deliberately a different file from the audit database); that file is not part of this backend choice.
• HITL state is no longer tied to one process lifetime — but it is single-writer. Since v0.4.0 the checkpoints are stored in a local SQLite file, so a run paused at human_approval is not lost when the process that paused it exits. This is not a distributed store: it is single-writer, and a SQLite file on a shared volume is not safe for multiple replicas. Moving audit data to PostgreSQL does not change any of this.
• Append-only is a database control, not tamper-proofing. The triggers stop UPDATE, DELETE, and TRUNCATE issued through the application and through the runtime role. They do not stop a privileged administrator, a superuser, a table owner, or a forged INSERT of a plausible-looking record. The trail is append-only by construction, not notarised.
• Not production-ready. No high-availability or backup story, no operational guarantees, no rate limiting, no JWT/OIDC and no signed audit. This is an opt-in persistence backend for a prototype.
• No performance or capacity claim. The backend was exercised on one machine against one throwaway server. No throughput, latency, or scaling claim is made, and none should be inferred from it.
• No exactly-once claim. Audit writes are not deduplicated across processes; a retry can append a second record.
• /chat is not the gated workflow. /chat runs the ReAct graph only. The plan → policy gate → human-approval path exists on /triage and /resume. Switching the audit backend changes neither.

The design rationale and the exact boundary of these controls are in docs/architecture.md §11.6.

MCP Interface (read-only)

The same tool functions are also published over MCP (Model Context Protocol) as an alternate read-only interface for external clients. It does not replace the LangGraph agent path, the policy gate, or HITL.

python -m app.mcp.server        # local stdio server

Transport is local stdio only. No HTTP, SSE, or Streamable HTTP endpoint is implemented, and there is no MCP authentication server — the host process that spawns the server owns the identity boundary.

Three read-only tools are published:

plan_response is not exposed through MCP: response planning remains behind the existing policy/HITL path and is not published as an MCP capability. MCP does not create a second authorization system.

Client-visible MCP schemas do not expose data_path, logs_path, or intel_path; the server binds repository-controlled data locations before delegating to the existing read-only domain functions. This is a property of the published interface as tested, not a general claim about filesystem access.

Key Features

• LangGraph orchestration — a five-node state graph (agent, tools, plan, policy_gate, human_approval) with a conditional ReAct loop and a business-level iteration cap separate from the framework’s recursion limit.
• Structured security-event analysis — events are validated against a Pydantic model with closed vocabularies for event type, severity, and status, plus IPv4 and port validation. Malformed records fail at the boundary instead of flowing into the agent.
• Security tool calling — four declared tools registered in a single source of truth shared by the agent and the graph. The graph routes by tool name and never evaluates model-produced code.
• Threat-intelligence lookup — exact-match indicator lookup over IP, domain, and SHA256 records, deliberately not a similarity search: indicators are unique identifiers, and approximate matching would produce false associations.
• Deterministic risk analysis — a pure rule engine that scores evidence (intelligence hits, failed-login volume) with an explicit false-positive suppression path for indicators marked trusted.
• Risk-aware response planning — a pure rule engine that maps a risk assessment to a closed vocabulary of actions, each carrying its own approval requirement and reversibility flag. It consumes the assessment rather than re-deriving it.
• Policy enforcement — a rule-based policy gate implemented as a mandatory graph node rather than a tool, so the decision cannot be skipped.
• Human-in-the-loop approval — a genuine graph interrupt with a server-issued thread handle, a resume validation gate that rejects unknown, completed, expired, or lost threads, and a lazy approval-timeout lifecycle.
• Append-only audit trail — four tables (incidents, action_requests, audit_logs, thread_owners), eight anti-mutation triggers on SQLite (twelve on PostgreSQL, which adds TRUNCATE coverage), a closed six-event vocabulary that includes failure events, and approval state derived from the audit stream rather than stored as a mutable column.
• Offline evaluation harness — three baselines, five separately reported metric classes, an independent evidence oracle, metamorphic relations, and no composite score.
• Real-provider evaluation harness — a budget governor, a network-egress guard built on interpreter audit hooks, per-cell raw-record persistence with fsync, and frozen failure taxonomy.
• Read-only MCP interface — three read-only tools (security-log query, threat-intel lookup, deterministic risk analysis) published over a local stdio MCP server, reusing the existing tool functions as an alternate read-only surface. Response planning is not published, and no remote transport or authentication server is implemented.
• Read-only audit query API — GET /audit/events exposes the existing append-only audit trail as a bounded, deterministically ordered read surface (thread_id / event / limit / order). It is a read path only: no audit writes, no policy / approval / HITL / checkpoint mutation, and no client control over the audit database path. Reads are scope-filtered by role (viewer: all threads; analyst: owned threads; approver: owned or explicitly assigned threads), enforced before any content is fetched.
• Optional PostgreSQL audit backend — an explicit opt-in persistence backend behind the same AuditStore contract, with a separated migration-owner role and a runtime role limited to SELECT / INSERT. SQLite stays the default, there is no automatic fallback and no dual-write, and the PostgreSQL backend stores audit / incident / action-request data only — not LangGraph checkpoints.
• Static API-key authentication — every business route requires Authorization: Bearer . Configuration stores only SHA-256 digests; authentication is always on and fail-closed (a missing or invalid key configuration stops the process at startup). Role admission (viewer / analyst / approver) is enforced per route as a FastAPI dependency.
• Per-thread object authorization — POST /triage records a single immutable ownership row (owner + full approver set) before the graph runs; POST /resume and GET /audit/events resolve the authenticated principal against that row before touching any graph, checkpointer, or audit content. Self-approval is rejected, and the audit actor is the authenticated principal, not a client-supplied string.
• Fully offline test suite — 1,737 tests that require no network access and no API key, plus a PostgreSQL integration suite (366 cases) that the default run deselects and a separate CI job runs against a throwaway server.

Security Design

Untrusted LLM boundary. The model produces intent and prose, never authority. It holds no shell, filesystem, or network capability, and its output is never the basis for an authorization decision. Concretely, the two nodes that produce authoritative security artifacts read only the caller-supplied indicator and the deterministic plan — they do not read the message history at all. Model output therefore cannot reach the risk level, the response plan, the approval requirement, or the audit record.

Controlled tools. Every capability the model can invoke is a pre-declared tool with a closed argument schema. Tool failures are converted into structured, retryable results rather than raised into the graph, and error payloads omit paths, arguments, and tracebacks. Response planning is deliberately withheld from the model on the approval path so that exactly one source of the plan exists.

Deterministic planning and policy boundaries. Risk scoring, response planning, and policy evaluation are pure functions: no I/O, no clock, no model call, no configuration input. The policy gate is a node, not a tool — a tool would be optional for the model to call, which would make the policy advisory instead of structural. Action properties such as whether an action requires approval, or whether it is reversible, are set by the rule engine.

Human-in-the-loop. High-impact actions pause the graph. Resuming requires a decision payload plus a server-issued thread handle, and the resume path verifies that the thread is genuinely awaiting approval before accepting anything. Whether an action needs approval is computed by the rule engine, never inferred by the model.

Explicit approval assignment. Each triage run records who owns it and which configured approvers may decide it. Only an assigned approver may resume that thread, and the submitter can never approve their own run — the assignment is a recorded fact, not a default derived from a role. Ownership is a single immutable row: owner and the full approver set are written together before the graph starts, so a partial assignment (“owner recorded, no approver”) cannot exist.

Ownership is registered before the graph runs. If the graph then fails (for example a required data source is unavailable), the ownership row legitimately remains — it is immutable and is never silently deleted or reassigned. Such a thread is inert: no approval was requested, so it cannot be resumed; it is not readable by an unrelated analyst or approver; and re-registering the same thread_id fails loudly on the primary key. The only audit event written is a truthful plan.failed.

Append-only audit trail. Every decision and every approval is written to an append-only store, and history is enforced as history by the database: the store exposes no update or delete path, primary keys make duplicate writes fail loudly, and per-table triggers abort any UPDATE or DELETE issued outside the application (PostgreSQL adds a TRUNCATE trigger, because a statement-level wipe does not fire row-level triggers). There is no mutable status column — pending state is derived by querying for the absence of a terminal event. Failures are recorded alongside successes, and failure details deliberately exclude absolute paths and raw exception messages. The audit actor is the authenticated principal, not a client-supplied string: POST /resume ignores the request body’s operator field and attributes the decision to the verified subject.

Known limitation: authentication is a static API key, and the audit actor field is authenticated but not non-repudiable — there is no signature and no separate record of a client-claimed identity, so a principal holding a valid key can still deny having acted. No non-repudiation claim is made anywhere in this prototype.

Evaluation

Evaluation has two independent parts, and they answer different questions. Part A is deterministic, offline, and reproducible from the repository alone: it checks whether the architecture behaves as designed. Part B is a bounded, one-off observation with a real provider: it records what the system did under one specific configuration. Neither part is a security proof.

(A) Offline architecture harness — deterministic, no network. Three baselines — a rule-only pipeline, a ReAct graph with no policy gate, and the full agent with policy enforcement and HITL — are run over a fixed, hand-authored case set. Metrics are reported in five separate classes and are deliberately never combined into a single “agent score”, because a weighted total would fold implementation self-consistency into correctness. The oracle recomputes evidence from raw records without importing the rule modules it is checking; case expectations carry provenance; and golden-set and dataset digests are emitted with every result so that it can be re-verified independently. Metamorphic relations are asserted without referencing any weight constant, and capabilities with no independent oracle are recorded as not-evaluable rather than scored zero.

(B) Real-provider pilot — bounded, one configuration. Evaluated the agent architecture with a controlled 108-run real-provider pilot and audited tool-use, injection-exposure, invocation, and architecture-invariant behavior.

Scope: 108 runs, 8 tasks, 4 parallel baseline configurations, n = 3 repetitions per cell, 197 logical LLM invocations; a single provider and endpoint configuration, requested model deepseek-flash, single execution window. The baselines are parallel experimental conditions, not a ranking. Token accounting is descriptive only; monetary cost is not evaluated; provider-side HTTP attempt counts are unobservable and recorded as unknown rather than zero.

Matched injection testing exposed model-side limitations: of 12 payload-bearing matched cases, 6 entered the model-visible context and were evaluable — all 6 followed the injected instruction, and none resisted. Cases that were never exposed are not counted as resistance. This is a single-provider, single-model, single-window observation over one matched injection task with a denominator of 6; it is not a claim about prompt-injection resistance in general, nor about the provider or the model family.

Architecture invariants — plan digest, policy outcome, and audit event sequence — were invariant across the pilot. They are regression guards on the separation between model behavior and the authoritative safety path, not a security score, and they hold by construction in the current design.

Pilot artifacts are anchored by artifact-level SHA256 digests; 34/34 data-quality checks passed; a secret scan over the final artifacts returned zero hits.

Nothing in this evaluation establishes that the system is secure, and no configuration is claimed to be the best.

Known Limitations

This is a research and engineering prototype. The limitations below are properties of the current implementation, not a backlog of things that were overlooked.

Not production-ready. There is no production hardening: authentication is a static API key (no JWT/OIDC, no rotation, no expiry), there is no rate limiting, no signed audit, and no non-repudiation for the audit actor field. GET /audit/events is read-only and scope-filtered, but that is still not the same as tamper-proof or production-hardened.

Historical threads may have no ownership row. Audit records written before per-thread ownership existed carry no owner. A viewer can still read them; analyst and approver cannot resolve them, so a scoped audit read or a /resume against such a thread returns 404.

Single-service, single-instance persistence. The application is one API process. By default it is backed by a local SQLite file; a PostgreSQL audit backend can be enabled explicitly, but it is optional and the deployment remains a single instance with no cache and no coordination between replicas. Nothing here is claimed to scale horizontally, and the Compose topology is deliberately one service with one named volume.

Audit persistence is pluggable; checkpoint storage is a separate, local concern. Choosing the PostgreSQL backend moves audit, incident, and action-request rows to PostgreSQL. It does not move LangGraph checkpoints: since v0.4.0 those live in a local SQLite file, deliberately a different file from the audit database (the application refuses to start if both point at the same path). A run paused at human_approval can therefore outlive the process that paused it, but this is a single-writer store, not a distributed one.

Checkpoint storage is trusted storage. Checkpoints are deserialised through a narrow allow-list — a fixed set of project models, datetime, and the LangGraph Interrupt type — and anything else is rejected with an error rather than reconstructed. That closes the “construct an arbitrary object from a type identifier stored in the database” path. It does not make a writable checkpoint database safe: whoever can write that file can still replace or corrupt a paused run’s state. Treat that file as you treat the audit database.

A recorded decision and a completed run are not atomic. The audit store and the checkpoint store are independent and share no transaction, so a crash between “the decision was appended” and “the checkpoint advanced” is always possible. The decision is at most once by durable constraint; that is not the same claim as end-to-end exactly-once resume.

Append-only is enforced by the database, but it is not tamper-proofing. The anti-mutation triggers stop UPDATE, DELETE, and TRUNCATE through the application and through the runtime role. They do not stop a privileged administrator, a superuser, a table owner, or a forged INSERT that looks plausible. Pending-approval state is derived rather than stored, which removes one class of silent rewrite, but the trail is append-only by construction — not notarised.

Audit write and audit read fail differently when the database is unavailable. With the PostgreSQL backend, a failed audit write surfaces as HTTP 500, while a failed audit read (GET /audit/events) is mapped to HTTP 503. That asymmetry is the current behaviour, documented rather than promised.

No exactly-once audit write. Writes are not deduplicated across processes; a retry can append a second record. A repeated primary key fails loudly, but a retry that mints a new identifier does not.

The MCP interface is local only. It is a local stdio server with no remote transport and no authentication server; the identity boundary is the host process that spawns it.

The agent path requires a provider. /chat, /triage, and /resume run through the model, and there is no offline mode for that path. The deterministic and read-only parts of the system — tools, policy, audit, MCP — can be exercised without a provider; the agent cannot.

Evaluation scope is narrow. The real-provider pilot is 108 runs over 8 tasks, one provider and endpoint configuration, one requested model, one execution window, and a single matched injection task with a denominator of 6. It is a bounded observation, not a general result.

Invariant checks are regression guards, not a security score. Plan digest, policy outcome, and audit event sequence were invariant across the pilot; they hold by construction in the current design and are reported as architecture regression invariants, not as a measure of safety.

Nothing here establishes that the system is secure. No configuration is claimed to be best, and no claim is made about prompt-injection resistance in general or about any provider or model family.

Quick Start

Requires Python 3.12+ and uv.

uv sync                                    # create the environment and install dependencies

cp .env.example .env                       # then fill in the required variables (below)

uv run python scripts/seed_logs.py         # → data/security_events.jsonl  (144 events)
uv run python scripts/seed_threat_intel.py # → data/threat_intel.jsonl    (29 IOC records)

uv run uvicorn app.api.main:app --reload

The application validates its configuration at startup and refuses to boot if a required value is missing or malformed. Two groups are required:

• LLM access — LLM_MODEL, LLM_BASE_URL, LLM_API_KEY.
• API authentication — AUTH_API_KEYS. Without it the process stops with a validation error (auth_api_keys: Field required).

data/audit.db is created on first run. No AUDIT_* variable is needed for this default path; to opt into PostgreSQL instead, see PostgreSQL audit backend (opt-in).

1. Generate API keys and configure AUTH_API_KEYS

Keys are generated by you and shown once; the configuration stores only their SHA-256 digests, so a leaked config does not hand over a usable credential.

# repeat once per principal; keep each raw key in your secret store
KEY=$(python -c "import secrets; print(secrets.token_urlsafe(32))")
echo "alice's key (store it now): $KEY"
printf '%s' "$KEY" | sha256sum     # → put this digest into AUTH_API_KEYS

A minimal three-role configuration — all values are placeholders, replace the digests with your own:

AUTH_API_KEYS=[{"sha256":"","subject":"alice","role":"analyst"},{"sha256":"","subject":"bob","role":"approver"},{"sha256":"","subject":"carol","role":"viewer"}]

Export the raw keys (not the digests) so the examples below can use them:

export ALICE_KEY=""
export BOB_KEY=""
export CAROL_KEY=""

2. Call the API

Every business route requires the bearer header. Below, alice is an analyst, bob an approver, and carol a viewer.

# Conversational analysis (analyst / approver)
curl -X POST http://127.0.0.1:8000/chat \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ALICE_KEY" \
  -d '{"message": "分析一下 203.0.113.66 最近的登录活动"}'

# Start a triage run (analyst / approver). `approvers` is required and must list
# configured approver subjects other than the caller.
curl -X POST http://127.0.0.1:8000/triage \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ALICE_KEY" \
  -d '{"indicator": "203.0.113.66", "event_type": "login_failed", "approvers": ["bob"]}'

# Resume the paused run (approver only, and only an assigned one).
# thread_id comes from the /triage response.
curl -X POST http://127.0.0.1:8000/resume \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BOB_KEY" \
  -d '{"thread_id": "", "status": "approved", "operator": "bob"}'

# A viewer reads the whole trail
curl "http://127.0.0.1:8000/audit/events?limit=50&order=desc" \
  -H "Authorization: Bearer $CAROL_KEY"

# A scoped read — analyst / approver must supply thread_id
curl "http://127.0.0.1:8000/audit/events?thread_id=&order=asc" \
  -H "Authorization: Bearer $ALICE_KEY"

operator is retained for request compatibility only: the authoritative audit actor is the authenticated principal, so POST /resume attributes the decision to bob regardless of what the body says.

Expected errors (full table under Authentication and Authorization): no header or a bad key → 401; a viewer calling /triage → 403; an unassigned approver resuming a thread → 404; a missing or invalid approvers list → 422.

Interactive API documentation is served at — the docs routes themselves stay public.

Run the test suite — it needs no API key and makes no network calls:

uv run pytest -q

CI runs on GitHub-hosted Ubuntu in two independent jobs. The test job installs the locked uv environment and runs the offline pytest suite. The test-postgres job starts a throwaway PostgreSQL 16 service container, provisions the two separated roles, and runs pytest -m postgres explicitly — a missing or unreachable database fails that job rather than skipping it. Both jobs have been executed on a remote runner. The suite has grown since the run below (the current local count is 1,737 offline / 366 PostgreSQL; see Project Structure). The most recent verified remote observation is run 37737872748 at commit 4334b14d, where the test job reported 1,517 passed, 311 deselected, 2 warnings and the test-postgres job reported 311 passed, 1,517 deselected, 7 warnings, with both jobs succeeding. An earlier observation was the Phase 9.3-F commit dd295c8, which passed 1,463 tests on GitHub-hosted Ubuntu 24.04. Each figure records one specific run on one specific commit — it is not a standing guarantee about the current head, and it is not a claim of universal Linux compatibility. Check the Actions tab for the latest result.

Demo Walkthrough

There are two independent ways to see the system work, and they have different prerequisites.

(A) Provider-backed agent walkthrough. /chat, /triage, and /resume all run through the LangGraph agent, so all three require a configured provider (LLM_MODEL / LLM_BASE_URL / LLM_API_KEY in .env); the application refuses to start without them. With valid credentials, a representative end-to-end run is:

uv run python scripts/seed_logs.py             # 1. generate the demo telemetry
uv run python scripts/seed_threat_intel.py     #    and the IOC records
uv run uvicorn app.api.main:app --reload       # 2. start the API

# 3. ask the agent a question — the model selects tools, the answer is prose
curl -X POST http://127.0.0.1:8000/chat \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ALICE_KEY" \
  -d '{"message": "帮我分析一下最近服务器有没有受到攻击"}'

# 4. start a triage run — a high-impact plan pauses for approval.
#    `approvers` must name a configured approver other than the caller.
curl -X POST http://127.0.0.1:8000/triage \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ALICE_KEY" \
  -d '{"indicator": "203.0.113.66", "event_type": "login_failed", "approvers": ["bob"]}'

# 5. resume the paused run — only an assigned approver may do this
#    (thread_id comes from /triage)
curl -X POST http://127.0.0.1:8000/resume \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BOB_KEY" \
  -d '{"thread_id": "", "status": "approved", "operator": "bob"}'

# 6. read the audit trail the run produced (viewer, or the assigned approver)
curl "http://127.0.0.1:8000/audit/events?thread_id=&limit=50&order=desc" \
  -H "Authorization: Bearer $BOB_KEY"

What is stable in this walkthrough: the four routes and their contracts, that /triage either ends in a policy decision or pauses for approval, that resuming requires a decision plus the server-issued handle, and that each step appends audit events. What is not stable: the model’s exact wording, and whether a given indicator trips the approval gate — the risk level and the approval requirement are produced by the rule engine, not by the model.

(B) Offline, provider-free capability demo. The MCP interface is a separate, deterministic read-only surface over the same tool functions. It needs no provider credentials, because it never calls a model:

uv run python -m app.mcp.server     # local stdio MCP server

It publishes three read-only tools — query_security_logs, query_threat_intel, analyze_risk — over local stdio. This demonstrates the deterministic tool layer (structured event query, exact-match IOC lookup, rule-based risk scoring), and nothing else.

This is not an offline substitute for the agent path. It does not exercise the model, the ReAct loop, the plan stage, the policy gate, the human-approval interrupt, or the audit lifecycle — none of those are reachable without a provider.

Running with Docker Compose

The repository also ships a minimal container deployment — a single Dockerfile and a compose.yaml at the repository root. It is an additional supported execution path, not a replacement for the local uv workflow above, and it reuses the same .env configuration.

cp .env.example .env          # same runtime configuration as the local workflow
docker compose up --build

The API is then served at , with interactive documentation at .

What is baked into the image. The image installs the locked runtime dependencies, copies the application, and generates the deterministic demo inputs at build time, so the container starts with its own data:

/app/data/security_events.jsonl   144 synthetic events  (image-baked, deterministic)
/app/data/threat_intel.jsonl       29 IOC records       (image-baked, deterministic)

What persists. Compose declares exactly one service and one named volume:

Mutable audit state is the only state that outlives a container. Because the volume mounts at /data and the demo inputs live under /app/data, the mount does not shadow the image-baked demo data. Recreating the container keeps the audit trail; removing the volume does not.

Runtime configuration stays outside the image. LLM_MODEL / LLM_BASE_URL / LLM_API_KEY are supplied at runtime from .env (or from the surrounding environment), exactly as in the local workflow. .env is excluded from the build context, so real credentials are never baked into the image — and, as with the local workflow, the application refuses to start without them.

MCP is not a Compose service. This deployment does not publish MCP over the network. It remains the local stdio server described under MCP Interface (read-only), started by whichever host process wants it; there is no remote MCP transport and no MCP authentication server.

Scaling boundary. The Compose deployment is intentionally a single API service with SQLite persistence — a local / demo / single-instance boundary. It is not a horizontally scaled or multi-replica design, and this topology deploys no external database or cache. The opt-in PostgreSQL backend is not part of it: the test service lives in a separate Compose project (compose.postgres.yaml), and enabling PostgreSQL for the application would require supplying a server that this topology does not provide.

Compose version. compose.yaml uses the optional env_file long syntax with required: false, which needs a Docker Compose release supporting that syntax (Compose ≥ 2.24).

Project Structure

app/
  api/          FastAPI application, routes, request/response contracts
  core/         configuration, model client, LangGraph state graph, agent, triage service
  schemas/      Pydantic models: events, threat intel, risk, response, approval, audit
  security/     policy engine, static API-key authentication and role admission, per-thread object authorization, append-only audit store (SQLite default, PostgreSQL opt-in)
  tools/        security tools (pure core functions + LangChain wrappers)
  evaluation/   offline architecture harness and real-provider evaluation harness
  mcp/          read-only MCP adapters and the local stdio server
data/           generated seed data and the runtime audit database (not committed)
docs/           architecture design notes
scripts/        deterministic seed-data generators; scripts/pg/ holds the role bootstrap SQL
tests/          1,737 offline tests, plus a PostgreSQL integration suite (deselected by default)
migrations/     Alembic environment and the hand-written baseline schema
alembic.ini     Alembic configuration — the connection URL comes from the environment, never from this file
Dockerfile      minimal container image (Phase 9.3-G)
.dockerignore   build-context exclusions (secrets, host data, agent artifacts)
compose.yaml    single-service Compose topology with the audit-data volume
compose.postgres.yaml  opt-in, isolated PostgreSQL test service (outside the default topology)

Roadmap

License

This project is licensed under the MIT License. See the LICENSE file for the full text.