aegrail

Layer 4 for AI agents. The runtime governance contract that every production agent is missing.


A small, opinionated, provider-neutral runtime that enforces identity, budget, audit, and capability ACL on AI agents — across any LLM provider, any framework, any vertical. Engineers can install it on Friday. CISOs can read the code on Monday.

---

The missing Layer 4

Every production AI agent stack looks like this:

┌──────────────────────────────────────────────────────────┐
│  Layer 5: Domain business logic                          │  insurance / finance / health / legal
├──────────────────────────────────────────────────────────┤
│  Layer 4: Runtime governance contract       ← THE GAP    │  identity, budget, audit, ACL
├──────────────────────────────────────────────────────────┤
│  Layer 3: Agent framework                                │  LangChain, LlamaIndex, AutoGen, MCP
├──────────────────────────────────────────────────────────┤
│  Layer 2: LLM provider SDK                               │  openai, anthropic, google-genai, bedrock
├──────────────────────────────────────────────────────────┤
│  Layer 1: LLM API                                        │  gpt-4o, claude-opus, gemini, llama
└──────────────────────────────────────────────────────────┘

Layer 4 is missing. Frameworks won't fill it (they compete on ergonomics, not opinions). SDKs can't (they ship per-provider). Domain teams shouldn't (every team re-inventing audit chains badly). The model itself can't (it's the thing you're trying to constrain).

So every company building agents in production today either reinvents Layer 4 in-house — eight engineers, eight repos, eight broken audit formats — or pretends it doesn't exist and waits for the regulator. aegrail is the open-source implementation of Layer 4.

---

Why agents specifically need it

A container runtime assumes deterministic code. An agent isn't deterministic. The cloud-native stack — Kubernetes, Istio, Prometheus, OPA — was built around assumptions a microservice satisfies. Agents violate them:

| Property | Microservice | Agent |
|---|---|---|
| Output for the same input | Same | Different every time |
| Execution path | Coded, finite | Decided by the LLM at runtime |
| Cost per request | Sub-cent, predictable | $0.01 to $20+, unbounded |
| Outbound calls | Static dependency graph | LLM decides at runtime |
| Failure mode | Crash / 500 / timeout | "Confidently wrong" — returns 200 with garbage |
| Identity | Service identity | Service identity + invoking user + agent role |
| Trust boundary | Code trusted, input untrusted | Plus: the LLM's own decisions are untrusted |

That's why your agent looped for 63 hours and burned $4,200. Why a malicious PR title made three production coding agents leak their own API keys. Why your platform team can't tell you how many agents are running right now.

Layer 4 codifies the runtime contract that fills the gap — the same way Kubernetes codified the microservice contract in 2014.

---

What it does

Five primitives. Nothing else.

1. Scoped identity — every agent run gets a session-bound principal of the form @sess__. The invoking user is stamped alongside. Audit logs are identity-linked from line one. No shared API keys.

2. Hard budget kill-switches — USD, tokens, wall-clock, recursion depth, tool calls. The runtime raises BudgetExceeded from Python (or returns HTTP 429 from the engine). Not the system prompt. Not the LLM. The runtime.

3. Tamper-evident audit log — append-only JSONL with SHA-256 chain links between events. Identity-linked, replayable, regulator-presentable. Provider-neutral schema: same fields whether the call was Anthropic or OpenAI or Bedrock or Vertex.

4. Per-agent tool ACL — each agent declares its tool registry at construction. Tools not in the registry cannot be called, regardless of what the LLM decides. Optional argument predicates. Three machine-readable deny reasons: not_registered, predicate_false, predicate_error. Maps to OWASP Top 10 for Agentic Applications: ASI02 (Tool Misuse) and ASI03 (Identity & Privilege Abuse).

5. Egress + data boundary _(network layer, via aegrail-engine)_ — the Go sidecar enforces allowlist + token budgets + audit chain at the L7 HTTPS boundary. Works for any agent in any language, including ones that never imported the Python SDK. HTTPS MITM with parsers for OpenAI / Anthropic / Bedrock / Vertex / Gemini / Ollama response shapes.

---

Provider-neutral by design

A serious production agent uses multiple providers — Anthropic for reasoning, OpenAI for structured output, Vertex Gemini for vision, Bedrock for data-residency. Layer 4 has to abstract over Layer 2.

| Capability | OpenAI SDK | Anthropic SDK | Bedrock | Vertex | LangChain | aegrail |
|---|---|---|---|---|---|---|
| Cross-provider identity | — | — | IAM | SA | callbacks | provider-neutral principal |
| Cross-provider cost rollup | — | — | — | — | partial | single Budget |
| Hard kill-switch at runtime boundary | — | — | — | — | — | BudgetExceeded |
| Tamper-evident audit chain | — | — | — | — | — | SHA-256 JSONL chain |
| Per-agent tool registry with arg predicates | — | — | — | — | partial | ASI02/03 aligned |
| Egress allowlist at L7 (any language) | — | — | VPC | VPC | — | engine sidecar |
| Token budget on direct HTTPS | — | — | — | — | — | MITM proxy + parser |
| Provider-neutral event schema | — | — | — | — | partial | yes |

session.record_llm(cost_usd=..., prompt_tokens=..., completion_tokens=..., model=...) accepts numbers, not provider objects. Where the caller got the numbers (provider response, LiteLLM, internal price table) is the caller's problem. The library enforces.

---

What it deliberately does NOT do

Layer 4's value is in being small and load-bearing, not big and ergonomic. aegrail is not:

• Not a prompt firewall — compose with Lakera or Prompt Security upstream
• Not an eval framework — use Langfuse, Braintrust, or Galileo
• Not a model gateway — use LiteLLM or your provider's SDK
• Not a multi-agent orchestrator — use LangGraph / CrewAI / AutoGen / MCP; the contract holds regardless
• Not an IAM replacement — identity attribution, yes; authentication, no

---

How it ships

| Layer | Component | License |
|---|---|---|
| In-process SDK | aegrail (Python) | Apache 2.0 |
| Network sidecar | aegrail-engine (Go + Helm) | Apache 2.0 |

Two components, both Apache 2.0. A security library you can't audit isn't a security library — read the code, audit the enforcement model, run it where you need it.

---

Install

bash
pip install aegrail

> Note: the first PyPI release will be v0.2.0. Until then, install from source: >

bash
> git clone https://github.com/aegrail/aegrail
> cd aegrail && pip install -e .
>

Python 3.10+. Zero hard dependencies beyond pydantic. Works with any LLM provider (OpenAI, Anthropic, Bedrock, raw HTTP). Works alongside any agent framework (LangChain, LlamaIndex, MCP, custom).

---

Hello world

python
from aegrail import Agent, AuditSink, Budget, Tool

def refund(order_id: int) -> str:
    # Your real tool — could be an API call, DB write, anything.
    return f"refunded order {order_id}"

agent = Agent(
    identity="support-bot/v1",
    budget=Budget(usd=5.0, tokens=100_000, wall_seconds=120, max_tool_calls=10),
    audit=AuditSink.file("./audit.jsonl"),
    tools={
        "refund_api.refund": Tool(
            name="refund_api.refund",
            fn=refund,
            description="Issue a refund for a customer order.",
            when=lambda args: isinstance(args.get("order_id"), int),
        ),
    },
)

with agent.session(user_id="alice", task="refund order #4521") as s:
    # 1. Call your LLM however you like (OpenAI SDK, Anthropic SDK, raw HTTP).
    #    Then tell the runtime what it cost. Provider-agnostic by design.
    s.record_llm(
        model="claude-sonnet-4-5",
        tokens_in=120,
        tokens_out=300,
        cost_usd=0.012,
    )

    # 2. Run a registered tool through the session — looked up by name,
    #    arg predicate enforced, counted against the budget, audited.
    result = s.call_tool("refund_api.refund", order_id=4521)

That's it. The session:

• Generates a short-lived per-session principal (support-bot/v1@sess__)
• Tracks tokens and dollars against the budget; raises BudgetExceeded deterministically when hit
• Emits a structured event for every LLM call, tool invocation, and policy denial — identity-linked, append-only
• Refuses tools the agent is not registered for, or tool args that fail the when predicate — raising ToolNotPermitted deterministically (mapped to OWASP ASI02 / ASI03)
• Stops the agent if wall-clock, recursion, or tool-call limits are hit, no matter what the LLM "decides"

If the budget is exceeded mid-loop, or a tool is denied, the session raises. The agent cannot talk its way out of it.

---

Async — AsyncSession (v0.2.2)

For agents running on asyncio (FastAPI, MCP servers, anything using the OpenAI/Anthropic async clients), use agent.async_session(...):

python
import asyncio
from aegrail import Agent, AuditSink, Budget, Tool

async def real_refund(order_id: int) -> str:
    # any async work here — DB call, async HTTP, etc.
    return f"refunded {order_id}"

agent = Agent(
    identity="support-bot/v1",
    budget=Budget(usd=5.0, wall_seconds=30, max_tool_calls=10),
    audit=AuditSink.file("./audit.jsonl"),
    tools={"refund": Tool(name="refund", fn=real_refund)},
)

async def main() -> None:
    async with agent.async_session(user_id="alice") as s:
        await s.record_llm(model="gpt-4", tokens_in=100, tokens_out=200, cost_usd=0.01)
        result = await s.call_tool("refund", order_id=4521)
        print(result)

asyncio.run(main())

The async surface mirrors the sync one — same exceptions, same audit events, same tool ACL semantics — and adds one load-bearing property: wall_seconds is enforced mid-tool-call via asyncio.wait_for. If a tool call hangs past the remaining wall-clock budget, the runtime raises BudgetExceeded('wall_seconds') deterministically, rather than waiting for the call to return. Sync Session could only check at event boundaries.

Tool functions can be sync or async — the runtime detects via inspect.iscoroutinefunction and dispatches accordingly. Sync functions are wrapped in asyncio.to_thread(...) so the timeout still applies at the asyncio level.

Full async demo (against local Ollama, no API key): [examples/async_demo.py](examples/async_demo.py).

---

First 60 seconds

bash
git clone https://github.com/aegrail/aegrail
cd aegrail
pip install -e .

# Happy path — synthetic LLM call, real audit log.
python examples/basic.py

# The kill-switch — agent loops greedily, runtime stops it deterministically.
python examples/budget_kill.py

examples/budget_kill.py prints:

iteration 1: state={'tokens_used': 500, 'usd_used': 0.01, ...}
iteration 2: state={'tokens_used': 1000, 'usd_used': 0.02, ...}
iteration 3: state={'tokens_used': 1500, 'usd_used': 0.03, ...}
iteration 4: state={'tokens_used': 2000, 'usd_used': 0.04, ...}
iteration 5: state={'tokens_used': 2500, 'usd_used': 0.05, ...}

[runtime] killed by reason=usd: usd budget exceeded: 0.0600 > 0.0500

That's the $4,200-weekend scenario, prevented in code.

---

Real-provider examples

Working end-to-end demos with live LLM calls:

• [examples/openai_demo.py](examples/openai_demo.py) — OpenAI Chat Completions
• [examples/anthropic_demo.py](examples/anthropic_demo.py) — Anthropic Messages
• [examples/basic.py](examples/basic.py) — provider-free walkthrough
• [examples/budget_kill.py](examples/budget_kill.py) — the runtime stopping a runaway loop
• [examples/multi_agent_acl.py](examples/multi_agent_acl.py) — _(v0.2)_ FinOps and Architect agents in one process, with cross-agent tool denial enforced deterministically

bash
pip install openai
export OPENAI_API_KEY=sk-...
python examples/openai_demo.py

---

Tool ACL — v0.2

Each Agent carries an explicit catalogue of tools it is permitted to invoke. Two agents in the same process with disjoint registries cannot cross-invoke each other's tools, no matter what the LLM is instructed to do.

python
from aegrail import Agent, AuditSink, Budget, Tool, ToolNotPermitted

finops = Agent(
    identity="finops/v1",
    budget=Budget(usd=1.0, max_tool_calls=10),
    audit=AuditSink.stdout(),
    tools={
        "cost_report": Tool(
            name="cost_report",
            fn=lambda period: f"AWS spend {period}: $84,201.47",
            when=lambda args: args.get("period") in {"mtd", "qtd", "ytd"},
        ),
    },
)

architect = Agent(
    identity="architect/v1",
    budget=Budget(usd=1.0, max_tool_calls=10),
    audit=AuditSink.stdout(),
    tools={
        "deploy_infra": Tool(
            name="deploy_infra",
            fn=lambda env: f"deployed infra to {env}",
            when=lambda args: args.get("env") in {"staging", "prod"},
        ),
    },
)

with finops.session(user_id="alice") as s:
    try:
        s.call_tool("deploy_infra", env="prod")  # not in finops's registry
    except ToolNotPermitted as exc:
        print(exc.reason)   # 'not_registered'
        print(exc.tool_name)  # 'deploy_infra'

Three denial reasons surface on ToolNotPermitted.reason:

• 'not_registered' — the tool name isn't in this agent's registry (ASI03).
• 'predicate_false' — the tool's when(args) predicate returned False (ASI02).
• 'predicate_error' — the predicate raised; the original exception is on __cause__.

Every denial emits a tool_denied audit event with the agent's identity, principal, and a snapshot of the budget — so denied attempts are forensically queryable, not just thrown away.

Tools also accept an optional redact(args) -> dict to control what shows up in the audit payload's args field. The default emits keys only, never values.

---

Where this sits — defense-in-depth at the capability layer

aegrail's tool ACL is one of three complementary layers. Each protects against a different threat; none replaces the others.

| Layer | Enforces | Threat it stops | aegrail role |
|---|---|---|---|
| Network egress (L3/L4) | Which hosts/ports the pod can reach | An agent dials an unapproved domain | _Out of scope today_ — use Kubernetes NetworkPolicy, Cilium, an egress proxy. v0.3 will add a proxy. |
| Tool ACL (L7 capability) | Which named callables an identity may invoke, and with what args | A FinOps agent invokes a deploy tool because the LLM was prompt-injected to | This is v0.2. |
| Process isolation | What the OS lets the agent's process do | A compromised agent reads another agent's memory or files | _Out of scope_ — use containers,

…

Collapse