> ## Documentation Index > Fetch the complete documentation index at: https://beige.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Beige Architecture > Detailed technical architecture and data flows This document provides detailed technical reference. For a beginner-friendly overview, see [The Gateway](/gateway). *** ## Overview Beige is a secure, sandboxed agent system where: * A **gateway** process orchestrates everything: LLM calls, policy enforcement, audit logging, sandbox lifecycle. * Each **agent** runs inside its own Docker container (sandbox). All code execution happens there. * The gateway exposes exactly **4 core tools** to the LLM: `read`, `write`, `patch`, `exec`. * Additional **tools** (like `git`, `browser`, `slack`) are mounted into sandboxes as executable launchers. Agents invoke them via `exec`. * Tool launchers are thin clients that call back to the gateway over a **Unix domain socket** for actual execution. * The gateway uses the **pi SDK** (`@mariozechner/pi-coding-agent`) for all LLM interaction, supporting Anthropic, OpenAI/ZAI, and any compatible provider. ## Data Flow ```text theme={null} User (Telegram/CLI) │ ▼ Gateway (Node.js) ├── Channel Adapter (GrammY / CLI) ├── Agent Manager (session per agent) ├── pi SDK (LLM calls) │ └── Core Tools: read, write, patch, exec │ │ │ ▼ │ Policy Engine (check permissions, log) │ │ │ ▼ │ Sandbox Router │ │ │ ▼ │ Docker Exec (run command in container) │ ├── Socket Server (one Unix socket per agent) │ │ │ ▼ │ Tool Request from sandbox launcher │ │ │ ▼ │ Policy Engine (check permissions, log) │ │ │ ▼ │ Tool Runner (execute on gateway host or in sandbox) │ │ │ ▼ │ Response back through socket → sandbox → exec result → LLM │ └── Audit Logger (every tool invocation) ``` ## Core Tool Call Flow (e.g. `exec curl https://example.com`) 1. LLM calls `exec` with args `["curl", "https://example.com"]` 2. Gateway receives the tool call (it's a pi SDK custom tool) 3. Gateway logs: `{agent: "travel", tool: "exec", args: ["curl", "..."], decision: "allowed"}` 4. Gateway runs `docker exec curl https://example.com` 5. Gateway captures stdout/stderr, returns to LLM ## Tool Launcher Call Flow (e.g. `exec git status`) Since `/tools/bin` is on `$PATH`, the agent calls tools naturally. When the LLM runs `exec git status`: 1. LLM calls `exec` with args `["git", "status"]` 2. Gateway receives the tool call, logs it as a core `exec` call 3. Gateway runs `docker exec sh -c "git status"` 4. The shell finds `/tools/bin/git` on `$PATH` — a generated launcher script that: * Connects to `/beige/gateway.sock` (Unix socket mounted into container) * Sends: `{"tool": "git", "args": ["status"]}` * Waits for response 5. Gateway socket server receives the request: * Identifies agent from socket identity (each agent has its own socket) * Logs: `{agent: "travel", tool: "git", args: ["status"], decision: "allowed"}` * Checks policy (is "travel" agent allowed to use "git"?) * Executes the git tool handler on the gateway host * Returns result through socket 6. Launcher prints the result to stdout, exits 7. `docker exec` captures output, returns to gateway 8. Gateway returns exec result to LLM ## Components ### 1. Config (`config.json5`) Single config file (JSON5 — JSON with comments) drives the entire system. No defaults — everything is explicit. ```json5 theme={null} { llm: { providers: { anthropic: { apiKey: "${ANTHROPIC_API_KEY}" }, // zai: { // baseUrl: "https://api.zai.com/v1", // apiKey: "${ZAI_API_KEY}", // api: "openai-completions", // }, }, }, plugins: { git: { path: "./plugins/git", }, // browser: { // path: "./plugins/browser", // config: { browserUrl: "ws://localhost:9222" }, // }, }, agents: { travel: { model: { provider: "anthropic", model: "claude-sonnet-4-6", thinkingLevel: "medium", }, fallbackModels: [ { provider: "anthropic", model: "claude-3-5-sonnet-20241022" }, ], tools: ["git"], // Per-agent plugin config overrides — deep-merged with plugins.git.config // pluginConfigs: { // git: { allowForcePush: ["del"] }, // }, sandbox: { image: "beige-sandbox:latest" }, }, }, } ``` ### 2. Gateway Core (`src/gateway/`) * **Config loader**: Reads and validates `config.json5`, resolves env vars. * **Agent manager**: Creates/destroys agent sessions. Maps session keys → pi SDK `AgentSession` + Docker container. Supports multiple concurrent sessions per agent (one per channel/chat/thread). Handles model fallback and rate limit tracking. * **Provider health tracker**: Monitors per-provider rate limits and cooldowns. Persisted to `~/.beige/data/provider-health.json`. * **Concurrency limiter**: Per-provider concurrency control. When a provider has a `concurrency` limit configured, requests beyond that limit queue up (FIFO) until a slot frees up. Prevents overwhelming providers with too many parallel LLM calls. * **LLM proxy**: The `/api/chat/stream` endpoint proxies LLM calls for the TUI. Resolves API keys server-side, executes prePrompt/postResponse hooks, applies fallback model logic on rate limits, and audit-logs every attempt. The TUI never needs API keys. * **Sandbox manager**: Creates Docker containers with correct mounts, generates tool launchers, manages lifecycle. * **Socket server**: One Unix domain socket per agent at `~/.beige/sockets/.sock`, mounted into container at `/beige/gateway.sock`. * **Policy engine**: Checks if agent is allowed to use a tool. Deny by default. * **Audit logger**: Logs every tool invocation with agent, tool, args, decision, timing, result summary. #### Model Fallback When an agent has `fallbackModels` configured, the gateway uses a multi-pass retry loop: 1. Tries each model in order (primary first, then fallbacks) 2. If a model fails, moves to the next one — but keeps it eligible for retry on the next pass 3. Loops through the model list up to **3 passes** before giving up 4. A model is removed from the retry pool when it either: * Has a rate-limit cooldown active (`retryAfter` in the future), OR * Has accumulated **3 consecutive failures** in this prompt run 5. Returns the first successful response, or an error if all models are exhausted This handles transient errors (e.g. Anthropic "overloaded") gracefully: if model A fails once and model B also fails once, model A gets another shot on the next pass — the overload may have cleared by then. With 2 models, this means up to 6 total attempts before giving up. Rate limit vs. transient error handling: * **Rate limits** (HTTP 429, `rate_limit` / `too many requests` patterns): marked as globally cooling down — all sessions skip this model until the cooldown expires * **Transient capacity errors** (`overloaded`, `capacity`, `temporarily unavailable`): NOT marked as globally cooling down — other sessions can still use the model. The failing session retries via the multi-pass loop. * If a rate-limit response includes a `retry-after` header, that time is used for the cooldown * Otherwise, hard rate limits (HTTP 429) default to 5-minute cooldown; soft rate limits (pattern-matched) default to 60 seconds * Cooldown state is persisted in `~/.beige/data/provider-health.json` and survives restarts * A `modelSwitched` hook fires when a session ends up on a non-primary model, allowing plugins to send notifications #### Provider Concurrency Limits When multiple agents or sessions share a provider, the total number of parallel LLM requests can spike — especially with channels that handle many conversations simultaneously. The `concurrency` setting on a provider caps how many requests can be in-flight at once: ```json5 theme={null} { llm: { providers: { zai: { apiKey: "...", concurrency: 5, // max 5 parallel LLM calls to this provider }, }, }, } ``` * Requests beyond the limit are queued (FIFO) and await until a slot frees up — they are **never rejected**. * Default: no limit (`-1` / omitted). All requests pass through immediately. * The limiter sits inside `executePromptWithModel`, so it applies to both `prompt()` and `promptStreaming()` calls uniformly. * Concurrency is tracked per provider, not per model — all models on the same provider share the same pool. This is complementary to the rate-limit fallback system: concurrency limits **prevent** overloading a provider in the first place, while fallback handles errors **after** they occur. ### 3. Core Tools (`src/tools/core.ts`) Implemented as pi SDK `ToolDefinition` objects: * **`read`**: Read a file from the sandbox. Gateway runs `docker exec cat `. * **`write`**: Write content to a file in the sandbox. Gateway pipes content via `docker exec`. * **`patch`**: Apply a find-and-replace patch to a file in the sandbox. Gateway reads, patches, writes back. * **`exec`**: Execute a command in the sandbox. Gateway runs `docker exec `. All four: log first, check policy, then execute. ### 4. Plugins (`src/plugins/`) Each plugin is a directory: ```text theme={null} plugins/git/ ├── plugin.json # manifest: name, description, what it provides ├── index.ts # entry point (exports createPlugin) ├── README.md # documentation (mounted for agent context) └── SKILL.md # agent usage guide ``` `plugin.json`: ```json theme={null} { "name": "git", "description": "Git version control operations.", "commands": ["status", "commit", "log"], "provides": { "tools": ["git"] } } ``` The gateway: 1. Loads all plugins from config (calls `createPlugin()` and `register()`) 2. Plugins register tools, channels, hooks, and skills via `PluginRegistrar` 3. Validates that all agent tool references resolve to registered tools 4. Generates a launcher script per tool per agent 5. Mounts launchers read-only into `/tools/bin/` (which is prepended to `$PATH`) 6. Mounts plugin packages read-only into `/tools/packages/` At runtime, `ToolRunner` routes tool calls to the handler registered by the plugin that provides that tool. ### 5. Tool Launcher (generated, mounted into sandbox) A small shell script that connects to the gateway socket: ```bash theme={null} #!/bin/sh # Auto-generated by beige gateway. DO NOT EDIT. # Tool: git | Target: gateway exec /beige/tool-client "$TOOL_NAME" "$@" ``` Where `/beige/tool-client` is a small binary/script (mounted read-only) that: * Connects to `/beige/gateway.sock` * Sends JSON request with tool name + args * Reads JSON response * Prints result to stdout * Exits with appropriate code ### 6. Socket Protocol Request (sandbox → gateway): ```json theme={null} { "type": "tool_request", "tool": "git", "args": ["set", "mykey", "myvalue"] } ``` Response (gateway → sandbox): ```json theme={null} { "type": "tool_response", "success": true, "output": "OK", "exitCode": 0 } ``` Error response: ```json theme={null} { "type": "tool_response", "success": false, "error": "Permission denied: tool 'git' not allowed for agent 'travel'", "exitCode": 1 } ``` ### 7. Channels The TUI is the only built-in channel. Other channels (Telegram, Discord, etc.) are provided by plugins. * **TUI** (built-in, separate process): Connects to gateway HTTP API. Runs pi's `InteractiveMode` locally for the full pi experience. LLM calls are proxied through the gateway's `/api/chat/stream` endpoint (the TUI never needs API keys). Tool execution is proxied through the gateway's `/api/agents/:name/exec` endpoint. The gateway handles auth, hooks, fallback models, and audit logging for both paths. * **Plugin channels** (in-process): Plugins can register channel adapters and run background processes (e.g. Telegram polling). They create sessions via `PluginContext.prompt()` and receive responses through the channel adapter. #### Session Settings System Sessions support layered settings with three levels of precedence (highest wins): 1. **System default** — hardcoded in the gateway 2. **Plugin/channel config default** — set in plugin config 3. **Session override** — set by user via commands, persisted in `~/.beige/sessions/session-settings.json` ```text theme={null} ┌─────────────────────────────────────────────────────────────┐ │ Setting Resolution │ ├─────────────────────────────────────────────────────────────┤ │ System Default (false) │ │ ↓ overridden by │ │ Plugin/Channel Config Default (plugin config: defaults.verbose) │ │ ↓ overridden by │ │ Session Override (user: /verbose on) │ └─────────────────────────────────────────────────────────────┘ ``` #### Verbose Mode When verbose mode is enabled, the channel receives a callback (`onToolStart`) before each tool execution and can notify the user: | Channel | Notification Method | Example | | -------- | ------------------- | ------------------------------------- | | Telegram | Bot message in chat | `🔧 exec: ls -la` | | TUI | stderr output | `🔧 exec: ls -la` (appears above TUI) | This gives users visibility into what the agent is doing without cluttering the main response. #### Channel Commands Commands are handled locally by the channel and **not** sent to the LLM: | Command | Telegram | TUI | Description | | | ------------- | -------- | --- | ------------------------------ | ---------------------- | | `/start` | ✅ | — | Welcome message + command list | | | `/new` | ✅ | — | Start fresh session | | | `/status` | ✅ | — | Show session info + settings | | | \`/verbose on | off\` | ✅ | ✅ | Toggle verbose mode | | \`/v on | off\` | ✅ | ✅ | Shorthand for /verbose | On startup, the Telegram channel registers its commands with the bot API (deleting stale commands first). ### 8. Sandbox Docker Image Minimal image with: * Deno runtime (for TypeScript execution) * Common utilities (curl, jq, etc.) * `/beige/tool-client` — socket client script (baked into image, not mounted) * No secrets, no env vars from host * Mounts: * `/workspace` (read-write) → `~/.beige/agents//workspace/` * `/tools/bin` (read-only) → generated launchers * `/tools/packages` (read-only) → tool source packages * `/beige/gateway.sock` (Unix socket) ### 9. Identity & Auth Model * Each agent has its own Unix socket file. Gateway creates socket, mounts it. * Gateway identifies agent by which socket received the connection. * No payload-based identity. Agent cannot claim to be another agent. * Threat model: even if an agent somehow accesses another agent's socket file (impossible due to separate containers), the gateway still enforces per-socket identity. ### 10. Audit Log Format JSONL file at `~/.beige/logs/audit.jsonl`: ```json theme={null} { "ts": "2026-03-05T00:00:00.000Z", "agent": "travel", "type": "core_tool", "tool": "exec", "args": ["curl", "https://example.com"], "decision": "allowed", "durationMs": 1234, "exitCode": 0, "outputBytes": 4567 } ``` ```json theme={null} { "ts": "2026-03-05T00:00:01.000Z", "agent": "travel", "type": "tool", "tool": "git", "args": ["set", "mykey", "myvalue"], "decision": "allowed", "target": "gateway", "durationMs": 12, "exitCode": 0, "outputBytes": 2 } ``` ## Tech Decisions | Decision | Choice | Rationale | | -------------------- | -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | LLM layer | pi SDK (`@mariozechner/pi-coding-agent`) | Handles providers (Anthropic, OpenAI/ZAI, etc.), streaming, sessions, auth. No need to reimplement. | | Gateway runtime | Node.js + TypeScript | pi SDK is Node-native. GrammY is Node-native. | | Sandbox runtime | Deno (inside Docker) | Native TS execution, no build step, secure-by-default permissions. | | Container runtime | Docker | Works on Docker Desktop (Mac) and Linux. | | Socket | Unix domain socket | Peer identity from connection (not payload). Simple, fast, no TCP overhead. | | Config format | JSON5 | JSON with comments. Human-readable, familiar syntax, env var interpolation. | | Audit format | JSONL | Append-only, streamable, parseable. | | Session settings | JSON file (`session-settings.json`) | Simple persistence for per-session overrides. Layered with channel defaults. | | Build tool | tsx (dev) / tsc (build) | Fast dev iteration with tsx, standard tsc for production. | | Install strategy | Lazy first-run setup (no `postinstall`) | `postinstall` runs on `npm install` in dev too and cannot distinguish `--global`. Lazy setup in `src/install.ts` fires only on first real command, skips entirely for source installs (detected via `.git` at package root). | | npm package contents | `dist/` + `tools/` (via `files` in `package.json`) | Tool packages must ship with the npm package so `beige setup` can copy them to `~/.beige/tools/`. |