Beige Architecture

Detailed technical architecture and data flows

This document provides detailed technical reference. For a beginner-friendly overview, see The Gateway.

---

Overview

Beige is a secure, sandboxed agent system where:

• A gateway process orchestrates everything: LLM calls, policy enforcement, audit logging, sandbox lifecycle.
• Each agent runs inside its own Docker container (sandbox). All code execution happens there.
• The gateway exposes exactly 4 core tools to the LLM: read, write, patch, exec.
• Additional tools (like git, browser, slack) are mounted into sandboxes as executable launchers. Agents invoke them via exec.
• Tool launchers are thin clients that call back to the gateway over a Unix domain socket for actual execution.
• The gateway uses the pi SDK (@mariozechner/pi-coding-agent) for all LLM interaction, supporting Anthropic, OpenAI/ZAI, and any compatible provider.

Data Flow

User (Telegram/CLI)
  │
  ▼
Gateway (Node.js)
  ├── Channel Adapter (GrammY / CLI)
  ├── Agent Manager (session per agent)
  ├── pi SDK (LLM calls)
  │     └── Core Tools: read, write, patch, exec
  │           │
  │           ▼
  │     Policy Engine (check permissions, log)
  │           │
  │           ▼
  │     Sandbox Router
  │           │
  │           ▼
  │     Docker Exec (run command in container)
  │
  ├── Socket Server (one Unix socket per agent)
  │     │
  │     ▼
  │   Tool Request from sandbox launcher
  │     │
  │     ▼
  │   Policy Engine (check permissions, log)
  │     │
  │     ▼
  │   Tool Runner (execute on gateway host or in sandbox)
  │     │
  │     ▼
  │   Response back through socket → sandbox → exec result → LLM
  │
  └── Audit Logger (every tool invocation)

Core Tool Call Flow (e.g. exec curl https://example.com)

1. LLM calls exec with args ["curl", "https://example.com"]
2. Gateway receives the tool call (it’s a pi SDK custom tool)
3. Gateway logs: {agent: "travel", tool: "exec", args: ["curl", "..."], decision: "allowed"}
4. Gateway runs docker exec <container> curl https://example.com
5. Gateway captures stdout/stderr, returns to LLM

Tool Launcher Call Flow (e.g. exec git status)

Since /tools/bin is on $PATH, the agent calls tools naturally. When the LLM runs exec git status:

1. LLM calls exec with args ["git", "status"]
2. Gateway receives the tool call, logs it as a core exec call
3. Gateway runs docker exec <container> sh -c "git status"
4. The shell finds /tools/bin/git on $PATH — a generated launcher script that:
   • Connects to /beige/gateway.sock (Unix socket mounted into container)
   • Sends: {"tool": "git", "args": ["status"]}
   • Waits for response
5. Gateway socket server receives the request:
   • Identifies agent from socket identity (each agent has its own socket)
   • Logs: {agent: "travel", tool: "git", args: ["status"], decision: "allowed"}
   • Checks policy (is “travel” agent allowed to use “git”?)
   • Executes the git tool handler on the gateway host
   • Returns result through socket
6. Launcher prints the result to stdout, exits
7. docker exec captures output, returns to gateway
8. Gateway returns exec result to LLM

Components

1. Config (config.json5)

Single config file (JSON5 — JSON with comments) drives the entire system. No defaults — everything is explicit.

{
  llm: {
    providers: {
      anthropic: { apiKey: "${ANTHROPIC_API_KEY}" },
      // zai: {
      //   baseUrl: "https://api.zai.com/v1",
      //   apiKey: "${ZAI_API_KEY}",
      //   api: "openai-completions",
      // },
    },
  },

  plugins: {
    git: {
      path: "./plugins/git",
    },
    // browser: {
    //   path: "./plugins/browser",
    //   config: { browserUrl: "ws://localhost:9222" },
    // },
  },

  agents: {
    travel: {
      model: {
        provider: "anthropic",
        model: "claude-sonnet-4-6",
        thinkingLevel: "medium",
      },
      fallbackModels: [
        { provider: "anthropic", model: "claude-3-5-sonnet-20241022" },
      ],
      tools: ["git"],
      // Per-agent plugin config overrides — deep-merged with plugins.git.config
      // pluginConfigs: {
      //   git: { allowForcePush: ["del"] },
      // },
      sandbox: { image: "beige-sandbox:latest" },
    },
  },
}

2. Gateway Core (src/gateway/)

• Config loader: Reads and validates config.json5, resolves env vars.
• Agent manager: Creates/destroys agent sessions. Maps session keys → pi SDK AgentSession + Docker container. Supports multiple concurrent sessions per agent (one per channel/chat/thread). Handles model fallback and rate limit tracking.
• Provider health tracker: Monitors per-provider rate limits and cooldowns. Persisted to ~/.beige/data/provider-health.json.
• Concurrency limiter: Per-provider concurrency control. When a provider has a concurrency limit configured, requests beyond that limit queue up (FIFO) until a slot frees up. Prevents overwhelming providers with too many parallel LLM calls.
• LLM proxy: The /api/chat/stream endpoint proxies LLM calls for the TUI. Resolves API keys server-side, executes prePrompt/postResponse hooks, applies fallback model logic on rate limits, and audit-logs every attempt. The TUI never needs API keys.
• Sandbox manager: Creates Docker containers with correct mounts, generates tool launchers, manages lifecycle.
• Socket server: One Unix domain socket per agent at ~/.beige/sockets/<agent>.sock, mounted into container at /beige/gateway.sock.
• Policy engine: Checks if agent is allowed to use a tool. Deny by default.
• Audit logger: Logs every tool invocation with agent, tool, args, decision, timing, result summary.

Model Fallback

When an agent has fallbackModels configured, the gateway uses a multi-pass retry loop:

1. Tries each model in order (primary first, then fallbacks)
2. If a model fails, moves to the next one — but keeps it eligible for retry on the next pass
3. Loops through the model list up to 3 passes before giving up
4. A model is removed from the retry pool when it either:
   • Has a rate-limit cooldown active (retryAfter in the future), OR
   • Has accumulated 3 consecutive failures in this prompt run
5. Returns the first successful response, or an error if all models are exhausted

This handles transient errors (e.g. Anthropic “overloaded”) gracefully: if model A fails once and model B also fails once, model A gets another shot on the next pass — the overload may have cleared by then. With 2 models, this means up to 6 total attempts before giving up.

Rate limit vs. transient error handling:

• Rate limits (HTTP 429, rate_limit / too many requests patterns): marked as globally cooling down — all sessions skip this model until the cooldown expires
• Transient capacity errors (overloaded, capacity, temporarily unavailable): NOT marked as globally cooling down — other sessions can still use the model. The failing session retries via the multi-pass loop.
• If a rate-limit response includes a retry-after header, that time is used for the cooldown
• Otherwise, hard rate limits (HTTP 429) default to 5-minute cooldown; soft rate limits (pattern-matched) default to 60 seconds
• Cooldown state is persisted in ~/.beige/data/provider-health.json and survives restarts
• A modelSwitched hook fires when a session ends up on a non-primary model, allowing plugins to send notifications

Provider Concurrency Limits

When multiple agents or sessions share a provider, the total number of parallel LLM requests can spike — especially with channels that handle many conversations simultaneously. The concurrency setting on a provider caps how many requests can be in-flight at once:

{
  llm: {
    providers: {
      zai: {
        apiKey: "...",
        concurrency: 5,  // max 5 parallel LLM calls to this provider
      },
    },
  },
}

• Requests beyond the limit are queued (FIFO) and await until a slot frees up — they are never rejected.
• Default: no limit (-1 / omitted). All requests pass through immediately.
• The limiter sits inside executePromptWithModel, so it applies to both prompt() and promptStreaming() calls uniformly.
• Concurrency is tracked per provider, not per model — all models on the same provider share the same pool.

This is complementary to the rate-limit fallback system: concurrency limits prevent overloading a provider in the first place, while fallback handles errors after they occur.

3. Core Tools (src/tools/core.ts)

Implemented as pi SDK ToolDefinition objects:

• read: Read a file from the sandbox. Gateway runs docker exec <container> cat <path>.
• write: Write content to a file in the sandbox. Gateway pipes content via docker exec.
• patch: Apply a find-and-replace patch to a file in the sandbox. Gateway reads, patches, writes back.
• exec: Execute a command in the sandbox. Gateway runs docker exec <container> <command>.

All four: log first, check policy, then execute.

4. Plugins (src/plugins/)

Each plugin is a directory:

plugins/git/
├── plugin.json        # manifest: name, description, what it provides
├── index.ts           # entry point (exports createPlugin)
├── README.md          # documentation (mounted for agent context)
└── SKILL.md           # agent usage guide

plugin.json:

{
  "name": "git",
  "description": "Git version control operations.",
  "commands": ["status", "commit", "log"],
  "provides": { "tools": ["git"] }
}

The gateway:

1. Loads all plugins from config (calls createPlugin() and register())
2. Plugins register tools, channels, hooks, and skills via PluginRegistrar
3. Validates that all agent tool references resolve to registered tools
4. Generates a launcher script per tool per agent
5. Mounts launchers read-only into /tools/bin/ (which is prepended to $PATH)
6. Mounts plugin packages read-only into /tools/packages/

At runtime, ToolRunner routes tool calls to the handler registered by the plugin that provides that tool.

5. Tool Launcher (generated, mounted into sandbox)

A small shell script that connects to the gateway socket:

#!/bin/sh
# Auto-generated by beige gateway. DO NOT EDIT.
# Tool: git | Target: gateway
exec /beige/tool-client "$TOOL_NAME" "$@"

Where /beige/tool-client is a small binary/script (mounted read-only) that:

• Connects to /beige/gateway.sock
• Sends JSON request with tool name + args
• Reads JSON response
• Prints result to stdout
• Exits with appropriate code

6. Socket Protocol

Request (sandbox → gateway):

{
  "type": "tool_request",
  "tool": "git",
  "args": ["set", "mykey", "myvalue"]
}

Response (gateway → sandbox):

{
  "type": "tool_response",
  "success": true,
  "output": "OK",
  "exitCode": 0
}

Error response:

{
  "type": "tool_response",
  "success": false,
  "error": "Permission denied: tool 'git' not allowed for agent 'travel'",
  "exitCode": 1
}

7. Channels

The TUI is the only built-in channel. Other channels (Telegram, Discord, etc.) are provided by plugins.

• TUI (built-in, separate process): Connects to gateway HTTP API. Runs pi’s InteractiveMode locally for the full pi experience. LLM calls are proxied through the gateway’s /api/chat/stream endpoint (the TUI never needs API keys). Tool execution is proxied through the gateway’s /api/agents/:name/exec endpoint. The gateway handles auth, hooks, fallback models, and audit logging for both paths.
• Plugin channels (in-process): Plugins can register channel adapters and run background processes (e.g. Telegram polling). They create sessions via PluginContext.prompt() and receive responses through the channel adapter.

Session Settings System

Sessions support layered settings with three levels of precedence (highest wins):

1. System default — hardcoded in the gateway
2. Plugin/channel config default — set in plugin config
3. Session override — set by user via commands, persisted in ~/.beige/sessions/session-settings.json

┌─────────────────────────────────────────────────────────────┐
│                    Setting Resolution                        │
├─────────────────────────────────────────────────────────────┤
│  System Default (false)                                     │
│       ↓ overridden by                                       │
│  Plugin/Channel Config Default (plugin config: defaults.verbose) │
│       ↓ overridden by                                       │
│  Session Override (user: /verbose on)                       │
└─────────────────────────────────────────────────────────────┘

Verbose Mode

When verbose mode is enabled, the channel receives a callback (onToolStart) before each tool execution and can notify the user:

Channel	Notification Method	Example
Telegram	Bot message in chat	🔧 exec: ls -la
TUI	stderr output	🔧 exec: ls -la (appears above TUI)

This gives users visibility into what the agent is doing without cluttering the main response.

Channel Commands

Commands are handled locally by the channel and not sent to the LLM:

Command	Telegram	TUI	Description
/start	✅	—	Welcome message + command list
/new	✅	—	Start fresh session
/status	✅	—	Show session info + settings
`/verbose on	off`	✅	✅	Toggle verbose mode
`/v on	off`	✅	✅	Shorthand for /verbose

On startup, the Telegram channel registers its commands with the bot API (deleting stale commands first).

8. Sandbox Docker Image

Minimal image with:

• Deno runtime (for TypeScript execution)
• Common utilities (curl, jq, etc.)
• /beige/tool-client — socket client script (baked into image, not mounted)
• No secrets, no env vars from host
• Mounts:
  • /workspace (read-write) → ~/.beige/agents/<name>/workspace/
  • /tools/bin (read-only) → generated launchers
  • /tools/packages (read-only) → tool source packages
  • /beige/gateway.sock (Unix socket)

9. Identity & Auth Model

• Each agent has its own Unix socket file. Gateway creates socket, mounts it.
• Gateway identifies agent by which socket received the connection.
• No payload-based identity. Agent cannot claim to be another agent.
• Threat model: even if an agent somehow accesses another agent’s socket file (impossible due to separate containers), the gateway still enforces per-socket identity.

10. Audit Log Format

JSONL file at ~/.beige/logs/audit.jsonl:

{
  "ts": "2026-03-05T00:00:00.000Z",
  "agent": "travel",
  "type": "core_tool",
  "tool": "exec",
  "args": ["curl", "https://example.com"],
  "decision": "allowed",
  "durationMs": 1234,
  "exitCode": 0,
  "outputBytes": 4567
}

{
  "ts": "2026-03-05T00:00:01.000Z",
  "agent": "travel",
  "type": "tool",
  "tool": "git",
  "args": ["set", "mykey", "myvalue"],
  "decision": "allowed",
  "target": "gateway",
  "durationMs": 12,
  "exitCode": 0,
  "outputBytes": 2
}

Tech Decisions

Decision	Choice	Rationale
LLM layer	pi SDK (@mariozechner/pi-coding-agent)	Handles providers (Anthropic, OpenAI/ZAI, etc.), streaming, sessions, auth. No need to reimplement.
Gateway runtime	Node.js + TypeScript	pi SDK is Node-native. GrammY is Node-native.
Sandbox runtime	Deno (inside Docker)	Native TS execution, no build step, secure-by-default permissions.
Container runtime	Docker	Works on Docker Desktop (Mac) and Linux.
Socket	Unix domain socket	Peer identity from connection (not payload). Simple, fast, no TCP overhead.
Config format	JSON5	JSON with comments. Human-readable, familiar syntax, env var interpolation.
Audit format	JSONL	Append-only, streamable, parseable.
Session settings	JSON file (session-settings.json)	Simple persistence for per-session overrides. Layered with channel defaults.
Build tool	tsx (dev) / tsc (build)	Fast dev iteration with tsx, standard tsc for production.
Install strategy	Lazy first-run setup (no postinstall)	postinstall runs on npm install in dev too and cannot distinguish --global. Lazy setup in src/install.ts fires only on first real command, skips entirely for source installs (detected via .git at package root).
npm package contents	dist/ + tools/ (via files in package.json)	Tool packages must ship with the npm package so beige setup can copy them to ~/.beige/tools/.