yanndegat.tngl.sh/drawgent at main · Tangled

drawgent connects your own Claude Code, Codex or opencode (your install, login, config and
repo) to an Excalidraw whiteboard. Ask for a diagram in the chat panel, or write AGENT: …
next to the part of a drawing you want changed. The agent looks at the canvas (screenshot +
scene), edits it live, checks the result, and marks the note DONE.

Quick start#

Prerequisite: one of claude, codex or opencode installed and logged in. The Claude and
Codex bridges also need Node.js ≥ 18 (npm).

drawgent setup claude        # once per agent: claude | codex | opencode
cd ~/my-repo
drawgent up                  # new agent session in this repo + canvas in your browser
drawgent up --attach         # or: pick one of your running sessions and connect the canvas to it

drawgent setup #

Checks everything once, fails with the exact fix when something is missing, and writes
~/.config/drawgent/config.toml:

1. The agent CLI. Your claude / codex / opencode on PATH.

2. Login. claude auth status, codex login status or opencode auth list.

3. ACP bridge.

   • opencode speaks ACP itself (opencode acp).
   • Claude Code and Codex use the official ACP adapters. They are installed once into
     ~/.cache/drawgent/adapters (~60 MB), without their bundled agent binaries, and pointed
     at your CLI (CLAUDE_CODE_EXECUTABLE, CODEX_PATH).
   • Setup then verifies the ACP handshake.

4. Canvas tools for attached sessions. Only Codex needs a change:
   codex mcp add drawgent -- drawgent mcp. Claude and opencode get the tools at attach time.

5. Headless Chrome for the renderer. Setup uses your Chrome/Chromium if you have one.
   Otherwise it proposes:

   • downloading Chrome Headless Shell (Chrome for Testing, ~120 MB, no sudo) into
     ~/.cache/drawgent/chrome, and telling you exactly which system libraries are missing, if
     any;
   • installing Chromium with your package manager (apt, snap, dnf, pacman, zypper,
     apk, brew, or nix without sudo).

   Non-interactive: --chrome download | system | /path/to/chrome.

drawgent up refuses to start until setup succeeded for that agent, or if something setup
recorded disappeared.

drawgent up#

Runs in the current directory (the workspace):

• starts the editor + API on 127.0.0.1:7300 (next free port if taken);
• starts a new session of your agent over ACP, working in the workspace;
• opens your browser. On a headless box it prints the ssh -L command instead.

The scene is kept in .drawgent/scene.json, which is git-ignored automatically.

drawgent up --attach [id]#

Connects the canvas to a session you already run. Without an id it lists the sessions it
finds (current directory first) and lets you pick one:

Using the canvas#

• Chat panel (right side): send a request, watch replies and tool calls stream in, approve
  permission prompts, press Stop to cancel a turn.
• On the canvas: write a text starting with AGENT: next to or inside a shape, or draw an
  arrow from the note to a shape. It fires about 2.5 s after you stop typing, with its
  position, what it points at and what is nearby. The agent resolves it into a green
  DONE: … note. Edit it back to AGENT: to send it again.
• Prompts are queued and run one at a time. A queued note that was already handled is skipped.

drawgent up --room 'https://excalidraw.com/#room=,'

• drawgent joins the room as a collaborator (🤖 Agent, with a cursor that follows its edits).
  Humans can stay on excalidraw.com: their AGENT: notes reach your agent, and its edits
  appear there live.
• Traffic is end-to-end encrypted with the room key. Empty rooms are loaded from and saved to
  excalidraw’s Firestore storage.
• The local editor mirrors the room and still has the chat panel.

Other commands#

• drawgent mcp: stdio MCP server with the canvas tools. Agents launch it; it finds the running
  drawgent up by itself.
• drawgent serve: low-level server with explicit agents (--agent claude,codex,opencode as set up,
  or name=command for any ACP agent); for scripts and containers.
• Options for up / serve: --port, --room, --token (API bearer + ?token= in the URL),
  --permissions canvas|ask|all (default canvas: drawing tools auto-approved, anything else
  asked in the chat panel), --data, --settle-ms.

Optional: Docker#

docker compose up --build runs a canvas server (drawgent + Chromium, no agents), e.g. to
host a shared canvas or a room bridge on a server. Agents are never bundled: they always run
with your own setup.

Build from source#

npm ci && npm run build      # editor + renderer pages -> dist/ (embedded into the binary)
cargo install --path .       # or: cargo build --release

get_scene, get_screenshot (vision; zoom with element_ids), add_elements (Excalidraw
skeletons; arrows bind by id and are routed edge-to-edge), add_mermaid (auto-layout),
update_elements (labels follow shapes, bound arrows re-route), delete_elements,
clear_canvas, list_instructions, resolve_instruction, set_status.

API#

GET /api/health · GET /api/scene · GET /api/screenshot?ids=&padding=&max= ·
POST|PATCH|DELETE /api/elements · POST /api/mermaid · POST /api/clear ·
GET /api/instructions · POST /api/instructions/🔥/resolve · POST /api/status ·
POST /api/chat ⚡ · WS /ws (browser sync + chat events)

Tests#

cargo test                          # fractional indices, room crypto/framing
node scripts/smoke.mjs [url]        # chat turn + AGENT: note against a running drawgent
node scripts/e2e-browser.mjs [url]  # real browser: chat panel + note typed on the canvas
node scripts/room-e2e.mjs           # fresh excalidraw.com room ↔ drawgent, both directions

Layout#

src/ (Rust):

• main.rs: CLI.
• setup.rs, config.rs, chrome.rs: setup, config, renderer install.
• attach.rs: session discovery and picker.
• agents.rs + acp.rs: ACP driver (new / fork sessions).
• live.rs: live opencode / Codex drivers.
• hub.rs: routing, notes, chat log.
• scene.rs: store and edit operations.
• renderer.rs: Chrome over CDP.
• mcp.rs: MCP server.
• room.rs: excalidraw.com client.
• fractional.rs, geometry.rs, el.rs: helpers.

web/: editor (main.jsx, chat.jsx) and renderer page (render.jsx).

Limits#

• The renderer needs Chrome (a native renderer is planned).
• Claude “attach” is a fork, because Claude Code has no public way to inject into a running
  terminal session.
• Codex live attach is implemented, but was not yet tested against a logged-in Codex.
• One scene per workspace. Images/files are not synced.