From e84f17efdb7abf158eeb653a26196fd76a009069 Mon Sep 17 00:00:00 2001 From: William Stuckey Date: Thu, 26 Mar 2026 18:26:08 -0500 Subject: [PATCH] Rewrite README.md with full documentation - Architecture flow diagram showing how requests are routed - Complete routing table with cost model info - Setup instructions (install, run, deploy as service) - Caddy reverse proxy example with SSE flush_interval note - Client configuration for Claude Code CLI and Roo Code - API documentation (health check, messages endpoint) - Important notes: thinking-block switching, MCP client-side, adding routes - Updated file listing with all new context files --- README.md | 204 +++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 172 insertions(+), 32 deletions(-) diff --git a/README.md b/README.md index 1364a56..b3f3b32 100644 --- a/README.md +++ b/README.md @@ -1,35 +1,130 @@ # ein-ai-proxy -Anthropic-compatible API proxy that routes requests to multiple AI providers based on model name. Built to run on a home server behind nginx, letting you use Claude Code and Roo Code from any machine while keeping costs optimized. +A zero-dependency Node.js proxy that speaks the [Anthropic Messages API](https://docs.anthropic.com/en/api/messages) and routes requests to different AI providers based on the model name. Clients think they're talking to Anthropic — the proxy transparently rewrites model names and forwards to the correct backend. -## How it works +Built to run on a home server behind a reverse proxy (Caddy), letting you use Claude Code CLI and Roo Code from anywhere while optimizing costs across providers. -Clients (Claude Code CLI, Roo Code, Cline, etc.) send standard Anthropic Messages API requests to your proxy. The proxy inspects the `model` field and forwards the request to the appropriate provider: +## How It Works -| Client requests | Routed to | Why | -|---|---|---| -| `claude-opus-4-6` | Anthropic API | Complex tasks that need the best model | -| `claude-sonnet-4-6` | Z.ai → GLM-4.7 | Daily workhorse, $10/mo flat rate | -| `claude-haiku-4-5-*` | DeepSeek → V3.2 | Trivial tasks, pennies per token | -| `glm-4.7` (direct) | Z.ai | Explicit GLM request | -| `deepseek-chat` (direct) | DeepSeek | Explicit DeepSeek request | +``` + Client (Claude Code / Roo Code) + │ + │ POST /v1/messages { "model": "claude-sonnet-4-6", ... } + ▼ + ein-ai-proxy (port 3100) + │ + ├─ model starts with "claude-opus" → Anthropic API (unchanged) + ├─ model starts with "claude-sonnet" → Z.ai API (rewritten to glm-4.7) + ├─ model starts with "claude-haiku" → DeepSeek API (rewritten to deepseek-chat) + └─ unknown model → Z.ai API (default, rewritten to glm-4.7) +``` -SSE streaming is fully supported — responses are piped through without buffering. +The proxy: +1. Authenticates the client (Bearer token or x-api-key header) +2. Looks up the model name in a routing table +3. Rewrites the model name if needed (e.g., `claude-sonnet-4-6` → `glm-4.7`) +4. Forwards the request to the correct provider with that provider's API key +5. Pipes the response back — including SSE streaming, which is forwarded without buffering + +## Routing Table + +| Client Sends | Provider | Rewrites To | Cost Model | +|---|---|---|---| +| `claude-opus-4-6` | Anthropic (api.anthropic.com) | *unchanged* | Pay-per-token | +| `claude-sonnet-4-6` | Z.ai (api.z.ai) | `glm-4.7` | Flat rate subscription | +| `claude-sonnet-4-5` | Z.ai | `glm-4.7` | Flat rate subscription | +| `claude-haiku-4-5-*` | DeepSeek (api.deepseek.com) | `deepseek-chat` | Cheap per-token | +| `glm-4.7` | Z.ai | *unchanged* | Flat rate subscription | +| `deepseek-chat` | DeepSeek | *unchanged* | Cheap per-token | +| *anything else* | Z.ai | `glm-4.7` | Flat rate subscription | + +Z.ai and DeepSeek both expose Anthropic-compatible APIs natively, so the proxy only needs to rewrite the model name and auth headers — no response translation required. ## Setup -```bash -# 1. Copy and edit config -cp config.example.json config.json -nano config.json # Add your API keys + set authToken +### Prerequisites -# 2. Deploy (installs systemd service, tests proxy) +- Node.js 18+ (the proxy uses zero npm dependencies — only Node.js built-ins) +- A reverse proxy with HTTPS (Caddy, nginx, etc.) if exposing to the internet +- API keys for whichever providers you want to use + +### Install + +```bash +git clone ssh://git@git.ein-softworks.com:3022/harrison/ein-ai-proxy.git +cd ein-ai-proxy + +# Create config from template +cp config.example.json config.json +``` + +Edit `config.json` with your API keys: + +```json +{ + "port": 3100, + "authToken": "a-long-random-string", + "providers": { + "anthropic": { "apiKey": "sk-ant-..." }, + "zai": { "apiKey": "your-zai-key" }, + "deepseek": { "apiKey": "your-deepseek-key" } + } +} +``` + +### Run + +```bash +# Direct +node proxy.mjs + +# Via npm +npm start + +# With auto-reload during development +npm run dev +``` + +### Deploy as a Service + +The included `deploy.sh` script validates your config, tests the proxy, and installs a systemd service: + +```bash chmod +x deploy.sh ./deploy.sh - -# 3. Follow the nginx + SSL steps printed by deploy.sh ``` +This installs `ein-ai-proxy.service` into systemd and starts it. After deployment: + +```bash +# Check status +sudo systemctl status ein-ai-proxy + +# Restart after changes +sudo systemctl restart ein-ai-proxy + +# View logs +sudo journalctl -u ein-ai-proxy -f --no-pager +``` + +### Reverse Proxy (Caddy) + +The proxy binds to `0.0.0.0:3100`. To expose it over HTTPS, put it behind a reverse proxy. Example Caddy config: + +``` +ai.example.com { + reverse_proxy localhost:3100 { + flush_interval -1 + transport http { + read_timeout 600s + write_timeout 600s + } + } +} +``` + +`flush_interval -1` is critical — it tells Caddy to flush SSE chunks immediately rather than buffering them. The 600s timeouts accommodate long reasoning responses. + ## Client Configuration ### Claude Code CLI @@ -45,27 +140,72 @@ Add to `~/.claude/settings.json`: } ``` +Claude Code sends model names like `claude-sonnet-4-6` by default, which the proxy routes to GLM-4.7. Use `/model` in Claude Code to switch to Opus when you need it. + ### Roo Code (VS Code) -1. Open Roo Code settings -2. Set API Provider to **Anthropic** -3. Enter your proxy auth token as the API Key -4. Check **Use custom base URL** -5. Enter `https://ai.ein-softworks.com` +1. Set API Provider to **Anthropic** +2. Enter your proxy auth token as the API Key +3. Enable **Use custom base URL** and enter your proxy URL (e.g., `https://ai.ein-softworks.com`) -## Switching models +You can create multiple profiles to switch between providers easily: +- **default** → `claude-sonnet-4-5` → routes to GLM-4.7 (reasoning on) +- **opus** → `claude-opus-4-6` → routes to real Anthropic (reasoning on) +- **cheap** → `claude-haiku-4-5` → routes to DeepSeek (reasoning off) -From Claude Code, the default Sonnet model routes to GLM-4.7. To use Opus for a complex task, use `/model` and select Opus — the proxy will route it to the real Anthropic API. +## API + +### Health Check + +```bash +curl https://ai.ein-softworks.com/health +``` + +Returns the proxy status and current routing table. No authentication required. + +### Messages Endpoint + +```bash +curl -X POST https://ai.ein-softworks.com/v1/messages \ + -H "Authorization: Bearer your-proxy-auth-token" \ + -H "Content-Type: application/json" \ + -H "anthropic-version: 2023-06-01" \ + -d '{ + "model": "claude-sonnet-4-6", + "max_tokens": 1024, + "messages": [{"role": "user", "content": "Hello"}] + }' +``` + +Supports both `Authorization: Bearer ` and `x-api-key: ` authentication styles — compatible with any Anthropic API client. + +## Important Notes + +### Thinking-Block Provider Switching + +**Do not switch between providers mid-conversation when reasoning/thinking is enabled.** GLM-4.7's thinking block format differs from Anthropic's and will be rejected if sent to the Anthropic API. Start a fresh session (`/clear` in Claude Code) before switching from Sonnet to Opus or vice versa. + +### MCP Tools + +MCP tools execute client-side (on whatever machine runs Claude Code or Roo Code), not on the proxy server. The proxy only routes LLM API calls. + +### Adding Routes + +To route additional models, add entries to the `MODEL_ROUTES` object in `proxy.mjs`. The proxy matches exact model names first, then tries prefix matching, then falls back to the default route (Z.ai / GLM-4.7). ## Files -- `proxy.mjs` — The proxy server (zero dependencies, Node.js 18+) -- `config.json` — Your API keys and auth token (gitignored) -- `config.example.json` — Template config -- `ein-ai-proxy.service` — Systemd unit file -- `deploy.sh` — Deployment script -- `CLAUDE.md` — Project-level context for AI agents -- `CONTEXT.md` — Full infrastructure documentation +| File | Description | +|---|---| +| `proxy.mjs` | Main proxy server — zero npm dependencies, Node.js 18+ built-ins only | +| `config.json` | API keys and auth token (**gitignored**, never commit) | +| `config.example.json` | Template config — copy to `config.json` and fill in | +| `ein-ai-proxy.service` | Systemd unit file for running as a service | +| `deploy.sh` | Deployment script — validates config, tests proxy, installs service | +| `package.json` | Package metadata (no dependencies) | +| `CLAUDE.md` | Project-level context loaded by AI coding agents | +| `CONTEXT.md` | Full infrastructure documentation (network, servers, operations) | +| `global-claude-md/CLAUDE.md` | User-level context to copy to `~/.claude/CLAUDE.md` | ## License