harrison/ein-ai-proxy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rewrite README.md with full documentation

Browse Source 
- Architecture flow diagram showing how requests are routed
- Complete routing table with cost model info
- Setup instructions (install, run, deploy as service)
- Caddy reverse proxy example with SSE flush_interval note
- Client configuration for Claude Code CLI and Roo Code
- API documentation (health check, messages endpoint)
- Important notes: thinking-block switching, MCP client-side, adding routes
- Updated file listing with all new context files
This commit is contained in:
William Stuckey
2026-03-26 18:26:08 -05:00
parent 3cfc7cffeb
commit e84f17efdb
1 changed files with 172 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+172 -32
View File
@@ -1,35 +1,130 @@
# ein-ai-proxy # ein-ai-proxy
Anthropic-compatible API proxy that routes requests to multiple AI providers based on model name. Built to run on a home server behind nginx, letting you use Claude Code and Roo Code from any machine while keeping costs optimized. A zero-dependency Node.js proxy that speaks the [Anthropic Messages API](https://docs.anthropic.com/en/api/messages) and routes requests to different AI providers based on the model name. Clients think they're talking to Anthropic — the proxy transparently rewrites model names and forwards to the correct backend.
## How it works Built to run on a home server behind a reverse proxy (Caddy), letting you use Claude Code CLI and Roo Code from anywhere while optimizing costs across providers.
Clients (Claude Code CLI, Roo Code, Cline, etc.) send standard Anthropic Messages API requests to your proxy. The proxy inspects the `model` field and forwards the request to the appropriate provider: ## How It Works
| Client requests | Routed to | Why | ```
|---|---|---| Client (Claude Code / Roo Code)
| `claude-opus-4-6` | Anthropic API | Complex tasks that need the best model | │
| `claude-sonnet-4-6` | Z.ai → GLM-4.7 | Daily workhorse, $10/mo flat rate | │ POST /v1/messages { "model": "claude-sonnet-4-6", ... }
| `claude-haiku-4-5-*` | DeepSeek → V3.2 | Trivial tasks, pennies per token | ▼
| `glm-4.7` (direct) | Z.ai | Explicit GLM request | ein-ai-proxy (port 3100)
| `deepseek-chat` (direct) | DeepSeek | Explicit DeepSeek request | │
├─ model starts with "claude-opus" → Anthropic API (unchanged)
├─ model starts with "claude-sonnet" → Z.ai API (rewritten to glm-4.7)
├─ model starts with "claude-haiku" → DeepSeek API (rewritten to deepseek-chat)
└─ unknown model → Z.ai API (default, rewritten to glm-4.7)
```
SSE streaming is fully supported — responses are piped through without buffering. The proxy:
1. Authenticates the client (Bearer token or x-api-key header)
2. Looks up the model name in a routing table
3. Rewrites the model name if needed (e.g., `claude-sonnet-4-6` → `glm-4.7`)
4. Forwards the request to the correct provider with that provider's API key
5. Pipes the response back — including SSE streaming, which is forwarded without buffering
## Routing Table
| Client Sends | Provider | Rewrites To | Cost Model |
|---|---|---|---|
| `claude-opus-4-6` | Anthropic (api.anthropic.com) | *unchanged* | Pay-per-token |
| `claude-sonnet-4-6` | Z.ai (api.z.ai) | `glm-4.7` | Flat rate subscription |
| `claude-sonnet-4-5` | Z.ai | `glm-4.7` | Flat rate subscription |
| `claude-haiku-4-5-*` | DeepSeek (api.deepseek.com) | `deepseek-chat` | Cheap per-token |
| `glm-4.7` | Z.ai | *unchanged* | Flat rate subscription |
| `deepseek-chat` | DeepSeek | *unchanged* | Cheap per-token |
| *anything else* | Z.ai | `glm-4.7` | Flat rate subscription |
Z.ai and DeepSeek both expose Anthropic-compatible APIs natively, so the proxy only needs to rewrite the model name and auth headers — no response translation required.
## Setup ## Setup
```bash ### Prerequisites
# 1. Copy and edit config
cp config.example.json config.json
nano config.json # Add your API keys + set authToken
# 2. Deploy (installs systemd service, tests proxy) - Node.js 18+ (the proxy uses zero npm dependencies — only Node.js built-ins)
- A reverse proxy with HTTPS (Caddy, nginx, etc.) if exposing to the internet
- API keys for whichever providers you want to use
### Install
```bash
git clone ssh://git@git.ein-softworks.com:3022/harrison/ein-ai-proxy.git
cd ein-ai-proxy
# Create config from template
cp config.example.json config.json
```
Edit `config.json` with your API keys:
```json
{
"port": 3100,
"authToken": "a-long-random-string",
"providers": {
"anthropic": { "apiKey": "sk-ant-..." },
"zai": { "apiKey": "your-zai-key" },
"deepseek": { "apiKey": "your-deepseek-key" }
}
}
```
### Run
```bash
# Direct
node proxy.mjs
# Via npm
npm start
# With auto-reload during development
npm run dev
```
### Deploy as a Service
The included `deploy.sh` script validates your config, tests the proxy, and installs a systemd service:
```bash
chmod +x deploy.sh chmod +x deploy.sh
./deploy.sh ./deploy.sh
# 3. Follow the nginx + SSL steps printed by deploy.sh
``` ```
This installs `ein-ai-proxy.service` into systemd and starts it. After deployment:
```bash
# Check status
sudo systemctl status ein-ai-proxy
# Restart after changes
sudo systemctl restart ein-ai-proxy
# View logs
sudo journalctl -u ein-ai-proxy -f --no-pager
```
### Reverse Proxy (Caddy)
The proxy binds to `0.0.0.0:3100`. To expose it over HTTPS, put it behind a reverse proxy. Example Caddy config:
```
ai.example.com {
reverse_proxy localhost:3100 {
flush_interval -1
transport http {
read_timeout 600s
write_timeout 600s
}
}
}
```
`flush_interval -1` is critical — it tells Caddy to flush SSE chunks immediately rather than buffering them. The 600s timeouts accommodate long reasoning responses.
## Client Configuration ## Client Configuration
### Claude Code CLI ### Claude Code CLI
@@ -45,27 +140,72 @@ Add to `~/.claude/settings.json`:
} }
``` ```
Claude Code sends model names like `claude-sonnet-4-6` by default, which the proxy routes to GLM-4.7. Use `/model` in Claude Code to switch to Opus when you need it.
### Roo Code (VS Code) ### Roo Code (VS Code)
1. Open Roo Code settings 1. Set API Provider to **Anthropic**
2. Set API Provider to **Anthropic** 2. Enter your proxy auth token as the API Key
3. Enter your proxy auth token as the API Key 3. Enable **Use custom base URL** and enter your proxy URL (e.g., `https://ai.ein-softworks.com`)
4. Check **Use custom base URL**
5. Enter `https://ai.ein-softworks.com`
## Switching models You can create multiple profiles to switch between providers easily:
- **default** → `claude-sonnet-4-5` → routes to GLM-4.7 (reasoning on)
- **opus** → `claude-opus-4-6` → routes to real Anthropic (reasoning on)
- **cheap** → `claude-haiku-4-5` → routes to DeepSeek (reasoning off)
From Claude Code, the default Sonnet model routes to GLM-4.7. To use Opus for a complex task, use `/model` and select Opus — the proxy will route it to the real Anthropic API. ## API
### Health Check
```bash
curl https://ai.ein-softworks.com/health
```
Returns the proxy status and current routing table. No authentication required.
### Messages Endpoint
```bash
curl -X POST https://ai.ein-softworks.com/v1/messages \
-H "Authorization: Bearer your-proxy-auth-token" \
-H "Content-Type: application/json" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "claude-sonnet-4-6",
"max_tokens": 1024,
"messages": [{"role": "user", "content": "Hello"}]
}'
```
Supports both `Authorization: Bearer <token>` and `x-api-key: <token>` authentication styles — compatible with any Anthropic API client.
## Important Notes
### Thinking-Block Provider Switching
**Do not switch between providers mid-conversation when reasoning/thinking is enabled.** GLM-4.7's thinking block format differs from Anthropic's and will be rejected if sent to the Anthropic API. Start a fresh session (`/clear` in Claude Code) before switching from Sonnet to Opus or vice versa.
### MCP Tools
MCP tools execute client-side (on whatever machine runs Claude Code or Roo Code), not on the proxy server. The proxy only routes LLM API calls.
### Adding Routes
To route additional models, add entries to the `MODEL_ROUTES` object in `proxy.mjs`. The proxy matches exact model names first, then tries prefix matching, then falls back to the default route (Z.ai / GLM-4.7).
## Files ## Files
- `proxy.mjs` — The proxy server (zero dependencies, Node.js 18+) | File | Description |
- `config.json` — Your API keys and auth token (gitignored) |---|---|
- `config.example.json` — Template config | `proxy.mjs` | Main proxy server — zero npm dependencies, Node.js 18+ built-ins only |
- `ein-ai-proxy.service` — Systemd unit file | `config.json` | API keys and auth token (**gitignored**, never commit) |
- `deploy.sh` — Deployment script | `config.example.json` | Template config — copy to `config.json` and fill in |
- `CLAUDE.md` — Project-level context for AI agents | `ein-ai-proxy.service` | Systemd unit file for running as a service |
- `CONTEXT.md` — Full infrastructure documentation | `deploy.sh` | Deployment script — validates config, tests proxy, installs service |
| `package.json` | Package metadata (no dependencies) |
| `CLAUDE.md` | Project-level context loaded by AI coding agents |
| `CONTEXT.md` | Full infrastructure documentation (network, servers, operations) |
| `global-claude-md/CLAUDE.md` | User-level context to copy to `~/.claude/CLAUDE.md` |
## License ## License