harrison/ein-ai-proxy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
c8777b45ffeb310002cacb10d0fd4443180154ab
ein-ai-proxy/README.md
T

11 KiB
Raw Blame History

ein-ai-proxy

A zero-dependency Node.js proxy that speaks the Anthropic Messages API and routes requests to different AI providers based on the model name. Clients think they're talking to Anthropic — the proxy transparently rewrites model names and forwards to the correct backend.

Built to run on a home server behind a reverse proxy (Caddy), letting you use Claude Code CLI and Roo Code from anywhere while optimizing costs across providers.

How It Works

  Client (Claude Code / Roo Code)
         │
         │  POST /v1/messages  { "model": "claude-sonnet-4-6", ... }
         ▼
  ein-ai-proxy (port 3100)
         │
         ├─ model starts with "claude-opus"    → Anthropic API (unchanged)
         ├─ model starts with "claude-sonnet"  → Z.ai API (rewritten to glm-4.7)
         ├─ model starts with "claude-haiku"   → DeepSeek API (rewritten to deepseek-chat)
         └─ unknown model                      → Z.ai API (default, rewritten to glm-4.7)

The proxy:

  1. Authenticates the client (Bearer token or x-api-key header)
  2. Looks up the model name in a routing table
  3. Rewrites the model name if needed (e.g., claude-sonnet-4-6 → glm-4.7)
  4. Forwards the request to the correct provider with that provider's API key
  5. Pipes the response back — including SSE streaming, which is forwarded without buffering

Routing Table

Client Sends Provider Rewrites To Cost Model
claude-opus-* Anthropic (api.anthropic.com) unchanged Pay-per-token
claude-sonnet-* Z.ai (api.z.ai) glm-4.7 Flat rate subscription
claude-haiku-* DeepSeek (api.deepseek.com) deepseek-chat Cheap per-token
glm-4.7 Z.ai unchanged Flat rate subscription
deepseek-chat DeepSeek unchanged Cheap per-token
anything else Z.ai glm-4.7 Flat rate subscription

The proxy matches exact model names first, then tries prefix matching (so dated variants like claude-sonnet-4-5-20250929 are handled automatically). See CONTEXT.md for the full routing table with every explicit entry.

Z.ai and DeepSeek both expose Anthropic-compatible APIs natively, so the proxy only needs to rewrite the model name and auth headers — no response translation required.

Setup

Prerequisites

  • Node.js 18+ (the proxy uses zero npm dependencies — only Node.js built-ins)
  • A reverse proxy with HTTPS (Caddy, nginx, etc.) if exposing to the internet
  • API keys for whichever providers you want to use

Install

git clone ssh://git@git.ein-softworks.com:3022/harrison/ein-ai-proxy.git
cd ein-ai-proxy

# Create config from template
cp config.example.json config.json

Edit config.json with your API keys and pricing:

{
  "port": 3100,
  "authToken": "a-long-random-string",
  "providers": {
    "anthropic": { "apiKey": "sk-ant-..." },
    "zai":       { "apiKey": "your-zai-key" },
    "deepseek":  { "apiKey": "your-deepseek-key" }
  },
  "pricing": {
    "anthropic": {
      "inputPerMTok": 15,
      "outputPerMTok": 75
    },
    "deepseek": {
      "inputPerMTok": 0.27,
      "outputPerMTok": 1.10
    },
    "zai": {
      "type": "flat",
      "monthlyCostUsd": 10
    }
  }
}

Pricing notes: Rates are per million tokens (input/output). Update these values based on current provider pricing. Z.ai uses a flat-rate model ($10/mo) regardless of token usage.

Run

# Direct
node proxy.mjs

# Via npm
npm start

# With auto-reload during development
npm run dev

Deploy as a Service

On mini-server (where the proxy runs), the deploy.sh script validates your config, tests the proxy, and installs a systemd service:

# SSH into mini-server first
ssh mini-server
cd ~/ai-proxy

chmod +x deploy.sh
./deploy.sh

This installs ein-ai-proxy.service into systemd and starts it. After deployment:

# Check status
sudo systemctl status ein-ai-proxy

# Restart after changes
sudo systemctl restart ein-ai-proxy

# View logs
sudo journalctl -u ein-ai-proxy -f --no-pager

Reverse Proxy (Caddy)

The proxy binds to 0.0.0.0:3100. To expose it over HTTPS, put it behind a reverse proxy. Example Caddy config:

ai.example.com {
    reverse_proxy localhost:3100 {
        flush_interval -1
        transport http {
            read_timeout 600s
            write_timeout 600s
        }
    }
}

flush_interval -1 is critical — it tells Caddy to flush SSE chunks immediately rather than buffering them. The 600s timeouts accommodate long reasoning responses.

Client Configuration

Finding Your Auth Token

The proxy auth token is the authToken value in config.json on the server where the proxy runs. To retrieve it:

ssh mini-server "node -e \"console.log(JSON.parse(require('fs').readFileSync('/home/harrison/ai-proxy/config.json','utf-8')).authToken)\""

Or just SSH in and read the file directly:

ssh mini-server
cat ~/ai-proxy/config.json | grep authToken

This is the token you'll use in all client configurations below.

Claude Code CLI

Add to ~/.claude/settings.json:

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://ai.ein-softworks.com",
    "ANTHROPIC_AUTH_TOKEN": "your-proxy-auth-token",
    "API_TIMEOUT_MS": "3000000"
  }
}

Claude Code sends model names like claude-sonnet-4-6 by default, which the proxy routes to GLM-4.7. Use /model in Claude Code to switch to Opus when you need it.

Roo Code (VS Code)

  1. Set API Provider to Anthropic
  2. Enter your proxy auth token as the API Key
  3. Enable Use custom base URL and enter your proxy URL (e.g., https://ai.ein-softworks.com)

You can create multiple profiles to switch between providers easily:

  • default → claude-sonnet-4-5 → routes to GLM-4.7 (reasoning on)
  • opus → claude-opus-4-6 → routes to real Anthropic (reasoning on)
  • cheap → claude-haiku-4-5 → routes to DeepSeek (reasoning off)

API

Health Check

curl https://ai.ein-softworks.com/health

Returns the proxy status and current routing table. No authentication required.

Usage Statistics

curl https://ai.ein-softworks.com/usage

Returns comprehensive usage data grouped by provider:

  • Per-model breakdown: Requested model name, actual provider model, total + monthly token counts and request counts
  • Cost calculations: Dollar amounts for each model (based on pricing in config.json)
  • Live balances: Account balances from providers that support it (DeepSeek)
  • Totals: Aggregated request counts and costs across all models

Example response:

{
  "service": "ein-ai-proxy",
  "generatedAt": "2026-03-27T00:38:00.000Z",
  "currentPeriod": "2026-03",
  "providers": {
    "anthropic": {
      "balance": null,
      "pricingNote": "No balance API available — check console.anthropic.com",
      "models": {
        "claude-opus-4-6": {
          "routesTo": "claude-opus-4-6",
          "total": { "inputTokens": 150000, "outputTokens": 50000, "requests": 42, "costUsd": 6.00 },
          "monthly": { "inputTokens": 30000, "outputTokens": 10000, "requests": 8, "costUsd": 1.20 }
        }
      }
    },
    "deepseek": {
      "balance": { "totalBalance": "8.52", "currency": "USD" },
      "models": { ... }
    },
    "zai": {
      "balance": null,
      "pricing": "flat_rate",
      "monthlyCostUsd": 10,
      "models": { ... }
    }
  },
  "totals": {
    "totalRequests": 252,
    "monthlyRequests": 53,
    "totalCostUsd": 6.14,
    "monthlyCostUsd": 1.24,
    "note": "Cost excludes flat-rate models. Z.ai costs $10/mo regardless of usage."
  }
}

Note: Monthly totals reset on the 1st of each month. Usage data is persisted to usage-data.json in the proxy directory.

Messages Endpoint

curl -X POST https://ai.ein-softworks.com/v1/messages \
  -H "Authorization: Bearer your-proxy-auth-token" \
  -H "Content-Type: application/json" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-sonnet-4-6",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Supports both Authorization: Bearer <token> and x-api-key: <token> authentication styles — compatible with any Anthropic API client.

Important Notes

Thinking-Block Provider Switching

Do not switch between providers mid-conversation when reasoning/thinking is enabled. GLM-4.7's thinking block format differs from Anthropic's and will be rejected if sent to the Anthropic API. Start a fresh session (/clear in Claude Code) before switching from Sonnet to Opus or vice versa.

MCP Tools

MCP tools execute client-side (on whatever machine runs Claude Code or Roo Code), not on the proxy server. The proxy only routes LLM API calls.

Adding Routes

To route additional models, add entries to the MODEL_ROUTES object in proxy.mjs. The proxy matches exact model names first, then tries prefix matching, then falls back to the default route (Z.ai / GLM-4.7).

AI Context Files

This repo includes context files for both Claude Code CLI and Roo Code (VS Code). The two tools use different file conventions, so we keep them in sync:

Project-level context

Claude Code reads Roo Code reads Source of truth
CLAUDE.md .clinerules CLAUDE.md
CONTEXT.md CONTEXT.md CONTEXT.md

.clinerules is a git-tracked symlink pointing to CLAUDE.md. Both tools read the same content from the same source — edit CLAUDE.md and both stay in sync automatically.

CONTEXT.md is the detailed infrastructure reference. Both tools can read it directly when needed.

Global context

Claude Code Roo Code
~/.claude/CLAUDE.md (file on disk) Custom Instructions setting (in Roo Code UI)

The source of truth is global-claude-md/CLAUDE.md in this repo. To update both tools, run the sync script:

# 1. Edit the source
vim global-claude-md/CLAUDE.md

# 2. Commit and push
git add -A && git commit -m "Update global context" && git push

# 3. Sync to both tools
./sync-context.sh

The script copies the file to ~/.claude/CLAUDE.md (for Claude Code) and prints the content for you to paste into Roo Code's Custom Instructions (Roo sidebar → gear icon → Custom Instructions).

To also update mini-server's Claude Code:

scp global-claude-md/CLAUDE.md mini-server:~/.claude/CLAUDE.md

Files

File Description
proxy.mjs Main proxy server — zero npm dependencies, Node.js 18+ built-ins only
config.json API keys and auth token (gitignored, never commit)
config.example.json Template config — copy to config.json and fill in
.gitignore Excludes config.json, node_modules/, and *.log
ein-ai-proxy.service Systemd unit file for running as a service
deploy.sh Deployment script — validates config, tests proxy, installs service
sync-context.sh Syncs global AI context to Claude Code and Roo Code
package.json Package metadata (no dependencies)
CLAUDE.md Project-level context for Claude Code
.clinerules Symlink → CLAUDE.md — project-level context for Roo Code
CONTEXT.md Full infrastructure documentation (network, servers, operations)
global-claude-md/CLAUDE.md Source of truth for global context (both tools)

License

MIT

Reference in New Issue View Git Blame Copy Permalink