harrison/ein-ai-proxy
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
ein-ai-proxy/CONTEXT.md
T

13 KiB
Raw Permalink Blame History

Ein AI Proxy - Full System Context

This document provides complete context for working on the ein-ai-proxy project and the surrounding infrastructure.

Architecture Diagram

                    Work Laptop (u229331)
                    ┌────────────────────┐
                    │  Claude Code CLI   │
                    │   OR Roo Code      │
                    └────────┬───────────┘
                             │ HTTPS (SSE streaming)
                             │ ANTHROPIC_BASE_URL=
                             │ https://ai.ein-softworks.com
                             ▼
                    ┌─────────────────────────────────────────────┐
                    │              Internet                       │
                    │    DNS: ai.ein-softworks.com → 198.98.201.213│
                    └─────────────────────────────────────────────┘
                             │
                             ▼
                    ┌────────────────────┐
                    │   big-server       │  Gateway (198.98.201.213)
                    │   192.168.1.61     │  ┌──────────────────────┐
                    │   Caddy (Docker)   │  │ HTTPS (443) routing: │
                    │   iptables rules   │  │ ai.ein-softworks.com │
                    │                    │  │ → 192.168.1.140:3100 │
                    │   TCP Forwarding:  │  └──────────────────────┘
                    │   41922 → mini:22   │
                    │   3022 → mini:3022 │
                    └─────────┬──────────┘
                              │ Internal LAN (192.168.1.0/24)
                              ▼
                    ┌────────────────────┐
                    │   mini-server      │  Home AI Server
                    │   192.168.1.140    │  ┌──────────────────────┐
                    │                    │  │ ein-ai-proxy (3100) │
                    │   Services:        │  │ systemd: ein-ai-     │
                    │   - ein-ai-proxy   │  │       proxy          │
                    │   - Gitea (3000)   │  │ source: ~/ai-proxy/ │
                    │   - Caddy (local)  │  └──────────────────────┘
                    └─────────┬──────────┘
                              │ Routes by model name:
         ┌────────────────────┼────────────────────┬──────────────────┐
         │                    │                    │                  │
         ▼                    ▼                    ▼                  ▼
   ┌──────────┐        ┌────────────┐      ┌───────────────┐   ┌────────────┐
   │Anthropic │        │   Z.ai     │      │  DeepSeek     │   │  Default   │
   │   API    │        │ GLM-4.7    │      │   V3.2        │   │  (fallback)│
   └──────────┘        └────────────┘      └───────────────┘   └────────────┘
   (pay-per-t)        (flat $10/mo)      (cheap per token)   (GLM-4.7)

Model Routing Table

Requested Model Provider Rewritten To Auth Style Use Case
claude-opus-4-6 Anthropic unchanged x-api-key Complex tasks
claude-opus-4-5 Anthropic unchanged x-api-key Complex tasks
claude-opus-4-5-20251101 Anthropic unchanged x-api-key Complex tasks
claude-sonnet-4-6 Z.ai glm-4.7 Bearer Daily workhorse
claude-sonnet-4-5 Z.ai glm-4.7 Bearer Daily workhorse
claude-sonnet-4-5-20250929 Z.ai glm-4.7 Bearer Daily workhorse
claude-haiku-4-5-20251001 DeepSeek deepseek-chat Bearer Trivial tasks
claude-haiku-4-5 DeepSeek deepseek-chat Bearer Trivial tasks
glm-4.7 Z.ai unchanged Bearer Direct GLM request
glm-4.5-air Z.ai unchanged Bearer Direct GLM request
deepseek-chat DeepSeek unchanged Bearer Direct DeepSeek
unknown Z.ai glm-4.7 Bearer Default fallback

Network Architecture

DNS Records

All resolve to 198.98.201.213 (big-server's public IP):

  • ai.ein-softworks.com — Direct A record
  • git.ein-softworks.com — CNAME → maybezone.duckdns.org
  • ssh.ein-softworks.com — CNAME → maybezone.duckdns.org
  • ssh.maybe.zone — CNAME → maybezone.duckdns.org

Servers

big-server (192.168.1.61)

  • SSH: ssh big-server (host: ssh.ein-softworks.com, port: 30141, user: harrison, key: ~/.ssh/id_personal)
  • OS: Linux
  • Caddy: Docker container with network_mode: host
    • Config inside container: /etc/caddy/Caddyfile
    • Edit: sudo su, cd /opt/reverse-proxy, edit Caddyfile, docker restart caddy
    • Logs: docker logs caddy --tail 20
  • SSE streaming config: flush_interval -1, timeouts 600s
  • TCP forwarding via iptables:
    • Port 41922 → mini-server:22 (SSH)
    • Port 3022 → 192.168.1.140:3022 (Gitea SSH)
    • Port 30141 → big-server SSH directly

mini-server (192.168.1.140)

  • SSH: ssh mini-server (host: ssh.maybe.zone, port: 41922, user: harrison, key: ~/.ssh/id_personal)
  • OS: Debian Linux, zsh shell
  • Node: v24 via nvm (scripts must source $NVM_DIR/nvm.sh before using npm/node)
  • Docker: Requires sudo; user harrison does not have passwordless sudo
  • Services (Docker):
    • Gitea: /opt/gitea/docker-compose.yml, ports 3000:3000 (web), 3022:2222 (SSH)
    • Caddy: /opt/reverse-proxy/docker-compose.yml, network_mode: host (local reverse proxy)
  • AI proxy:
    • Systemd unit: ein-ai-proxy
    • Source: ~/ai-proxy/
    • Port: 3100, bound 0.0.0.0
    • Config: ~/ai-proxy/config.json (gitignored, contains API keys and auth token)
  • Gitea: https://git.ein-softworks.com
  • Shell aliases (~/.zshrc.local):
    • ai → GLM-4.7 (Z.ai)
    • ai-opus → Claude Opus (Anthropic)
    • ai-deep → DeepSeek V3.2
    • These set env vars per-provider and call claude directly (no proxy needed locally)
  • Dotfiles: Public on GitHub; secrets only in ~/.zshrc.local (sourced via [[ -f ~/.zshrc.local ]] && source ~/.zshrc.local)

Traffic Routing

HTTPS (ports 80/443) — Handled by Caddy on big-server:

  • git.ein-softworks.com → 192.168.1.140:3000 (Gitea web UI)
  • ai.ein-softworks.com → 192.168.1.140:3100 (AI proxy)
  • maybe.zone / matrix.maybe.zone / etc. → local Matrix services
  • Media stack, UniFi Controller, VPN container → local services

TCP port forwarding — Handled by iptables on big-server:

  • Port 41922 → mini-server:22 (SSH)
  • Port 3022 → 192.168.1.140:3022 (Gitea SSH)

Work Laptop Configuration

OS: macOS User: u229331

SSH Config

Host big-server
  HostName ssh.ein-softworks.com
  Port 30141
  User harrison
  IdentityFile ~/.ssh/id_personal

Host mini-server
  HostName ssh.maybe.zone
  Port 41922
  User harrison
  IdentityFile ~/.ssh/id_personal

Claude Code CLI

  • Location: ~/.local/bin/claude
  • Config: ~/.claude/settings.json
  • Environment variables:

Roo Code (VS Code Extension)

Three profiles configured:

Profile Model Routes To Reasoning Context
"default" claude-sonnet-4-5 GLM-4.7 (Z.ai) ON Standard
"opus" claude-opus-4-6 Anthropic Opus ON 1M context enabled
"cheap" claude-haiku-4-5 DeepSeek V3.2 OFF Fast/cheap tasks

AI Context File Mapping

Both Claude Code and Roo Code use context files, but with different conventions:

Level Claude Code Roo Code Source of truth
Project CLAUDE.md .clinerules (symlink → CLAUDE.md) CLAUDE.md
Global ~/.claude/CLAUDE.md Custom Instructions in Roo UI global-claude-md/CLAUDE.md

Run ./sync-context.sh from the repo root to sync global context to both tools.

Proxy Internals

File Structure

~/ai-proxy/
├── proxy.mjs              # Main proxy server (zero dependencies)
├── config.json            # API keys + auth token (gitignored)
├── config.example.json    # Template config
├── ein-ai-proxy.service   # Systemd unit file
├── deploy.sh              # Deployment script
├── sync-context.sh        # Syncs global context to Claude Code & Roo Code
├── package.json           # Package metadata
├── .gitignore             # Excludes config.json
├── CONTEXT.md             # This file
├── CLAUDE.md              # Project-level context (Claude Code)
├── .clinerules            # Symlink → CLAUDE.md (Roo Code)
├── global-claude-md/      # Global context source of truth
│   └── CLAUDE.md          # Synced to ~/.claude/ and Roo Custom Instructions
└── README.md              # Public documentation

Provider Configuration

const PROVIDERS = {
  anthropic: {
    baseUrl: "https://api.anthropic.com",
    authStyle: "api-key"        // x-api-key header
  },
  zai: {
    baseUrl: "https://api.z.ai/api/anthropic",
    authStyle: "bearer"         // Authorization: Bearer token
  },
  deepseek: {
    baseUrl: "https://api.deepseek.com/anthropic",
    authStyle: "bearer"         // Authorization: Bearer token
  }
};

SSE Streaming

  • Streaming responses (stream: true) are piped through without buffering
  • Caddy on big-server uses flush_interval -1 for immediate SSE delivery
  • Timeouts set to 600s to accommodate long reasoning responses

Authentication

Proxy authenticates clients via:

  1. Authorization: Bearer <token> header, OR
  2. x-api-key: <token> header

Token must match config.json.authToken.

Common Operations

On mini-server (where proxy runs)

# Restart proxy
sudo systemctl restart ein-ai-proxy

# View live logs
sudo journalctl -u ein-ai-proxy -f --no-pager

# Check proxy health
curl https://ai.ein-softworks.com/health

# Check usage statistics
curl https://ai.ein-softworks.com/usage

# Pull and deploy changes
cd ~/ai-proxy && git pull
sudo systemctl restart ein-ai-proxy

On big-server (where Caddy runs)

# Restart Caddy
sudo su
cd /opt/reverse-proxy
docker restart caddy

# View Caddy logs
docker logs caddy --tail 20

Important Limitations

MCP Tools Execute Client-Side

MCP tools run on the work laptop (where the AI client is), NOT on the proxy server. This means:

  • Company-internal MCP servers work fine because the client is on the company network
  • The proxy only routes LLM API calls — it doesn't execute MCP tools

Thinking-Block Provider Switching

You cannot switch between providers mid-conversation when reasoning/thinking is enabled.

GLM-4.7's thinking block signatures are different from Anthropic's and will be rejected by the Anthropic API. If you start a conversation with Sonnet (GLM-4.7) and then switch to Opus (Anthropic API), the provider will reject the request.

Solution: Start a fresh session with /clear before switching to a different provider.

Zero Dependencies

The proxy has zero npm dependencies — it uses only Node.js built-ins (node:http, node:fs, etc.). Do not add dependencies unless absolutely necessary.

Usage Tracking

The proxy automatically tracks token usage for all requests:

  • Token counts are extracted from both streaming and non-streaming responses
  • Data is persisted to usage-data.json (gitignored)
  • Monthly totals reset on the 1st of each month
  • Costs are calculated based on pricing in config.json
  • Live account balances are fetched from providers that support it (DeepSeek)
  • Use curl https://ai.ein-softworks.com/usage to view all usage data

Key Reference URLs

Reference in New Issue View Git Blame Copy Permalink