diff --git a/plans/usage-api-enhancements.md b/plans/usage-api-enhancements.md new file mode 100644 index 0000000..a8787c2 --- /dev/null +++ b/plans/usage-api-enhancements.md @@ -0,0 +1,336 @@ +o# Usage API Enhancements Plan + +## Goal +Enhance the `/usage` API endpoint to provide complete data for a dashboard widget, including models with zero usage, daily breakdowns, provider health, and lifetime token totals. + +## All changes in `proxy.mjs` only — zero new dependencies. + +--- + +## Pre-existing Bug to Fix + +**`buildUsageResponse()` line 232 has an unresolved Promise bug:** + +```js +const balances = fetchProviderBalances(); // Start fetching but don't await +``` + +This assigns a Promise to `balances`, then on line 303: +```js +if (!balances.anthropic) { // always undefined — checking property on a Promise object +``` + +The notes about balances always fire incorrectly. The actual await happens later in the `/usage` handler (line 557). **Fix**: Remove the balance-related code from `buildUsageResponse()` entirely since it's handled in the request handler already. Move the notes logic to after balances are resolved in the `/usage` handler. + +--- + +## Change 1: Seed All Configured Models (Must Have) + +**What**: Ensure every route in `MODEL_ROUTES` appears in the response even with zero usage. + +**Where**: `buildUsageResponse()` — add a loop after the existing `usageData.models` iteration (after line ~282). + +**Logic**: +``` +For each [requestedModel, route] in MODEL_ROUTES: + providerName = route.provider + if providers[providerName].models[requestedModel] does NOT exist: + add it with: + routesTo: route.rewriteModel || requestedModel + total/monthly/today: all zeros + costUsd: null for flat-rate, 0 for token-based +``` + +**Result**: The widget sees all available routes including zero-usage models like: +```json +"deepseek": { + "models": { + "claude-haiku-4-5": { "routesTo": "deepseek-chat", ... }, + "claude-haiku-4-5-20251001": { "routesTo": "deepseek-chat", ... }, + "deepseek-chat": { "routesTo": "deepseek-chat", ... } + } +} +``` + +The widget can deduplicate by `routesTo` if it wants to show only unique upstream models. + +--- + +## Change 2: Daily Usage Tracking (Nice to Have) + +**What**: Track per-day usage alongside existing total and monthly tracking. + +### 2a. Add `getCurrentDate()` helper (new function, near `getCurrentPeriod()` at line 82) + +```js +function getCurrentDate() { + const now = new Date(); + return now.toISOString().slice(0, 10); // "2026-03-27" +} +``` + +### 2b. Extend `recordUsage()` (line 117) + +After the monthly period rollover check (line 133), add daily rollover: +```js +const todayStr = getCurrentDate(); +if (!modelData.daily || modelData.daily.date !== todayStr) { + modelData.daily = { date: todayStr, inputTokens: 0, outputTokens: 0, requests: 0 }; +} +modelData.daily.inputTokens += inputTokens || 0; +modelData.daily.outputTokens += outputTokens || 0; +modelData.daily.requests += 1; +``` + +### 2c. Backward compatibility + +Existing `usage-data.json` entries won't have `daily`. In `buildUsageResponse()`, check for its existence: +```js +const daily = modelData.daily?.date === getCurrentDate() ? modelData.daily : null; +``` +If `daily` is null or stale date, use zeros. + +### 2d. Schema addition to `usage-data.json` (auto-updated on next request) + +```json +{ + "daily": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0 } +} +``` + +--- + +## Change 3: Provider Status Tracking (Nice to Have) + +**What**: In-memory map tracking last request outcome per provider. Resets to `"idle"` on restart. + +### 3a. Declare `providerStatus` (new module-level const, after line ~78) + +```js +const providerStatus = {}; +for (const name of Object.keys(PROVIDERS)) { + providerStatus[name] = { status: "idle", lastSeen: null, lastError: null }; +} +``` + +### 3b. Update status in `handleMessages()` — three insertion points: + +1. **Streaming success** (after line ~475, after usage extraction in streaming path): + ```js + providerStatus[route.provider] = { status: "ok", lastSeen: new Date().toISOString(), lastError: null }; + ``` + +2. **Non-streaming response** (after line ~487, inside the non-streaming else branch): + - If `statusCode >= 200 && statusCode < 300`: status `"ok"` + - Else: status `"degraded"` with `lastError: "HTTP " + statusCode` + +3. **Fetch failure** (inside catch at line ~505): + ```js + providerStatus[route.provider] = { status: "error", lastSeen: new Date().toISOString(), lastError: fetchErr.message }; + ``` + +### 3c. Include in `buildUsageResponse()` output + +For each provider, add `status`, `lastSeen`, `lastError` from the in-memory map. + +--- + +## Change 4: Lifetime Token Totals in `totals` (Low Priority) + +**What**: Add `totalInputTokens`, `totalOutputTokens`, `monthlyInputTokens`, `monthlyOutputTokens` to the `totals` response object. + +**Where**: `buildUsageResponse()` aggregation loop (line ~248). Add four new accumulators initialized to 0, incremented alongside existing `totalRequests`/`monthlyRequests`. + +**New fields in `totals`**: +```json +{ + "totalInputTokens": 105000, + "totalOutputTokens": 262000, + "monthlyInputTokens": 105000, + "monthlyOutputTokens": 262000 +} +``` + +--- + +## Change 5: Per-Model Pricing (Low Priority) + +**What**: Include pricing info on each model entry so the widget doesn't have to cross-reference provider pricing. + +**Where**: `buildUsageResponse()` — when building each model's output object. + +**Logic**: +```js +const pricing = config.pricing?.[providerName] || DEFAULT_PRICING[providerName]; +modelEntry.pricing = pricing?.type === "flat" ? "flat_rate" : { inputPerMTok: pricing.inputPerMTok, outputPerMTok: pricing.outputPerMTok }; +``` + +--- + +## Change 6: Today's Aggregates in `totals` (Nice to Have) + +**What**: Add `todayRequests`, `todayCostUsd`, `todayInputTokens`, `todayOutputTokens` to the `totals` object. + +**Where**: `buildUsageResponse()` — aggregate daily data across all models. Depends on Change 2 (daily tracking). + +**New accumulators**: `todayRequests`, `todayCost`, `todayInputTokens`, `todayOutputTokens` — sum of all models' `today` values. + +--- + +## Proposed `/usage` Response Shape (complete) + +```json +{ + "service": "ein-ai-proxy", + "generatedAt": "2026-03-27T15:51:00.000Z", + "currentPeriod": "2026-03", + "providers": { + "anthropic": { + "status": "ok", + "lastSeen": "2026-03-27T14:30:00.000Z", + "lastError": null, + "balance": null, + "pricingNote": "No balance API available - check console.anthropic.com", + "pricing": { "inputPerMTok": 15, "outputPerMTok": 75 }, + "models": { + "claude-opus-4-6": { + "routesTo": "claude-opus-4-6", + "pricing": { "inputPerMTok": 15, "outputPerMTok": 75 }, + "total": { "inputTokens": 5000, "outputTokens": 12000, "requests": 3, "costUsd": 0.975 }, + "monthly": { "period": "2026-03", "inputTokens": 5000, "outputTokens": 12000, "requests": 3, "costUsd": 0.975 }, + "today": { "date": "2026-03-27", "inputTokens": 2000, "outputTokens": 5000, "requests": 1, "costUsd": 0.405 } + }, + "claude-opus-4-5": { + "routesTo": "claude-opus-4-5", + "pricing": { "inputPerMTok": 15, "outputPerMTok": 75 }, + "total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }, + "monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }, + "today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 } + }, + "claude-opus-4-5-20251101": { + "routesTo": "claude-opus-4-5-20251101", + "pricing": { "inputPerMTok": 15, "outputPerMTok": 75 }, + "total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }, + "monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }, + "today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 } + } + } + }, + "deepseek": { + "status": "idle", + "lastSeen": null, + "lastError": null, + "balance": { "totalBalance": "4.51", "currency": "USD" }, + "pricing": { "inputPerMTok": 0.27, "outputPerMTok": 1.10 }, + "models": { + "claude-haiku-4-5-20251001": { + "routesTo": "deepseek-chat", + "pricing": { "inputPerMTok": 0.27, "outputPerMTok": 1.10 }, + "total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }, + "monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }, + "today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 } + }, + "claude-haiku-4-5": { + "routesTo": "deepseek-chat", + "pricing": { "inputPerMTok": 0.27, "outputPerMTok": 1.10 }, + "total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }, + "monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }, + "today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 } + }, + "deepseek-chat": { + "routesTo": "deepseek-chat", + "pricing": { "inputPerMTok": 0.27, "outputPerMTok": 1.10 }, + "total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }, + "monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }, + "today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 } + } + } + }, + "zai": { + "status": "ok", + "lastSeen": "2026-03-27T15:00:00.000Z", + "lastError": null, + "balance": null, + "pricingNote": "Flat-rate subscription, no balance API available", + "pricing": "flat_rate", + "monthlyCostUsd": 10, + "models": { + "claude-sonnet-4-6": { + "routesTo": "glm-4.7", + "pricing": "flat_rate", + "total": { "inputTokens": 100000, "outputTokens": 250000, "requests": 50, "costUsd": null }, + "monthly": { "period": "2026-03", "inputTokens": 100000, "outputTokens": 250000, "requests": 50, "costUsd": null }, + "today": { "date": "2026-03-27", "inputTokens": 10000, "outputTokens": 25000, "requests": 5, "costUsd": null } + }, + "claude-sonnet-4-5": { + "routesTo": "glm-4.7", + "pricing": "flat_rate", + "total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }, + "monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }, + "today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null } + }, + "claude-sonnet-4-5-20250929": { + "routesTo": "glm-4.7", + "pricing": "flat_rate", + "total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }, + "monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }, + "today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null } + }, + "glm-4.7": { + "routesTo": "glm-4.7", + "pricing": "flat_rate", + "total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }, + "monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }, + "today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null } + }, + "glm-4.5-air": { + "routesTo": "glm-4.5-air", + "pricing": "flat_rate", + "total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }, + "monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }, + "today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null } + } + } + } + }, + "totals": { + "totalRequests": 53, + "monthlyRequests": 53, + "todayRequests": 6, + "totalInputTokens": 105000, + "totalOutputTokens": 262000, + "monthlyInputTokens": 105000, + "monthlyOutputTokens": 262000, + "todayInputTokens": 12000, + "todayOutputTokens": 30000, + "totalCostUsd": 0.975, + "monthlyCostUsd": 0.975, + "todayCostUsd": 0.405, + "note": "Cost excludes flat-rate provider(s). Z.ai costs $10/mo regardless of usage. Anthropic has no public balance API - check console.anthropic.com. Z.ai is flat-rate; no balance API available." + } +} +``` + +--- + +## Implementation Order + +The changes should be applied in this order to minimize conflicts: + +1. **Bug fix**: Remove stale `balances` variable and balance-related notes from `buildUsageResponse()`. Move notes logic to `/usage` handler. +2. **`getCurrentDate()` helper**: Add near `getCurrentPeriod()`. +3. **`providerStatus` map**: Add module-level constant. +4. **`recordUsage()` changes**: Add daily tracking field. +5. **`handleMessages()` changes**: Add provider status updates at the 3 insertion points. +6. **`buildUsageResponse()` rewrite**: The big change — model seeding, daily/today aggregation, lifetime token totals, per-model pricing, provider status inclusion. +7. **`/usage` handler update**: Updated notes logic after balances resolve. + +--- + +## Files Modified + +| File | Change | +|------|--------| +| `proxy.mjs` | All code changes (6 changes + 1 bug fix) | + +No config changes. No new files. No new dependencies. Backward-compatible with existing `usage-data.json`. diff --git a/proxy.mjs b/proxy.mjs index b52316e..df0642c 100644 --- a/proxy.mjs +++ b/proxy.mjs @@ -675,12 +675,21 @@ const server = createServer(async (req, res) => { notes.push("Anthropic has no public balance API — check console.anthropic.com"); notes.push("Z.ai is flat-rate; no balance API available"); + // Build routing summary (the 3 Claude Code / Roo Code tiers) + const routing = { + sonnet: { provider: "zai", model: "glm-4.7", description: "Daily workhorse (Z.ai flat-rate)" }, + opus: { provider: "anthropic", model: "claude-opus-4-6", description: "Complex tasks (Anthropic pay-per-token)" }, + haiku: { provider: "deepseek", model: "deepseek-chat", description: "Trivial/fast tasks (DeepSeek cheap tier)" }, + default: { provider: "zai", model: "glm-4.7", description: "Unknown models fall back here" }, + }; + res.writeHead(200, { "content-type": "application/json" }); res.end( JSON.stringify({ service: "ein-ai-proxy", generatedAt: new Date().toISOString(), currentPeriod, + routing, providers, totals: { ...totals,