added router details to usage endpoint
This commit is contained in:
William Stuckey
@@ -0,0 +1,336 @@
|
||||
o# Usage API Enhancements Plan
|
||||
|
||||
## Goal
|
||||
Enhance the `/usage` API endpoint to provide complete data for a dashboard widget, including models with zero usage, daily breakdowns, provider health, and lifetime token totals.
|
||||
|
||||
## All changes in `proxy.mjs` only — zero new dependencies.
|
||||
|
||||
---
|
||||
|
||||
## Pre-existing Bug to Fix
|
||||
|
||||
**`buildUsageResponse()` line 232 has an unresolved Promise bug:**
|
||||
|
||||
```js
|
||||
const balances = fetchProviderBalances(); // Start fetching but don't await
|
||||
```
|
||||
|
||||
This assigns a Promise to `balances`, then on line 303:
|
||||
```js
|
||||
if (!balances.anthropic) { // always undefined — checking property on a Promise object
|
||||
```
|
||||
|
||||
The notes about balances always fire incorrectly. The actual await happens later in the `/usage` handler (line 557). **Fix**: Remove the balance-related code from `buildUsageResponse()` entirely since it's handled in the request handler already. Move the notes logic to after balances are resolved in the `/usage` handler.
|
||||
|
||||
---
|
||||
|
||||
## Change 1: Seed All Configured Models (Must Have)
|
||||
|
||||
**What**: Ensure every route in `MODEL_ROUTES` appears in the response even with zero usage.
|
||||
|
||||
**Where**: `buildUsageResponse()` — add a loop after the existing `usageData.models` iteration (after line ~282).
|
||||
|
||||
**Logic**:
|
||||
```
|
||||
For each [requestedModel, route] in MODEL_ROUTES:
|
||||
providerName = route.provider
|
||||
if providers[providerName].models[requestedModel] does NOT exist:
|
||||
add it with:
|
||||
routesTo: route.rewriteModel || requestedModel
|
||||
total/monthly/today: all zeros
|
||||
costUsd: null for flat-rate, 0 for token-based
|
||||
```
|
||||
|
||||
**Result**: The widget sees all available routes including zero-usage models like:
|
||||
```json
|
||||
"deepseek": {
|
||||
"models": {
|
||||
"claude-haiku-4-5": { "routesTo": "deepseek-chat", ... },
|
||||
"claude-haiku-4-5-20251001": { "routesTo": "deepseek-chat", ... },
|
||||
"deepseek-chat": { "routesTo": "deepseek-chat", ... }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The widget can deduplicate by `routesTo` if it wants to show only unique upstream models.
|
||||
|
||||
---
|
||||
|
||||
## Change 2: Daily Usage Tracking (Nice to Have)
|
||||
|
||||
**What**: Track per-day usage alongside existing total and monthly tracking.
|
||||
|
||||
### 2a. Add `getCurrentDate()` helper (new function, near `getCurrentPeriod()` at line 82)
|
||||
|
||||
```js
|
||||
function getCurrentDate() {
|
||||
const now = new Date();
|
||||
return now.toISOString().slice(0, 10); // "2026-03-27"
|
||||
}
|
||||
```
|
||||
|
||||
### 2b. Extend `recordUsage()` (line 117)
|
||||
|
||||
After the monthly period rollover check (line 133), add daily rollover:
|
||||
```js
|
||||
const todayStr = getCurrentDate();
|
||||
if (!modelData.daily || modelData.daily.date !== todayStr) {
|
||||
modelData.daily = { date: todayStr, inputTokens: 0, outputTokens: 0, requests: 0 };
|
||||
}
|
||||
modelData.daily.inputTokens += inputTokens || 0;
|
||||
modelData.daily.outputTokens += outputTokens || 0;
|
||||
modelData.daily.requests += 1;
|
||||
```
|
||||
|
||||
### 2c. Backward compatibility
|
||||
|
||||
Existing `usage-data.json` entries won't have `daily`. In `buildUsageResponse()`, check for its existence:
|
||||
```js
|
||||
const daily = modelData.daily?.date === getCurrentDate() ? modelData.daily : null;
|
||||
```
|
||||
If `daily` is null or stale date, use zeros.
|
||||
|
||||
### 2d. Schema addition to `usage-data.json` (auto-updated on next request)
|
||||
|
||||
```json
|
||||
{
|
||||
"daily": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0 }
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Change 3: Provider Status Tracking (Nice to Have)
|
||||
|
||||
**What**: In-memory map tracking last request outcome per provider. Resets to `"idle"` on restart.
|
||||
|
||||
### 3a. Declare `providerStatus` (new module-level const, after line ~78)
|
||||
|
||||
```js
|
||||
const providerStatus = {};
|
||||
for (const name of Object.keys(PROVIDERS)) {
|
||||
providerStatus[name] = { status: "idle", lastSeen: null, lastError: null };
|
||||
}
|
||||
```
|
||||
|
||||
### 3b. Update status in `handleMessages()` — three insertion points:
|
||||
|
||||
1. **Streaming success** (after line ~475, after usage extraction in streaming path):
|
||||
```js
|
||||
providerStatus[route.provider] = { status: "ok", lastSeen: new Date().toISOString(), lastError: null };
|
||||
```
|
||||
|
||||
2. **Non-streaming response** (after line ~487, inside the non-streaming else branch):
|
||||
- If `statusCode >= 200 && statusCode < 300`: status `"ok"`
|
||||
- Else: status `"degraded"` with `lastError: "HTTP " + statusCode`
|
||||
|
||||
3. **Fetch failure** (inside catch at line ~505):
|
||||
```js
|
||||
providerStatus[route.provider] = { status: "error", lastSeen: new Date().toISOString(), lastError: fetchErr.message };
|
||||
```
|
||||
|
||||
### 3c. Include in `buildUsageResponse()` output
|
||||
|
||||
For each provider, add `status`, `lastSeen`, `lastError` from the in-memory map.
|
||||
|
||||
---
|
||||
|
||||
## Change 4: Lifetime Token Totals in `totals` (Low Priority)
|
||||
|
||||
**What**: Add `totalInputTokens`, `totalOutputTokens`, `monthlyInputTokens`, `monthlyOutputTokens` to the `totals` response object.
|
||||
|
||||
**Where**: `buildUsageResponse()` aggregation loop (line ~248). Add four new accumulators initialized to 0, incremented alongside existing `totalRequests`/`monthlyRequests`.
|
||||
|
||||
**New fields in `totals`**:
|
||||
```json
|
||||
{
|
||||
"totalInputTokens": 105000,
|
||||
"totalOutputTokens": 262000,
|
||||
"monthlyInputTokens": 105000,
|
||||
"monthlyOutputTokens": 262000
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Change 5: Per-Model Pricing (Low Priority)
|
||||
|
||||
**What**: Include pricing info on each model entry so the widget doesn't have to cross-reference provider pricing.
|
||||
|
||||
**Where**: `buildUsageResponse()` — when building each model's output object.
|
||||
|
||||
**Logic**:
|
||||
```js
|
||||
const pricing = config.pricing?.[providerName] || DEFAULT_PRICING[providerName];
|
||||
modelEntry.pricing = pricing?.type === "flat" ? "flat_rate" : { inputPerMTok: pricing.inputPerMTok, outputPerMTok: pricing.outputPerMTok };
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Change 6: Today's Aggregates in `totals` (Nice to Have)
|
||||
|
||||
**What**: Add `todayRequests`, `todayCostUsd`, `todayInputTokens`, `todayOutputTokens` to the `totals` object.
|
||||
|
||||
**Where**: `buildUsageResponse()` — aggregate daily data across all models. Depends on Change 2 (daily tracking).
|
||||
|
||||
**New accumulators**: `todayRequests`, `todayCost`, `todayInputTokens`, `todayOutputTokens` — sum of all models' `today` values.
|
||||
|
||||
---
|
||||
|
||||
## Proposed `/usage` Response Shape (complete)
|
||||
|
||||
```json
|
||||
{
|
||||
"service": "ein-ai-proxy",
|
||||
"generatedAt": "2026-03-27T15:51:00.000Z",
|
||||
"currentPeriod": "2026-03",
|
||||
"providers": {
|
||||
"anthropic": {
|
||||
"status": "ok",
|
||||
"lastSeen": "2026-03-27T14:30:00.000Z",
|
||||
"lastError": null,
|
||||
"balance": null,
|
||||
"pricingNote": "No balance API available - check console.anthropic.com",
|
||||
"pricing": { "inputPerMTok": 15, "outputPerMTok": 75 },
|
||||
"models": {
|
||||
"claude-opus-4-6": {
|
||||
"routesTo": "claude-opus-4-6",
|
||||
"pricing": { "inputPerMTok": 15, "outputPerMTok": 75 },
|
||||
"total": { "inputTokens": 5000, "outputTokens": 12000, "requests": 3, "costUsd": 0.975 },
|
||||
"monthly": { "period": "2026-03", "inputTokens": 5000, "outputTokens": 12000, "requests": 3, "costUsd": 0.975 },
|
||||
"today": { "date": "2026-03-27", "inputTokens": 2000, "outputTokens": 5000, "requests": 1, "costUsd": 0.405 }
|
||||
},
|
||||
"claude-opus-4-5": {
|
||||
"routesTo": "claude-opus-4-5",
|
||||
"pricing": { "inputPerMTok": 15, "outputPerMTok": 75 },
|
||||
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
||||
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
||||
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }
|
||||
},
|
||||
"claude-opus-4-5-20251101": {
|
||||
"routesTo": "claude-opus-4-5-20251101",
|
||||
"pricing": { "inputPerMTok": 15, "outputPerMTok": 75 },
|
||||
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
||||
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
||||
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }
|
||||
}
|
||||
}
|
||||
},
|
||||
"deepseek": {
|
||||
"status": "idle",
|
||||
"lastSeen": null,
|
||||
"lastError": null,
|
||||
"balance": { "totalBalance": "4.51", "currency": "USD" },
|
||||
"pricing": { "inputPerMTok": 0.27, "outputPerMTok": 1.10 },
|
||||
"models": {
|
||||
"claude-haiku-4-5-20251001": {
|
||||
"routesTo": "deepseek-chat",
|
||||
"pricing": { "inputPerMTok": 0.27, "outputPerMTok": 1.10 },
|
||||
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
||||
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
||||
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }
|
||||
},
|
||||
"claude-haiku-4-5": {
|
||||
"routesTo": "deepseek-chat",
|
||||
"pricing": { "inputPerMTok": 0.27, "outputPerMTok": 1.10 },
|
||||
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
||||
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
||||
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }
|
||||
},
|
||||
"deepseek-chat": {
|
||||
"routesTo": "deepseek-chat",
|
||||
"pricing": { "inputPerMTok": 0.27, "outputPerMTok": 1.10 },
|
||||
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
||||
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
||||
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }
|
||||
}
|
||||
}
|
||||
},
|
||||
"zai": {
|
||||
"status": "ok",
|
||||
"lastSeen": "2026-03-27T15:00:00.000Z",
|
||||
"lastError": null,
|
||||
"balance": null,
|
||||
"pricingNote": "Flat-rate subscription, no balance API available",
|
||||
"pricing": "flat_rate",
|
||||
"monthlyCostUsd": 10,
|
||||
"models": {
|
||||
"claude-sonnet-4-6": {
|
||||
"routesTo": "glm-4.7",
|
||||
"pricing": "flat_rate",
|
||||
"total": { "inputTokens": 100000, "outputTokens": 250000, "requests": 50, "costUsd": null },
|
||||
"monthly": { "period": "2026-03", "inputTokens": 100000, "outputTokens": 250000, "requests": 50, "costUsd": null },
|
||||
"today": { "date": "2026-03-27", "inputTokens": 10000, "outputTokens": 25000, "requests": 5, "costUsd": null }
|
||||
},
|
||||
"claude-sonnet-4-5": {
|
||||
"routesTo": "glm-4.7",
|
||||
"pricing": "flat_rate",
|
||||
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
||||
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
||||
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }
|
||||
},
|
||||
"claude-sonnet-4-5-20250929": {
|
||||
"routesTo": "glm-4.7",
|
||||
"pricing": "flat_rate",
|
||||
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
||||
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
||||
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }
|
||||
},
|
||||
"glm-4.7": {
|
||||
"routesTo": "glm-4.7",
|
||||
"pricing": "flat_rate",
|
||||
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
||||
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
||||
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }
|
||||
},
|
||||
"glm-4.5-air": {
|
||||
"routesTo": "glm-4.5-air",
|
||||
"pricing": "flat_rate",
|
||||
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
||||
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
||||
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"totals": {
|
||||
"totalRequests": 53,
|
||||
"monthlyRequests": 53,
|
||||
"todayRequests": 6,
|
||||
"totalInputTokens": 105000,
|
||||
"totalOutputTokens": 262000,
|
||||
"monthlyInputTokens": 105000,
|
||||
"monthlyOutputTokens": 262000,
|
||||
"todayInputTokens": 12000,
|
||||
"todayOutputTokens": 30000,
|
||||
"totalCostUsd": 0.975,
|
||||
"monthlyCostUsd": 0.975,
|
||||
"todayCostUsd": 0.405,
|
||||
"note": "Cost excludes flat-rate provider(s). Z.ai costs $10/mo regardless of usage. Anthropic has no public balance API - check console.anthropic.com. Z.ai is flat-rate; no balance API available."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Order
|
||||
|
||||
The changes should be applied in this order to minimize conflicts:
|
||||
|
||||
1. **Bug fix**: Remove stale `balances` variable and balance-related notes from `buildUsageResponse()`. Move notes logic to `/usage` handler.
|
||||
2. **`getCurrentDate()` helper**: Add near `getCurrentPeriod()`.
|
||||
3. **`providerStatus` map**: Add module-level constant.
|
||||
4. **`recordUsage()` changes**: Add daily tracking field.
|
||||
5. **`handleMessages()` changes**: Add provider status updates at the 3 insertion points.
|
||||
6. **`buildUsageResponse()` rewrite**: The big change — model seeding, daily/today aggregation, lifetime token totals, per-model pricing, provider status inclusion.
|
||||
7. **`/usage` handler update**: Updated notes logic after balances resolve.
|
||||
|
||||
---
|
||||
|
||||
## Files Modified
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `proxy.mjs` | All code changes (6 changes + 1 bug fix) |
|
||||
|
||||
No config changes. No new files. No new dependencies. Backward-compatible with existing `usage-data.json`.
|
||||
@@ -675,12 +675,21 @@ const server = createServer(async (req, res) => {
|
||||
notes.push("Anthropic has no public balance API — check console.anthropic.com");
|
||||
notes.push("Z.ai is flat-rate; no balance API available");
|
||||
|
||||
// Build routing summary (the 3 Claude Code / Roo Code tiers)
|
||||
const routing = {
|
||||
sonnet: { provider: "zai", model: "glm-4.7", description: "Daily workhorse (Z.ai flat-rate)" },
|
||||
opus: { provider: "anthropic", model: "claude-opus-4-6", description: "Complex tasks (Anthropic pay-per-token)" },
|
||||
haiku: { provider: "deepseek", model: "deepseek-chat", description: "Trivial/fast tasks (DeepSeek cheap tier)" },
|
||||
default: { provider: "zai", model: "glm-4.7", description: "Unknown models fall back here" },
|
||||
};
|
||||
|
||||
res.writeHead(200, { "content-type": "application/json" });
|
||||
res.end(
|
||||
JSON.stringify({
|
||||
service: "ein-ai-proxy",
|
||||
generatedAt: new Date().toISOString(),
|
||||
currentPeriod,
|
||||
routing,
|
||||
providers,
|
||||
totals: {
|
||||
...totals,
|
||||
|
||||
Reference in New Issue
Block a user