William Stuckey
337 lines
13 KiB
Markdown
337 lines
13 KiB
Markdown
o# Usage API Enhancements Plan
|
|
|
|
## Goal
|
|
Enhance the `/usage` API endpoint to provide complete data for a dashboard widget, including models with zero usage, daily breakdowns, provider health, and lifetime token totals.
|
|
|
|
## All changes in `proxy.mjs` only — zero new dependencies.
|
|
|
|
---
|
|
|
|
## Pre-existing Bug to Fix
|
|
|
|
**`buildUsageResponse()` line 232 has an unresolved Promise bug:**
|
|
|
|
```js
|
|
const balances = fetchProviderBalances(); // Start fetching but don't await
|
|
```
|
|
|
|
This assigns a Promise to `balances`, then on line 303:
|
|
```js
|
|
if (!balances.anthropic) { // always undefined — checking property on a Promise object
|
|
```
|
|
|
|
The notes about balances always fire incorrectly. The actual await happens later in the `/usage` handler (line 557). **Fix**: Remove the balance-related code from `buildUsageResponse()` entirely since it's handled in the request handler already. Move the notes logic to after balances are resolved in the `/usage` handler.
|
|
|
|
---
|
|
|
|
## Change 1: Seed All Configured Models (Must Have)
|
|
|
|
**What**: Ensure every route in `MODEL_ROUTES` appears in the response even with zero usage.
|
|
|
|
**Where**: `buildUsageResponse()` — add a loop after the existing `usageData.models` iteration (after line ~282).
|
|
|
|
**Logic**:
|
|
```
|
|
For each [requestedModel, route] in MODEL_ROUTES:
|
|
providerName = route.provider
|
|
if providers[providerName].models[requestedModel] does NOT exist:
|
|
add it with:
|
|
routesTo: route.rewriteModel || requestedModel
|
|
total/monthly/today: all zeros
|
|
costUsd: null for flat-rate, 0 for token-based
|
|
```
|
|
|
|
**Result**: The widget sees all available routes including zero-usage models like:
|
|
```json
|
|
"deepseek": {
|
|
"models": {
|
|
"claude-haiku-4-5": { "routesTo": "deepseek-chat", ... },
|
|
"claude-haiku-4-5-20251001": { "routesTo": "deepseek-chat", ... },
|
|
"deepseek-chat": { "routesTo": "deepseek-chat", ... }
|
|
}
|
|
}
|
|
```
|
|
|
|
The widget can deduplicate by `routesTo` if it wants to show only unique upstream models.
|
|
|
|
---
|
|
|
|
## Change 2: Daily Usage Tracking (Nice to Have)
|
|
|
|
**What**: Track per-day usage alongside existing total and monthly tracking.
|
|
|
|
### 2a. Add `getCurrentDate()` helper (new function, near `getCurrentPeriod()` at line 82)
|
|
|
|
```js
|
|
function getCurrentDate() {
|
|
const now = new Date();
|
|
return now.toISOString().slice(0, 10); // "2026-03-27"
|
|
}
|
|
```
|
|
|
|
### 2b. Extend `recordUsage()` (line 117)
|
|
|
|
After the monthly period rollover check (line 133), add daily rollover:
|
|
```js
|
|
const todayStr = getCurrentDate();
|
|
if (!modelData.daily || modelData.daily.date !== todayStr) {
|
|
modelData.daily = { date: todayStr, inputTokens: 0, outputTokens: 0, requests: 0 };
|
|
}
|
|
modelData.daily.inputTokens += inputTokens || 0;
|
|
modelData.daily.outputTokens += outputTokens || 0;
|
|
modelData.daily.requests += 1;
|
|
```
|
|
|
|
### 2c. Backward compatibility
|
|
|
|
Existing `usage-data.json` entries won't have `daily`. In `buildUsageResponse()`, check for its existence:
|
|
```js
|
|
const daily = modelData.daily?.date === getCurrentDate() ? modelData.daily : null;
|
|
```
|
|
If `daily` is null or stale date, use zeros.
|
|
|
|
### 2d. Schema addition to `usage-data.json` (auto-updated on next request)
|
|
|
|
```json
|
|
{
|
|
"daily": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0 }
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Change 3: Provider Status Tracking (Nice to Have)
|
|
|
|
**What**: In-memory map tracking last request outcome per provider. Resets to `"idle"` on restart.
|
|
|
|
### 3a. Declare `providerStatus` (new module-level const, after line ~78)
|
|
|
|
```js
|
|
const providerStatus = {};
|
|
for (const name of Object.keys(PROVIDERS)) {
|
|
providerStatus[name] = { status: "idle", lastSeen: null, lastError: null };
|
|
}
|
|
```
|
|
|
|
### 3b. Update status in `handleMessages()` — three insertion points:
|
|
|
|
1. **Streaming success** (after line ~475, after usage extraction in streaming path):
|
|
```js
|
|
providerStatus[route.provider] = { status: "ok", lastSeen: new Date().toISOString(), lastError: null };
|
|
```
|
|
|
|
2. **Non-streaming response** (after line ~487, inside the non-streaming else branch):
|
|
- If `statusCode >= 200 && statusCode < 300`: status `"ok"`
|
|
- Else: status `"degraded"` with `lastError: "HTTP " + statusCode`
|
|
|
|
3. **Fetch failure** (inside catch at line ~505):
|
|
```js
|
|
providerStatus[route.provider] = { status: "error", lastSeen: new Date().toISOString(), lastError: fetchErr.message };
|
|
```
|
|
|
|
### 3c. Include in `buildUsageResponse()` output
|
|
|
|
For each provider, add `status`, `lastSeen`, `lastError` from the in-memory map.
|
|
|
|
---
|
|
|
|
## Change 4: Lifetime Token Totals in `totals` (Low Priority)
|
|
|
|
**What**: Add `totalInputTokens`, `totalOutputTokens`, `monthlyInputTokens`, `monthlyOutputTokens` to the `totals` response object.
|
|
|
|
**Where**: `buildUsageResponse()` aggregation loop (line ~248). Add four new accumulators initialized to 0, incremented alongside existing `totalRequests`/`monthlyRequests`.
|
|
|
|
**New fields in `totals`**:
|
|
```json
|
|
{
|
|
"totalInputTokens": 105000,
|
|
"totalOutputTokens": 262000,
|
|
"monthlyInputTokens": 105000,
|
|
"monthlyOutputTokens": 262000
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Change 5: Per-Model Pricing (Low Priority)
|
|
|
|
**What**: Include pricing info on each model entry so the widget doesn't have to cross-reference provider pricing.
|
|
|
|
**Where**: `buildUsageResponse()` — when building each model's output object.
|
|
|
|
**Logic**:
|
|
```js
|
|
const pricing = config.pricing?.[providerName] || DEFAULT_PRICING[providerName];
|
|
modelEntry.pricing = pricing?.type === "flat" ? "flat_rate" : { inputPerMTok: pricing.inputPerMTok, outputPerMTok: pricing.outputPerMTok };
|
|
```
|
|
|
|
---
|
|
|
|
## Change 6: Today's Aggregates in `totals` (Nice to Have)
|
|
|
|
**What**: Add `todayRequests`, `todayCostUsd`, `todayInputTokens`, `todayOutputTokens` to the `totals` object.
|
|
|
|
**Where**: `buildUsageResponse()` — aggregate daily data across all models. Depends on Change 2 (daily tracking).
|
|
|
|
**New accumulators**: `todayRequests`, `todayCost`, `todayInputTokens`, `todayOutputTokens` — sum of all models' `today` values.
|
|
|
|
---
|
|
|
|
## Proposed `/usage` Response Shape (complete)
|
|
|
|
```json
|
|
{
|
|
"service": "ein-ai-proxy",
|
|
"generatedAt": "2026-03-27T15:51:00.000Z",
|
|
"currentPeriod": "2026-03",
|
|
"providers": {
|
|
"anthropic": {
|
|
"status": "ok",
|
|
"lastSeen": "2026-03-27T14:30:00.000Z",
|
|
"lastError": null,
|
|
"balance": null,
|
|
"pricingNote": "No balance API available - check console.anthropic.com",
|
|
"pricing": { "inputPerMTok": 15, "outputPerMTok": 75 },
|
|
"models": {
|
|
"claude-opus-4-6": {
|
|
"routesTo": "claude-opus-4-6",
|
|
"pricing": { "inputPerMTok": 15, "outputPerMTok": 75 },
|
|
"total": { "inputTokens": 5000, "outputTokens": 12000, "requests": 3, "costUsd": 0.975 },
|
|
"monthly": { "period": "2026-03", "inputTokens": 5000, "outputTokens": 12000, "requests": 3, "costUsd": 0.975 },
|
|
"today": { "date": "2026-03-27", "inputTokens": 2000, "outputTokens": 5000, "requests": 1, "costUsd": 0.405 }
|
|
},
|
|
"claude-opus-4-5": {
|
|
"routesTo": "claude-opus-4-5",
|
|
"pricing": { "inputPerMTok": 15, "outputPerMTok": 75 },
|
|
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
|
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
|
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }
|
|
},
|
|
"claude-opus-4-5-20251101": {
|
|
"routesTo": "claude-opus-4-5-20251101",
|
|
"pricing": { "inputPerMTok": 15, "outputPerMTok": 75 },
|
|
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
|
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
|
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }
|
|
}
|
|
}
|
|
},
|
|
"deepseek": {
|
|
"status": "idle",
|
|
"lastSeen": null,
|
|
"lastError": null,
|
|
"balance": { "totalBalance": "4.51", "currency": "USD" },
|
|
"pricing": { "inputPerMTok": 0.27, "outputPerMTok": 1.10 },
|
|
"models": {
|
|
"claude-haiku-4-5-20251001": {
|
|
"routesTo": "deepseek-chat",
|
|
"pricing": { "inputPerMTok": 0.27, "outputPerMTok": 1.10 },
|
|
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
|
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
|
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }
|
|
},
|
|
"claude-haiku-4-5": {
|
|
"routesTo": "deepseek-chat",
|
|
"pricing": { "inputPerMTok": 0.27, "outputPerMTok": 1.10 },
|
|
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
|
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
|
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }
|
|
},
|
|
"deepseek-chat": {
|
|
"routesTo": "deepseek-chat",
|
|
"pricing": { "inputPerMTok": 0.27, "outputPerMTok": 1.10 },
|
|
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
|
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 },
|
|
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": 0 }
|
|
}
|
|
}
|
|
},
|
|
"zai": {
|
|
"status": "ok",
|
|
"lastSeen": "2026-03-27T15:00:00.000Z",
|
|
"lastError": null,
|
|
"balance": null,
|
|
"pricingNote": "Flat-rate subscription, no balance API available",
|
|
"pricing": "flat_rate",
|
|
"monthlyCostUsd": 10,
|
|
"models": {
|
|
"claude-sonnet-4-6": {
|
|
"routesTo": "glm-4.7",
|
|
"pricing": "flat_rate",
|
|
"total": { "inputTokens": 100000, "outputTokens": 250000, "requests": 50, "costUsd": null },
|
|
"monthly": { "period": "2026-03", "inputTokens": 100000, "outputTokens": 250000, "requests": 50, "costUsd": null },
|
|
"today": { "date": "2026-03-27", "inputTokens": 10000, "outputTokens": 25000, "requests": 5, "costUsd": null }
|
|
},
|
|
"claude-sonnet-4-5": {
|
|
"routesTo": "glm-4.7",
|
|
"pricing": "flat_rate",
|
|
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
|
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
|
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }
|
|
},
|
|
"claude-sonnet-4-5-20250929": {
|
|
"routesTo": "glm-4.7",
|
|
"pricing": "flat_rate",
|
|
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
|
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
|
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }
|
|
},
|
|
"glm-4.7": {
|
|
"routesTo": "glm-4.7",
|
|
"pricing": "flat_rate",
|
|
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
|
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
|
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }
|
|
},
|
|
"glm-4.5-air": {
|
|
"routesTo": "glm-4.5-air",
|
|
"pricing": "flat_rate",
|
|
"total": { "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
|
"monthly": { "period": "2026-03", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null },
|
|
"today": { "date": "2026-03-27", "inputTokens": 0, "outputTokens": 0, "requests": 0, "costUsd": null }
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"totals": {
|
|
"totalRequests": 53,
|
|
"monthlyRequests": 53,
|
|
"todayRequests": 6,
|
|
"totalInputTokens": 105000,
|
|
"totalOutputTokens": 262000,
|
|
"monthlyInputTokens": 105000,
|
|
"monthlyOutputTokens": 262000,
|
|
"todayInputTokens": 12000,
|
|
"todayOutputTokens": 30000,
|
|
"totalCostUsd": 0.975,
|
|
"monthlyCostUsd": 0.975,
|
|
"todayCostUsd": 0.405,
|
|
"note": "Cost excludes flat-rate provider(s). Z.ai costs $10/mo regardless of usage. Anthropic has no public balance API - check console.anthropic.com. Z.ai is flat-rate; no balance API available."
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Implementation Order
|
|
|
|
The changes should be applied in this order to minimize conflicts:
|
|
|
|
1. **Bug fix**: Remove stale `balances` variable and balance-related notes from `buildUsageResponse()`. Move notes logic to `/usage` handler.
|
|
2. **`getCurrentDate()` helper**: Add near `getCurrentPeriod()`.
|
|
3. **`providerStatus` map**: Add module-level constant.
|
|
4. **`recordUsage()` changes**: Add daily tracking field.
|
|
5. **`handleMessages()` changes**: Add provider status updates at the 3 insertion points.
|
|
6. **`buildUsageResponse()` rewrite**: The big change — model seeding, daily/today aggregation, lifetime token totals, per-model pricing, provider status inclusion.
|
|
7. **`/usage` handler update**: Updated notes logic after balances resolve.
|
|
|
|
---
|
|
|
|
## Files Modified
|
|
|
|
| File | Change |
|
|
|------|--------|
|
|
| `proxy.mjs` | All code changes (6 changes + 1 bug fix) |
|
|
|
|
No config changes. No new files. No new dependencies. Backward-compatible with existing `usage-data.json`.
|