Requests and tokens
Compare input, output, cache and media usage by model. Separate provider-reported totals from estimates.
LLM observability
See what every AI request costs.
Track requests, tokens, latency, failures and estimated provider spend in one workspace. Keep your existing backend, use the secure gateway, or combine both.
No card required1,000 requests includedPrompts stay out of analytics
What you can measure
Costs with the context needed to act.
A total is useful only when you can explain it. HyperProxy keeps request, model, client and session dimensions connected.
Estimated spend
Apply the model catalog or your own price overrides. Unknown usage remains visibly incomplete instead of becoming a misleading zero.
Latency and errors
Use the same date range across cost, latency, status codes and request history to find what changed.
Clients and sessions
Group calls with the client and session identifiers your application supplies. HyperProxy never guesses the platform.
Budgets and alerts
Email at 80% and 100%, detect daily spend anomalies, or stop gateway traffic when recorded monthly spend reaches your budget.
Gateway or direct
Measure traffic routed through HyperProxy and server-reported events in the same project, with their sources clearly labelled.
Three integration paths
Start where your traffic already runs.
Secure gateway
Change the request origin and auth header. HyperProxy forwards the provider-native request and meters completed usage.
External events API
Keep direct provider calls in your backend. Submit one idempotent event after each completed call from a background worker.
OpenTelemetry
Point an OTLP/HTTP trace exporter at the project endpoint and send completed GenAI client spans.
Analytics only
Your provider key and payload stay in your backend.
Send the outcome after the AI call finishes. HyperProxy needs provider, model, status, duration and available token totals—not the prompt or response body.
- Persist one
event_idand reuse it for retries. - Queue telemetry so it never delays the user response.
- Use
client_idandsession_idonly when they help your own analysis.
external-event.jsonPOST /events
{
"event_id": "8fc1…",
"provider": "openai",
"model": "gpt-4o-mini",
"status_code": 200,
"duration_ms": 320,
"tokens_in": 1000,
"tokens_out": 500
}Clear boundaries
Know what the number means.
| Question | HyperProxy reports | Source of record |
|---|---|---|
| How much provider usage did the app record? | Requests, tokens and estimated model cost | HyperProxy analytics |
| What will the provider charge? | An operational estimate with coverage warnings | Provider invoice |
| What does HyperProxy itself cost? | Plan allowance and overage history | HyperProxy billing |
FAQ
LLM cost tracking questions.
Does HyperProxy replace provider billing?
No. It gives you an operational estimate tied to request history. Provider invoices remain the billing source of record.
Can I track costs without proxying prompts?
Yes. Use external events or OpenTelemetry from your backend. Those paths do not include provider credentials, prompts or response bodies.
What happens when pricing or usage is missing?
The request is marked as incomplete cost. Totals show recorded cost and a coverage warning, so an unknown request never looks free.
Related