Requests and tokens
Compare input, output, cache and media usage by model. Separate provider-reported totals from estimates.
LLM observability
LLM cost tracking for every AI request.
Track requests, tokens, latency, failures and estimated provider spend in one workspace. Keep your existing backend, use the secure gateway, or combine both.
No card required1,000 requests/month freePrompts stay out of analytics
What you can measure
Costs with the context needed to act.
A total is useful only when you can explain it. HyperProxy keeps request, model, client and session dimensions connected.
Estimated spend
Apply the model catalog or your own price overrides. Unknown usage remains visibly incomplete instead of becoming a misleading zero.
Latency and errors
Use the same date range across cost, latency, status codes and request history to find what changed.
Clients and sessions
Group calls with the client and session identifiers your application supplies. HyperProxy never guesses the platform.
Budgets and alerts
Email at 80% and 100%, detect daily spend anomalies, or stop gateway traffic when recorded monthly spend reaches your budget.
Gateway or direct
Measure traffic routed through HyperProxy and server-reported events in the same project, with their sources clearly labelled.
Three integration paths
Start where your traffic already runs.
Secure gateway
Change the request origin and auth header. HyperProxy forwards the provider-native request and meters completed usage.
External events API
Keep direct provider calls in your backend. Submit one idempotent event after each completed call from a background worker.
OpenTelemetry
Point an OTLP/HTTP trace exporter at the project endpoint and send completed GenAI client spans.
Try analytics without a gateway
See a request in your dashboard before changing your app.
Send a sample event with cURL. You do not need a provider key, an SDK or a payment card to check the connection.
- Create a free account and a project.
- Open Overview → Set up analytics and create a key with Send external events only.
- Copy the project ID and key into the placeholders below. Run the command, then click Refresh metrics in the dashboard.
This example records sample metrics and uses one request from your monthly allowance. It does not call OpenAI or incur an AI provider charge.
When connecting your backend, replace the sample values with actual status, duration and token usage. Send one event after each completed call from a background worker. Provider keys, prompts and responses stay in your backend.
Terminal · macOS / Linux with uuidgen
PROJECT_ID='YOUR_PROJECT_ID'
INGEST_KEY='YOUR_INGEST_KEY'
EVENT_ID=$(uuidgen)
OCCURRED_AT=$(date -u '+%Y-%m-%dT%H:%M:%SZ')
curl --fail-with-body \
"https://app.hyperproxyai.com/api/v1/admin/projects/$PROJECT_ID/events" \
-H "Authorization: Bearer $INGEST_KEY" \
-H 'Content-Type: application/json' \
--data "{\"event_id\":\"$EVENT_ID\",\"occurred_at\":\"$OCCURRED_AT\",\"provider\":\"openai\",\"model\":\"gpt-4o-mini\",\"status_code\":200,\"duration_ms\":320,\"tokens_in\":1000,\"tokens_out\":500}"Accepted events return HTTP 201 with a request ID. A retry with the same event ID and payload returns 200 and does not count again. For retries, rerun only the cURL call with the saved variables, so the ID and timestamp stay the same. Read the production integration guide →
From usage to an estimate
How LLM cost tracking calculates a request.
Start with the exact provider and model ID, then apply rates to the usage that request actually reports.
Match the billing units
For token-based calls, input cost is input tokens × input rate / 1,000,000; output cost follows the same formula. Cache reads, cache writes, long-context tiers, tools and media can have separate rates and units.
Check the selected price
Open Model prices in your project. Inspect the selected source, review date, alternative estimates and billing units. Project-specific overrides take precedence when you have an agreed provider rate.
Keep unknown costs visible
As an illustrative calculation, 1,000 input tokens at $2 per million and 500 output tokens at $8 per million total $0.006. These are example rates, not a provider quote, and exclude additional meters. Missing usage or pricing stays incomplete.
To investigate a cost spike, compare model mix, output length, cache coverage and failed requests in the same period. For slower calls, follow the AI gateway latency guide; a request's duration does not isolate gateway overhead.
Clear boundaries
Know what the number means.
| Question | HyperProxy reports | Source of record |
|---|---|---|
| How much provider usage did the app record? | Requests, tokens and estimated model cost | HyperProxy analytics |
| What will the provider charge? | An operational estimate with coverage warnings | Provider invoice |
| What does HyperProxy itself cost? | Plan allowance and overage history | HyperProxy billing |
FAQ
LLM cost tracking questions.
Does HyperProxy replace provider billing?
No. It gives you an operational estimate tied to request history. Provider invoices remain the billing source of record.
Can I track costs without proxying prompts?
Yes. Use external events or OpenTelemetry from your backend. Those paths do not include provider credentials, prompts or response bodies.
What happens when pricing or usage is missing?
The request is marked as incomplete cost. Totals show recorded cost and a coverage warning, so an unknown request never looks free.
Related