What it does
Every completion made through POST /v1/llm/complete is recorded as metadata: who called, which action label, which provider and model, tokens when the provider reported them, latency, retries, fallback and error class. GET /v1/llm/observability returns a tenant view and GET /admin/db/model-metrics returns the fleet view, grouped by model, action, provider, project or run.
How it works
The record type has no field for a prompt or a completion, so content cannot reach the store or the API by construction. A test drives a completion with a marker string in both prompt and answer and checks that the marker appears in no serialised field.
Tokens are present when the provider reports them and absent otherwise, and a missing count is a typed not-available state, never zero. Cost is derived from tokens and the built-in price table and is also not-available when tokens are. Latency is always measured at the call site. Cache, region and redaction have no source on this path and are reported as explicit not-available states. Retries count the providers that failed before the one that answered. A failed call is classified as timeout, rate_limit, provider_error, invalid_request, network, configuration or other, and the aggregates count each class.
History is an in-process ring of at most 4,096 calls per project, dropping the oldest. The tenant route takes its scope from the credential and ignores any project query parameter; the admin route may filter by project. The ?by= parameter accepts model, action, provider, project and run.
The model routing and incident response capabilities read the same aggregates.
Where you reach it
- HTTP
GET /v1/llm/observability?by=model|action|provider|project|run - HTTP
GET /admin/db/model-metrics?by=...&project=
Limits
- The /v1/llm/complete call site is the one instrumented. Completions made inside the agent loop, streaming runs, memory consolidation and evaluation runs are not recorded, so these metrics cover HTTP-originated calls.
- History is held in memory, is bounded, and starts empty after a restart.
- Cost is an estimate from a built-in price table; use token totals for budgeting.
- The tenant route needs a platform credential with the editor role, and the fleet route an admin role.
Evidence
- On this siteAI features catalogue
- Tests9 aggregation tests cover per-dimension sums, honest not-available states, bounded dimension strings and project isolation; 4 hook tests cover cost derivation from tokens, failed calls and the no-content property
Test coverage is listed as present in the engine repository at this edition. This page does not report test results.
KynetraDB is in private preview
Request a preview to talk through the limits above against your own workload.