- Requests routed
- 1,525Every call that entered the router, whichever upstream took it.
- Routing decisions
- 1,525How many times the router chose an upstream for a request.
- Requests with a cached prefix
- 1,411Calls where a reusable prefix was found, so the upstream did not have to recompute it.
- Tokens read from cache
- 127,854,687Context served from a stored prefix instead of being recomputed from scratch.
- Fresh input tokens
- 39,051,470The part of the prompt that had to be processed cold.
- Output tokens
- 1,027,723What the model wrote back.
- Prefixes left stable
- 679Calls whose context prefix was already in the order that makes it cacheable.
- Prefixes reordered
- 846Calls whose prefix had to be reshuffled before it could be cached.
- Limit hits
- 10Times a request hit a provider limit and was retried against another upstream.
- Recovered after a limit
- 10Of those retries, how many ended in a successful answer.
- Circuit breaker opens
- 0Times an upstream was taken out of rotation after failing repeatedly.
- Total outages
- 0Times no upstream was available and the request could not be served.
- Blended cost
- $10.356380Total spend across all upstreams, after any failover.