---
title: Inspect AI Latency, Cost & Token Traces | APIkit
description: Pinpoint slow AI calls and unexpected token spikes with granular per-request trace logs and cost breakdowns.
canonical: "https://apikit.ai/j/inspect-latency-cost-token-traces"
modified: 2026-11-14
author: apikit.ai editorial team
source: "https://apikit.ai"
category: Quality Evals, Logs & Versioning
category_url: "https://apikit.ai/j#group-evals-logging-versioning"
---

# Debug slow AI calls with per-request latency and token traces

> Break down every endpoint request into gateway checks, context lookups, time-to-first-token, output tokens, and exact dollar cost.

## Prepared starting prompt

Some requests take 9 seconds while others finish in 600ms, and token costs crept up 30%. I want detailed trace spans for every endpoint call.

### Quick scenarios
- **Span-level timing:** See exact milliseconds spent on validation, context fetch, TTFT, and model generation.
- **Filter slow p99 calls:** Search logs instantly for calls exceeding 3 seconds, hitting retries, or failing JSON checks.
- **Exact cost per trace:** Inspect input tokens, cached tokens, output tokens, and USD cost on every single run.

- [Join APIkit early access](https://apikit.ai/#waitlist-f73a0d)

## Steps to diagnose slow or costly AI traces

1. **Separate TTFT from stream duration** — Check whether delays come from slow provider queuing or oversized multi-paragraph outputs.
2. **Filter traces by retry or failover** — Spot routes silently burning latency on JSON repair retries or primary model timeouts.
3. **Spot unexpected prompt token bloat** — Compare input token counts on outlier traces to catch unclipped user pastes or huge fetched rows.
4. **Check cache hit ratios per route** — Identify high-volume endpoints with low cache hits where minor key tweaks unlock big savings.

## How it works in practice

### Trace Filter: latency > 2500ms

trace_id=tr_9941a | endpoint=/v1/support-draft | status=200

### Span Breakdown & Root Cause

1. Auth & Rate Limit:    3ms
2. Postgres Context SQL: 18ms
3. Primary Model (Timeout): 2,500ms -> 529 Overloaded
4. Failover Model TTFT:  290ms | Gen: 610ms
Total: 3,421ms | Cost: $0.0024 | Tokens: 1,140 in / 148 out

Instead of guessing why p95 jumped, traces reveal the exact upstream timeout and failover path in seconds.

## Frequently Asked Questions

### Can I correlate APIkit traces with logs in my own backend?

Yes. Pass your own X-Request-Id or trace header on requests and APIkit indexes it alongside every internal span.

### Do trace logs capture whether a response came from edge cache?

Every trace marks exact cache hits, semantic hits, failover switches, and schema repair attempts clearly.

### How can I avoid storing sensitive user data in debug traces?

Combine trace metrics with automatic PII redaction or zero-payload logging mode on regulated endpoints.

## Related tasks in this category

- [Gate AI endpoint deployments behind automated quality evals](https://apikit.ai/j/run-automated-evals-before-deploy.md)
- [Compare prompt versions side by side on golden datasets](https://apikit.ai/j/compare-prompt-versions-golden-dataset.md)
- [Roll back faulty endpoint releases in a single click](https://apikit.ai/j/rollback-bad-endpoint-release-instantly.md)
- [Promote AI endpoints safely from staging to production](https://apikit.ai/j/promote-endpoint-staging-to-production.md)
- [Replay failed production payloads against draft fixes](https://apikit.ai/j/replay-failed-production-payloads.md)
- [Score response faithfulness to curb AI hallucinations](https://apikit.ai/j/score-hallucination-faithfulness-evals.md)

---

*Published by **Wondel.ai sp. z o.o.** · [apikit.ai](https://apikit.ai) · [About](https://apikit.ai/about) · [Editorial Guidelines](https://apikit.ai/editorial) · [Contact](https://apikit.ai/contact) · [Wondel.ai Products](https://wondel.ai/products)*