Skip to content

Cache identical AI responses at the edge for instant hits

Serve repeated identical prompts in under 20 milliseconds while slashing token expenses to zero for deterministic endpoints.

Copy the prompt and paste it into the AI assistant you already use.

Checklist for high-hit exact response caching

  1. 01

    Use temperature zero on cacheable routes

    Set deterministic sampling so cached outputs match what a fresh model run would generate.

  2. 02

    Exclude volatile fields from keys

    Strip request_id or client timestamp headers from cache keys to maximize hit rates.

  3. 03

    Scope cache by endpoint version

    Include the published prompt version hash in keys so updates never serve stale formats.

  4. 04

    Inspect cache hit headers

    Verify X-Cache-Status hit vs miss headers and hit-ratio charts inside analytics.

How it works in practice

Cache Policy & Duplicate Requests

Key fields: [product_title, target_locale] Ignored fields: [trace_id] TTL: 7 days Req 1: {"product_title":"Oak Desk","target_locale":"de","trace_id":"a1"} Req 2: {"product_title":"Oak Desk","target_locale":"de","trace_id":"b9"}

Response Telemetry

Req 1 -> MISS | model: 640ms | $0.0012 | X-Cache: MISS Req 2 -> HIT | edge: 14ms | $0.0000 | X-Cache: HIT

Repetitive calls drop from 640ms and paid tokens to 14ms edge hits at zero inference cost.

Common questions

What happens when I update my endpoint prompt or schema?

APIkit incorporates your endpoint version into the cache key automatically, so newly deployed logic produces fresh responses right away.

Can I prevent user-specific private answers from leaking across tenants?

Yes. Include tenant_id or user_id inside the cache key definition whenever outputs depend on caller-specific context.

Do cached hits count against upstream model RPM caps?

Zero upstream load—cached hits resolve entirely inside APIkit edge memory without touching your LLM provider.