Skip to content

Route simple and complex prompts to tiered models

Send short routine requests to sub-second lightweight models while escalating nuanced or long-context inputs to frontier models.

Copy the prompt and paste it into the AI assistant you already use.

Checklist for smart complexity routing

  1. 01

    Start with deterministic rules

    Route on payload length, plan tier, or explicit mode flags first before adding classifier hops.

  2. 02

    Compare eval scores across tiers

    Measure accuracy drop on your easy slice to prove compact models meet your quality bar.

  3. 03

    Keep schemas identical across tiers

    Ensure compact and frontier branches emit identical JSON keys so callers remain unaware.

  4. 04

    Monitor per-tier cost savings

    Track blended cost per 1,000 requests in logs to quantify real savings after enabling tiering.

How it works in practice

Visual Routing Rules

IF input.tokens > 2500 OR input.plan == "enterprise" -> route: frontier-tier ELSE -> route: fast-compact-tier

Cost & Latency Split (10k Calls)

Fast tier (82% traffic): avg 310ms, $0.18 / 1k calls Frontier tier (18% traffic): avg 1.4s, $2.10 / 1k calls Blended savings vs all-frontier: -68% spend

Dynamic routing slashes monthly token bills while preserving top-tier reasoning exactly where inputs warrant it.

Common questions

Do callers have to call two different URLs for fast vs smart models?

No. One unified endpoint inspects routing rules on the fly and invokes the optimal model tier transparently.

Can I route based on customer subscription tier?

Yes. Map JWT claims, API key metadata, or request tier fields directly inside visual routing branches.

How do I verify cheap models do not hurt quality?

Run built-in APIkit evals comparing compact vs frontier outputs on historical easy inputs before activating traffic rules.