---
title: Enforce Per-Request AI Token Budgets | APIkit
description: Protect unit economics by enforcing per-request token budgets, smart truncation, and max output limits at the gateway.
canonical: "https://apikit.ai/j/cap-per-request-token-budgets"
modified: 2026-11-14
author: apikit.ai editorial team
source: "https://apikit.ai"
category: Multi-Model Routing & Fallbacks
category_url: "https://apikit.ai/j#group-multi-model-routing"
---

# Cap per-request token budgets inside routing rules

> Prevent massive accidental documents from blowing out per-call spend by enforcing strict input token limits and max-output caps.

## Prepared starting prompt

Users occasionally paste 80-page logs into our endpoint, spiking costs. I need strict input token caps and model-tier budget limits per request.

### Quick scenarios
- **Input token ceilings:** Reject or truncate oversized inputs above 4,000 tokens before invoking costly models.
- **Tight max_tokens output:** Clamp maximum generated tokens per endpoint so runaway completions never happen.
- **Route huge inputs cheaper:** Direct unexpectedly large payloads to long-context low-cost models automatically.

- [Join APIkit early access](https://apikit.ai/#waitlist-f73a0d)

## Checklist for tight per-request token budgets

1. **Calculate real unit cost limits** — Pick max input and output token counts that keep every single call profitable under your pricing.
2. **Choose reject vs truncate policy** — Return clear HTTP 413 payload errors for API partners or smart tail-trimming for UI widgets.
3. **Prevent JSON truncation bugs** — Ensure max_tokens leaves ample margin over expected JSON payload sizes to avoid broken brackets.
4. **Monitor token distribution p99** — Review p95 and p99 token histograms to tighten over-generous ceilings safely.

## How it works in practice

### Token Guardrail Policy

max_input_tokens: 3000 (on_exceed: "route_cheap_long_context" up to 12000, else HTTP 413)
max_output_tokens: 450
max_cost_usd_per_call: 0.015

### Blocked & Rerouted Examples

Call A (1,100 in): Primary model -> $0.004 OK
Call B (8,400 in): Rerouted -> cheap long-context -> $0.006 OK
Call C (65,000 in): Blocked at gateway -> 413 Payload Too Large ($0.00 spent)

One runaway user paste can no longer generate a $0.80 inference bill or timeout upstream models.

## Frequently Asked Questions

### How accurately are input tokens counted before calling the model?

Fast local tokenizers approximate token counts at the gateway in microseconds before sending the payload to upstream providers.

### What happens when a caller needs more tokens on paid tiers?

Bind different budget ceilings to plan-tier metadata so enterprise keys enjoy higher limits than free accounts.

### Does capping output tokens risk cutting off JSON?

APIkit warns you if configured max_tokens sits too close to observed output length and routes long completions cleanly.

## Related tasks in this category

- [Configure automatic LLM failover across providers](https://apikit.ai/j/configure-automatic-provider-failover.md)
- [Route simple and complex prompts to tiered models](https://apikit.ai/j/route-simple-vs-complex-prompts.md)
- [Split live endpoint traffic to A/B test AI models](https://apikit.ai/j/split-traffic-ab-test-models.md)
- [Load-balance AI traffic across multiple keys and pools](https://apikit.ai/j/balance-rate-limits-across-providers.md)
- [Minimize AI response latency with regional routing](https://apikit.ai/j/optimize-endpoint-latency-by-region.md)
- [Pin exact model snapshots to stop silent drift](https://apikit.ai/j/pin-model-snapshots-prevent-drift.md)

---

*Published by **Wondel.ai sp. z o.o.** · [apikit.ai](https://apikit.ai) · [About](https://apikit.ai/about) · [Editorial Guidelines](https://apikit.ai/editorial) · [Contact](https://apikit.ai/contact) · [Wondel.ai Products](https://wondel.ai/products)*