Pinning 100% of enterprise traffic to frontier models (claude-sonnet-4 / gemini-2.5-pro) inflates inference spend by 5x–15x on routine lookups, ticket classification, and repeated FAQ paraphrases. Conversely, running a heavy neural classifier or dense bi-encoder inside the gateway hot path adds 15–40ms of CPU/GPU latency and multi-hundred-MB container bloat.
< 50µs across reasoning depth, code/math token density, constraint markers, and multi-hop compositionality.D = 256 Weinberger feature hashing with numeric & negation guards).score < 0.48 speculatively to the fast tier (gemini-2.5-flash / vllm-qwen-32b) and escalate to frontier iff upstream confidence < min_confidence or RAG faithfulness < 0.75, recording honest dual-invocation FinOps costs whenever escalation triggers.p50 gateway overhead with strict zero-leakage tenant cache partitioning.