Applied Compute Router Cuts Costs 47%

This result shows that routing can turn inference from a raw compute bill into a software optimization problem. Instead of paying Sol prices on every Claygent request, Applied Compute trained a smaller Qwen based router to score each query, decide when a cheaper model is good enough, and keep expensive frontier usage for the hard cases. That shifts value from owning the best model to controlling traffic, evals, and serving economics.1 2 3 4

• The practical workflow is simple. Claygent used to send every request to one user chosen model. The router now sits in front, inspects each query, predicts expected quality across candidate models, applies a cost penalty, and forwards the request to the cheapest model that still clears guardrails.1 2
• This matters because Claygent runs at very high volume. Applied Compute said the router handles hundreds of millions of queries per month, and Claygent has processed more than 5 billion queries. At that scale, even small per request savings compound into a large recurring infrastructure budget win.1 3
• The competitive angle is that routing is becoming a standard layer across inference platforms and gateways. OpenRouter sells it as orchestration for many providers, and Kong applies similar logic inside enterprise AI gateways. Applied Compute is pushing the same idea deeper into customer workflows by pairing routing with custom evals and dedicated serving on AC2.5 6 7

The next step is broader budget capture. If Applied Compute can keep proving that its routers preserve answer quality while cutting spend, it can expand from custom model training into always on inference control, where the prize is recurring production traffic rather than one off research projects.4 7