Gimlet Mixed-Silicon Inference Runtime

This is a bet that the datacenter, not the individual chip, becomes the real product. Gimlet is not just helping customers pick between NVIDIA, AMD, Cerebras, or d-Matrix. It is operating the control layer that breaks one agent workload into pieces, sends each piece to the cheapest or fastest silicon for that step, and makes the whole job look like one inference service instead of six incompatible hardware islands.

• The practical advantage is at the workflow level. Prefill, decode, retrieval, reranking, and tool calls do not stress hardware in the same way. NVIDIA Dynamo already separates prefill and decode across devices inside GPU fleets. Gimlet extends that logic across different chip families, which is a much bigger control problem and a larger wedge if it works.
  1 sacra 3 nvidia
• Most inference clouds still optimize mostly within one hardware class. Fireworks focuses on low latency model serving in the cloud, and Baseten Chains lets each workflow step choose its own hardware and autoscaling policy. Gimlet goes further by owning mixed silicon capacity itself, so it can route work based on real datacenter level supply, power, and performance tradeoffs.
  1 sacra 5 sacra 6 baseten
• The closest alternative is vertical integration from a single chip vendor. Groq is building a full chip plus cloud plus agent stack around its own LPU system, while Modular focuses on portable compilation across diverse hardware. Gimlet sits between those models, acting like the runtime that can make many vendors useful together instead of asking customers to commit to one ecosystem.
  1 sacra 4 sacra 7 groq 8 sacra

If this model keeps working, inference infrastructure will look less like renting GPUs and more like buying an outcome, fast agent responses at a target cost and power budget. That would push the market toward schedulers, compilers, and datacenter operators that can turn mixed hardware supply into one dependable service layer.