NVIDIA could absorb Gimlet stack
Gimlet Labs
This risk is really about control of the software layer around the chip. If NVIDIA can bundle routing, scheduling, and low level inference optimization into the default stack that customers already use, then Gimlet stops looking like a new control plane and starts looking like an extra layer. Gimlet still matters where workloads span AMD, Intel, Cerebras, d-Matrix, and private infrastructure, but that cross-vendor wedge has to become valuable before NVIDIA closes most of the gap inside its own ecosystem.
-
Gimlet is not just a model host. Its product breaks an agent workload into pieces, sends each piece to the most suitable chip, compiles fragments for each accelerator, and uses kforge to generate low level kernels across CUDA, ROCm, and Metal. That is powerful, but it is also exactly the kind of infrastructure layer a dominant accelerator vendor can absorb over time.
-
NVIDIA already offers much of the adjacent stack. Dynamo supports disaggregated serving, KV-aware routing, and multiple inference backends including TensorRT-LLM, vLLM, and SGLang. Run:ai adds dynamic GPU allocation and cluster level orchestration inside NVIDIA AI Enterprise. That means customers can get more of the scheduling and serving stack from one vendor contract and one supported platform.
-
The same pattern has shown up elsewhere. Modular is exposed to the same pressure from NVIDIA absorbing OctoAI and Run:ai, while Luminal and Kernelize are attacking similar heterogeneity and kernel portability problems from the startup side. In practice, that means Gimlet is racing both an expanding incumbent bundle and younger point solutions at the same time.
Going forward, the durable version of Gimlet is a company that becomes the best way to run mixed silicon, not just a better way to run inference. If heterogeneous datacenters keep growing because cost, supply, and specialized chips matter more, then the scheduler, compiler, and kernel stack become a core system layer. If not, the center of gravity keeps moving into the accelerator vendor bundle.