Cross-Stack Inference Optimization

The key point is that inference speed is becoming a systems engineering problem, not just a chip or model problem. Wafer is selling the work of finding hidden bottlenecks across the whole serving path, from the runtime and scheduler to parallelism settings and GPU specific kernels. The published AMD results matter because they show large gains came from different fixes on different workloads, which is exactly what makes this hard to standardize inside one default stack.

• The workflow is concrete. A team brings an open model and real traffic patterns, then Wafer tests serving engines like vLLM and SGLang, quantization schemes, batching and routing policies, kernel swaps, and hardware choices until it finds the cheapest setup that still preserves model behavior.
  1 sacra 2 nvidia 3 nvidia
• The AMD case shows why one knob is rarely enough. Wafer reported about 80% of Nvidia B200 performance on GLM-5.2 with MI355X at less than half the cost, but other gains came from ROCm path fixes, custom kernels, topology changes, and scheduler tuning, which means each model and traffic mix breaks in a different place.
  1 sacra 4 stackfutures 5 amd
• This also explains the competitive split. Fireworks AI packages a managed inference cloud, while Nvidia is pushing Dynamo and TensorRT-LLM deeper into scheduling, routing, and kernels. Wafer sits in the gap as a cross stack optimizer, especially where customers want non Nvidia hardware or workload specific tuning that upstream defaults still miss.
  2 nvidia 6 sacra 7 sacra 8 nvidia

The market is moving toward automated inference tuning as a standard layer between models and hardware. As Nvidia, AMD, and open source runtimes absorb more baseline optimizations, the durable edge will come from faster search across heterogeneous stacks, better workload specific data, and the ability to make cheaper chips perform close enough to premium ones for production use.