Etched's Co-Designed Inference Rack

Etched is selling a new unit of AI infrastructure, not a faster part. The point of bundling silicon, boards, cooling, interconnect, and software is to remove the messy handoff points where normal GPU clusters lose speed, power efficiency, and reliability. In practice, the customer buys a rack that is already wired, cooled, and scheduled to behave like one inference computer, which matters most for frontier models that swing between memory heavy prefill and latency sensitive decode.

• This is the same product boundary Nvidia has moved toward with NVL72. Its rack ships with compute trays, NVLink switch trays, power shelves, liquid cooling, and software as one preconfigured system. Etched is using the same rack scale logic, but with a purpose built inference architecture instead of a general GPU platform.
  2 nvidia 3 nvidia 4 nvidia
• The reason to co design the whole rack is that inference bottlenecks sit between components as much as inside the chip. Etched says its shared memory layer and proprietary interconnect make the rack act more like one large machine. That is especially useful for long prompts and mixture of experts models, where data movement across chips can dominate token generation speed.
  1 sacra 5 etched
• Competitors are converging on different versions of the same idea. Groq pairs its custom silicon with GroqCloud and on prem GroqRack. SambaNova competes with a turnkey enterprise stack. The fight is no longer chip versus chip, it is complete system versus complete system, with the winner defined by delivered tokens per second, power draw, and deployment pain.
  6 sacra 7 sacra 8 sacra

The inference market is heading toward appliance style buying, where customers choose a full machine tuned for a narrow workload instead of assembling parts themselves. That favors companies that can make hardware, networking, thermal design, and runtime software work as one system, and it pushes standalone chip vendors toward deeper vertical integration.