Cost Per Completed Task Matters
Etched
This changes the buying unit from raw model output to finished work. In short chat sessions, a customer can compare vendors on cost per token. In long running agents, the bigger cost is everything wrapped around each token, including waiting for long prompts to load, keeping memory live across many steps, and reusing prior context instead of recomputing it. That is why lower latency and better context handling can matter more than the headline token price.
-
An agent does not just answer once. It reads a long prompt, calls tools, loops through results, and writes more tokens over minutes rather than seconds. That makes prefill speed, decode speed, and memory reuse part of the unit economics of one completed job.
-
The bottleneck is often memory, not arithmetic. NVIDIA highlights that KV cache grows with model size, batch size, and context length, and that cache reuse avoids expensive recomputation. Etched is built around the same pain point, but at rack level through shared memory and one system for both prompt processing and generation.
-
This is also why a hardware vendor can expand into software and managed inference. Cerebras already sells inference through pay per token APIs and enterprise hosted capacity, turning fast hardware into recurring revenue. If Etched proves lower cost per completed task, the same path is open.
As AI shifts toward coding agents, research agents, and document workflows, the winning systems will be the ones that finish more jobs per rack hour, not the ones with the cheapest listed token. That favors architectures that keep long context hot in memory, move quickly between prompt processing and generation, and package hardware with serving software.