Together AI Per-Token Pricing Model

Together AI’s key move was turning raw GPU rentals into a usage based product that felt like software, not infrastructure. Instead of asking a startup to reserve clusters and eat the cost when engineers were idle, it let them send prompts to an API, pay by tokens used, and get access to open models and training workflows without managing capacity planning or GPU scheduling themselves.1 2 3

• CoreWeave and Lambda made money by selling GPU time and long reservations, which fits teams running big training jobs on fixed clusters. In customer interviews, those buyers cared about price per GPU hour, InfiniBand quality, and 12 to 18 month commitments, not simple API billing.2 3
• Together sat one layer higher. It bundled open model access, inference, and training behind an API, which matched startup demand better because inference and experimentation are bursty. That let a small team try Llama or Mistral, launch a feature, and pay only when users actually generated tokens.1 4
• The tradeoff is margin and control. Together’s research shows about 45 percent gross margin versus roughly 80 percent for GPU clouds like CoreWeave, because it absorbs more of the orchestration work while buying underlying capacity from infrastructure providers.1 2

The market is moving toward convergence. GPU clouds are adding more managed software and inference products, while API platforms are pushing downward into scheduling and performance control. The winners will be the ones that make expensive GPUs feel cheap, instant, and easy to consume for each new model or application launch.3 4