Serverless GPUs via Idle Time Arbitrage

This is a cloud margin business built around idle time arbitrage. Customers like Suno or Runway do not want to reserve GPUs all day for traffic that comes in bursts, so Modal buys access to large GPU pools, starts workers only when requests arrive, bills by the second, and keeps the difference between its blended supply cost and what application teams will pay for convenience, autoscaling, and faster deployment.1 2 3 4

• The concrete savings come from avoiding always on capacity. Modal bills GPU, CPU, and memory by the second and scales functions to zero when idle. Its own pricing examples frame serverless as cheaper for spiky demand than reserving fixed GPUs around the clock.1 2
• This model depends on pooled supply. RunPod customers describe choosing serverless GPU vendors based on per second pricing, availability across many GPU sizes, and the ability to start on request and shut down after each job. That is the same economic pattern Modal is packaging for inference teams.5
• The main alternatives package the same raw compute differently. Replicate wraps serverless GPUs in a big model marketplace and simple API calls. RunPod leans into cheap capacity and dashboard control. Modal leans into Python native workflows, cold start speed, and programmable infrastructure that also extends into sandboxes.3 5 6

The next step is further bundling. As inference, training jobs, and agent sandboxes all run on the same underlying fleet logic, the winning platforms will be the ones that turn bursty GPU and container supply into a full developer cloud, with one control plane for serving models, running jobs, and executing agent work.4 7