Realtime Models Bypass Fish Audio

Diving deeper into

Fish Audio

Company Report
OpenAI's Realtime API and other native speech-to-speech models can collapse the ASR-LLM-TTS pipeline into a single model call and bypass modular TTS providers such as Fish Audio
Analyzed 9 sources

Native speech to speech shifts voice AI from a parts market into a bundled runtime market. In the old stack, a developer picked one vendor for transcription, one model for reasoning, and one vendor for voice output. Realtime systems now handle listening, reasoning, and speaking inside one low latency session, which removes an integration point where Fish Audio could win on voice quality alone and makes distribution and end to end ownership more important.1 2 3 4 5

  • OpenAI now offers realtime models that take audio in and return audio out over WebRTC, WebSocket, or SIP. That means a builder can ship a live voice agent without calling a separate TTS provider for every response, especially when default voices are good enough.1 2 5
  • Deepgram shows the intermediate version of this shift. Its Voice Agent API still exposes configurable listen, think, and speak components, but wraps them in one session and can even route speak to providers like ElevenLabs, Cartesia, or OpenAI. The control layer moves up, and the standalone component becomes easier to swap.3 4 6
  • Fish Audio is still meaningful where customers care about a specific cloned voice, multilingual expressiveness, or a branded voice library. But when the buyer is an agent platform or contact center team optimizing for latency, uptime, and total session cost, the bundled speech runtime becomes the more natural purchasing unit.7 8 9

The next step is voice vendors either moving up into full agent runtimes or specializing harder in premium voices that bundled models do not match. As realtime platforms improve reasoning and speech quality together, more spend will consolidate around the provider that owns the live conversation loop, not just the last step of turning text into sound.1 3 5 7 9