Sacra Logo
Unlock Full Access Sign In
Your Feed

LiteLLM compromise pushes gateway adoption

Diving deeper into

Augusto Marietti, CEO of Kong, on the end of tokenmaxxing

Interview
they didn't trust LiteLLM for security reasons, it's a strong framework, but not built to enterprise security standards
Analyzed 5 sources

This is a distribution trust story as much as a product story. LiteLLM became popular because it was the easiest Python layer for calling many models through one interface, but the March 24, 2026 supply chain compromise turned that convenience into an enterprise liability, because a package installed inside internal systems could expose environment variables, API keys, and other secrets. That shifts the buying test from does it work for developers to can a security team approve it for production.

1 github 2 github 3 sacra 4 sacra
  • Kong sells the opposite architecture from a loose developer library. Its gateway sits in the path of every API and LLM call, so platform teams can enforce authentication, rate limits, logging, PII sanitization, and routing rules from one control plane instead of trusting each app team to wire those controls in separately.
    3 sacra 4 sacra 5 konghq
  • The security concern was not theoretical. The compromised LiteLLM versions 1.82.7 and 1.82.8 on PyPI were reported to include a credential stealing payload that executed on Python startup and could exfiltrate environment variables, cloud credentials, SSH keys, and database secrets. That is exactly the class of failure large enterprises try to remove from core infrastructure dependencies.
    1 github 2 github
  • This also explains why AI gateways are converging with classic API management. Once enterprises run many models across many teams, the gateway is no longer just a model switchboard. It becomes the chokepoint for policy, audit, cost control, and vendor approval, which favors vendors like Kong that already sell into platform and security teams.
    3 sacra 4 sacra 5 konghq

Going forward, open source proxies will keep winning early developer adoption, but the larger budget will move to managed and governed traffic layers. As AI agents make more calls into internal tools and data, the winner is likely to be the gateway that security teams can bless, procurement can standardize on, and platform teams can run across APIs, models, MCP servers, and agents from one place.

3 sacra 4 sacra 5 konghq