LiteLLM
Open-source gateway that speaks every LLM API
LiteLLM is profiled here as a LLM tool for engineering teams. Read about features, pricing, and how it compares to related options in the tools directory.
Description
LiteLLM is an open-source LLM gateway from BerriAI, the Y Combinator company Ishaan Jaff and Krrish Dholakia founded in 2023. The Python SDK and proxy server translate 100+ provider APIs into the OpenAI format, so application code stays the same while models change underneath. Platform teams deploy the proxy to issue keys, set budgets, and meter usage across an organization. The proxy ships as a Docker image with Postgres-backed key management, and an enterprise build adds SSO, audit logs, and support contracts.
Key Capabilities:
OpenAI-format translation for 100+ model providers
Self-hostable proxy server with virtual keys and per-team budgets
Load balancing, retries, and fallback routing across deployments
Response caching for repeated requests
Logging callbacks to Langfuse, Datadog, Prometheus, and other backends
Python SDK plus an admin UI for key management
Alternative tools
- Kong AI Gateway
AI traffic governance built on the Kong gateway
- Eden AI
One API for many AI models across providers and modalities
- Reka
Multimodal models that reason across text, image, and video
- Moonshot AI
Open-weight Kimi models tuned for agentic and coding tasks
- xAI Grok
Grok model family with real-time search through one API
- LLM Guard
Open-source security toolkit for LLM interactions