LiteLLM
Open-source gateway that speaks every LLM API
LiteLLM is profiled here as a LLM tool for engineering teams. Read about features, pricing, and how it compares to related options in the tools directory.
Description
LiteLLM is an open-source LLM gateway from BerriAI, the Y Combinator company Ishaan Jaff and Krrish Dholakia founded in 2023. The Python SDK and proxy server translate 100+ provider APIs into the OpenAI format, so application code stays the same while models change underneath. Platform teams deploy the proxy to issue keys, set budgets, and meter usage across an organization. The proxy ships as a Docker image with Postgres-backed key management, and an enterprise build adds SSO, audit logs, and support contracts.
Key Capabilities:
OpenAI-format translation for 100+ model providers
Self-hostable proxy server with virtual keys and per-team budgets
Load balancing, retries, and fallback routing across deployments
Response caching for repeated requests
Logging callbacks to Langfuse, Datadog, Prometheus, and other backends
Python SDK plus an admin UI for key management
Alternative tools
- Jina AI
Search foundation models and web reading APIs
- Haystack
Composable pipeline framework for RAG and agent systems
- LlamaIndex
Data framework connecting language models to private documents
- OpenRouter
One API for hundreds of language models
- Cohere
Enterprise models for generation, embeddings, and reranking
- WhyLabs LangKit
Extract structured monitoring signals from LLM prompts and responses