Portkey
AI gateway with routing, guardrails, and prompt management
Portkey is profiled here as a Prompt Management tool for engineering teams. Read about features, pricing, and how it compares to related options in the tools directory.
Description
Portkey is an AI gateway and LLM operations platform founded in 2023 by Rohit Agarwal and Ayush Garg. It sits between an application and hundreds of model providers, adding fallbacks, caching, guardrails, and cost tracking through one integration. The gateway core is open source, and the hosted platform layers prompt management and organization-wide observability on top of it. Gateway configs are versioned objects, so routing behavior changes ship without redeploying application code.
Key Capabilities:
Unified API across hundreds of models with automatic fallbacks and load balancing
Simple and semantic caching to cut latency and spend
Prompt versioning, templating, and a collaborative playground
Input and output guardrails with PII redaction
OpenTelemetry-based logs, traces, and cost analytics
Virtual keys with budgets and rate limits per team
Alternative tools
- LangChain
The standard open-source framework for LLM applications
- Freeplay
Prompt management, evals, and observability for product teams
- DSPy
Declarative framework for programming and optimizing LLM pipelines
- MLflow
Track experiments, manage models, and evaluate LLM applications across the full ML lifecycle
- Langtrace
Trace LLM application calls with OpenTelemetry and route data to any observability backend
- Opik by Comet
Trace, evaluate, and monitor LLM applications across the full development lifecycle