Kong AI Gateway
AI traffic governance built on the Kong gateway
Kong AI Gateway is profiled here as a LLM tool for engineering teams. Read about features, pricing, and how it compares to related options in the tools directory.
Description
Kong AI Gateway is a set of AI-specific plugins that run on Kong Gateway, the open-source API gateway from Kong, to put governance and routing in front of language model traffic. It exposes one entry point for many model providers, and because the plugins inspect the prompts that pass through, they can apply rate limits, caching, and security to AI calls at the network edge. The gateway runs self-hosted or through Kong's Konnect control plane, giving platform teams central control over how an organization uses models.
Key Capabilities:
A single entry point that proxies many model providers
Prompt-aware plugins for rate limiting, caching, and security
AI metrics and OpenTelemetry traces for model traffic
Prompt templates and transformations applied at the gateway
Deployment options across self-hosted, hybrid, and Kubernetes
Central governance through the Konnect control plane
Alternative tools
- Eden AI
One API for many AI models across providers and modalities
- Amazon Redshift
Cloud data warehouse for large-scale analytics on AWS
- Agno
High-performance framework for building and running agents
- smolagents
Lightweight agent library built around code-writing agents
- GraphRAG
Graph-based retrieval-augmented generation from Microsoft Research
- Reka
Multimodal models that reason across text, image, and video