Haystack
Composable pipeline framework for RAG and agent systems
Haystack is profiled here as a LLM tool for engineering teams. Read about features, pricing, and how it compares to related options in the tools directory.
Description
Haystack is an open-source AI orchestration framework from deepset, the Berlin company Milos Rusic, Malte Pietsch, and Timo Moeller founded in 2018. Pipelines assemble typed components such as retrievers, embedders, rankers, and generators into explicit graphs, which keeps RAG and agent systems debuggable as they grow. The 2.x rewrite shipped in 2024 with a stable component API and serialization format. deepset has developed the framework since 2020, well before the current LLM wave, and that history shows in its mature retrieval and evaluation tooling.
Key Capabilities:
Composable pipeline graphs with typed component connections
Retrieval integrations for Elasticsearch, OpenSearch, Qdrant, Weaviate, and pgvector
Agents with tool calling and loop control
Hayhooks for serving pipelines as REST APIs
Evaluation integrations for RAG quality measurement
Apache 2.0 license alongside the commercial deepset AI Platform
Alternative tools
- Jina AI
Search foundation models and web reading APIs
- LlamaIndex
Data framework connecting language models to private documents
- LiteLLM
Open-source gateway that speaks every LLM API
- OpenRouter
One API for hundreds of language models
- Cohere
Enterprise models for generation, embeddings, and reranking
- WhyLabs LangKit
Extract structured monitoring signals from LLM prompts and responses