ragnostic
A production-grade Retrieval-Augmented Generation (RAG) pipeline for querying LangChain documentation in natural language.
I built ragnostic because RAG is one of the most popular techniques in the AI Engineering landscape. However, I’ve seen this concept applied many times without a proper evaluation pipeline, leading to suboptimal results, incoherent outputs, and lack of transparency. In this case, this project does ingest the LangChain documentation, although it can be easily adapted to other sources.
ragnostic uses a llm as a judge to evaluate the quality of the generated responses. It provides four different metrics: faithfulness, answer relevance, context precision and context recall.
It also features a Cross-Encoder reranker built with fastembed to enhance the relevance of the retrieved documents.
The project uses the following dependencies:
- streamlit: web interface for the application
- fastapi: web framework for the application
- qdrant: vector database for storing and retrieving documents
- langchain: framework for building RAG pipelines
- httpx: HTTP client for making async requests
- ragas: evaluation framework for RAG pipelines
- fastembed: Cross-Encoder reranker for enhancing document relevance
- langfuse: observability platform for monitoring and debugging RAG pipelines