Langchain-Chatchat
Build offline RAG and Agent apps using Langchain and models like ChatGLM.
Offline-first RAG and Agent application framework for open-source LLMs.
Langchain-Chatchat lets you build private, local knowledge base question-answering and agent workflows using open-source models like ChatGLM, Qwen, and Llama. It is built for developers and organizations wanting to run entirely offline RAG applications using frameworks like Ollama, Xinference, and LocalAI.
What it is
It connects Langchain with local model inference frameworks to handle document processing, text splitting, vectorisation, and semantic search via FAISS and other databases. The 0.3.x release adds advanced Agent capabilities, supporting tool calling, web search, database querying, and multi-modal image chat.
Who it's for
It suits developers and IT teams who need a private, self-hosted RAG platform running on open-source weights without sending data to third-party APIs. It does not suit non-technical users looking for a plug-and-play hosted SaaS customer support bot without managing local model servers.
Key features
Local RAG Pipeline
Implements document loading, text splitting, vectorisation, and similarity search entirely offline.
Agent Tool Calling
Optimised for ChatGLM3 and Qwen to dynamically invoke web search, database queries, and custom tools.
Multi-Framework Support
Integrates with Xinference, LocalAI, Ollama, and FastChat for flexible model deployment.
Streamlit WebUI
Provides a built-in web interface with multi-session support and custom system prompts.
Up and running in 3 steps
Our honest verdict
What's great
- Enables completely offline, private knowledge base deployments.
- Supports modern open-source models like Qwen2, GLM-4, and Llama 3.
- Includes built-in support for File RAG, database chat, and multi-modal image tasks.
- Compatible with multiple model serving backends including Ollama and Xinference.
Where it falls short
- Requires significant hardware resources such as GPUs to host large language models locally.
- Demands technical proficiency to configure model serving frameworks, embeddings, and vector databases.
- Setup involves managing multiple dependencies across Python, FastAPI, and Streamlit.
Pricing & license
Free under its open-source license. The only cost is running your own hardware infrastructure and GPU servers required to host the language models.
Open remote AI-training roles
880 open remote AI-training roles, updated November 15, 2026; most pay $20–$175 an hour. Zekai lists them free; the application goes to the employer’s platform.
Alternatives
These reviewed AI tools do the same job instantly — from our scored directory.
Use it, or have us set it up
FAQ
Can I run this without an internet connection?
Yes, it is designed for completely offline and air-gapped environments using local models and embeddings.
Which model inference frameworks are supported?
It supports Ollama, Xinference, LocalAI, and FastChat for local inference, alongside OpenAI-compatible APIs.
