An open-source, offline-deployable RAG and Agent application project based on large language models such as ChatGLM, Qwen, and Llama, and the Langchain framework.

Langchain-Chatchat Project Detailed Introduction

Project Overview

Langchain-Chatchat (formerly Langchain-ChatGLM) is an open-source, offline-deployable RAG (Retrieval-Augmented Generation) and Agent application project based on large language models and application frameworks such as Langchain. The project is dedicated to establishing a knowledge base question answering solution that is friendly to Chinese scenarios and open-source models, and can be run offline.

• GitHub Address: https://github.com/chatchat-space/Langchain-Chatchat

Core Features

1. Multi-Model Support

The project supports mainstream open-source large language models on the market:

• GLM Series: GLM-4-Chat, ChatGLM, etc.
• Qwen Series: Qwen2-Instruct, Qwen-VL-Chat, etc.
• Llama Series: Llama3, etc.
• Online API: Supports OpenAI GPT API calls

2. Multiple Deployment Frameworks

Supports multiple model deployment frameworks, providing flexible access methods:

Framework	Xinference	LocalAI	Ollama	FastChat
OpenAI API Interface Alignment	✅	✅	✅	✅
Accelerated Inference Engine	GPTQ, GGML, vLLM, TensorRT, mlx	GPTQ, GGML, vLLM, TensorRT	GGUF, GGML	vLLM
Model Types Supported	LLM, Embedding, Rerank, Text-to-Image, Vision, Audio	LLM, Embedding, Rerank, Text-to-Image, Vision, Audio	LLM, Text-to-Image, Vision	LLM, Vision

3. Rich Dialogue Features

Version 0.3.x provides multiple dialogue modes:

Feature	0.2.x	0.3.x
LLM Dialogue	✅	✅
Knowledge Base Dialogue	✅	✅
Search Engine Dialogue	✅	✅
File Dialogue	✅ Vector Retrieval Only	✅ Unified as File RAG, supports BM25+KNN and other retrieval methods
Database Dialogue	❌	✅
Multi-Modal Image Dialogue	❌	✅ Recommended to use qwen-vl-chat
ARXIV Literature Dialogue	❌	✅
Wolfram Dialogue	❌	✅
Text-to-Image	❌	✅
Agent	❌ Unstable	✅ Optimized for ChatGLM3 and Qwen

Technical Architecture

RAG Implementation Principle

The core technology of the project is based on the RAG (Retrieval-Augmented Generation) architecture:

Document Loading → Text Reading → Text Splitting → Text Vectorization → Question Vectorization →
Similarity Matching (Top-K) → Context Construction → LLM Generates Answer

Agent Functionality

The core functionality of version 0.3.x is implemented by the Agent, providing three operation modes:

Operation Method	Implemented Function	Applicable Scenarios
Enable Agent + Multiple Tools	LLM Automatic Tool Calling	Models with Agent capabilities such as ChatGLM3/Qwen
Enable Agent + Single Tool	LLM Only Parses Tool Parameters	Models with general Agent capabilities, manual function selection
Disable Agent + Single Tool	Manual Parameter Input	Models without Agent capabilities

Installation and Deployment

1. Python Library Installation

pip install langchain-chatchat -U

pip install "langchain-chatchat[xinference]" -U

2. Environment Requirements

• Python Version: 3.8-3.11
• Operating System: Windows, macOS, Linux
• Hardware Support: CPU, GPU, NPU, MPS

3. Configuration Steps

Set Root Directory (Optional)

# Linux/macOS
export CHATCHAT_ROOT=/path/to/chatchat_data

# Windows
set CHATCHAT_ROOT=/path/to/chatchat_data

Initialize Project

chatchat init

Configuration File Modification

The main configuration files include:

• model_settings.yaml: Model configuration
• basic_settings.yaml: Basic path configuration
• kb_settings.yaml: Knowledge base configuration

Knowledge Base Initialization

chatchat kb -r

Start Service

chatchat start -a

4. Docker Deployment

docker pull chatimage/chatchat:0.3.1.3-93e2c87-20240829


docker pull ccr.ccs.tencentyun.com/langchain-chatchat/chatchat:0.3.1.3-93e2c87-20240829

Use Cases

1. Enterprise Knowledge Base Question Answering

• Document Management and Retrieval
• Internal Knowledge Sharing
• Customer Service Automation

2. Academic Research

• Literature Retrieval and Analysis
• Academic Paper Dialogue
• Research Material Organization

3. Personal Knowledge Management

• Personal Document Organization
• Learning Note Retrieval
• Data Archiving Management

Advantages and Features

1. Fully Open Source: Follows the Apache-2.0 protocol, code is completely open
2. Offline Deployment: Supports completely offline operation, protecting data privacy
3. Chinese Friendly: Optimized for Chinese scenarios, supports Chinese models
4. Flexible Framework: Supports multiple model deployment frameworks
5. Rich Functionality: Provides RAG, Agent, multi-modal and other functions
6. Easy to Deploy: Provides pip installation and Docker deployment methods

Technology Stack

• Core Framework: Langchain
• Web Framework: FastAPI, Streamlit
• Vector Database: FAISS, etc.
• Model Inference: Xinference, Ollama, LocalAI, FastChat
• API Access: One API supports multiple online APIs

This project provides a complete solution for users who need to build a local knowledge base question answering system, and is especially suitable for enterprises and individual users with strict data privacy requirements.