📚 RAG Chat with Documents

A complete Retrieval-Augmented Generation system built from scratch — no frameworks, no black boxes. Every line of code is teaching material.

🧠 Python 🔢 Ollama Embeddings 💾 Chroma Vector DB 🌐 FastAPI 🤖 GLM-5.2 Cloud

🏗️ Architecture — How RAG Works

INGEST (one-time setup): documents/ → loader.py → chunker.py → embedder.py → chroma_db/ PDFs/TXT Load text Split into Convert to Vector DB + metadata ~500 char 768-dim (persistent) chunks w/ vectors via overlap Ollama ASK (per question): Question → embedder.py → retriever.py → generator.py → Answer + Citations "What is Convert question Search Chroma Send context + "Revenue is revenue?" to vector for top 5 question to Rs 45 lakhs (same model) similar chunks GLM-5.2 LLM (Source: report)"

📦 The 7 Modules

1
Loader
loader.py
Reads PDFs (PyMuPDF) and text files. Keeps page metadata for citations.
2
Chunker
chunker.py
Splits documents into ~500 char pieces with 50 char overlap to preserve context at boundaries.
3
Embedder
embedder.py
Converts text → 768-dim vectors using Ollama nomic-embed-text. Stores in Chroma.
4
Retriever
retriever.py
Embeds the question, searches Chroma for top 5 most similar chunks by meaning.
5
Generator
generator.py
GLM-5.2 Cloud answers FROM retrieved context with source citations. No hallucination.
6
CLI Interface
main.py
Terminal commands: ingest documents, ask questions, interactive mode.
7
Web App
web_app.py
FastAPI browser UI with beautiful chat interface and streaming responses.

🧠 Key Concepts for Students

Why chunking? A 50-page PDF as one vector = a "blur" — close to everything, specific to nothing. Breaking into ~500 char chunks makes search precise. Each chunk covers one topic, so "revenue" chunks are separate from "hiring" chunks.
Why overlap? If you cut at exactly 500 chars, you might split a sentence: "...revenue is based on" | "subscription fees...". The answer is at the boundary. Overlap (50 chars) ensures boundary info survives in both chunks.
Why embeddings? Computers can't "search" text. Embeddings convert text to numbers where similar MEANING = similar numbers. "Revenue" and "money" have similar vectors even though they're different words. This is semantic search.
Why "answer only from context"? Without this instruction, the LLM mixes its training data with your documents and hallucinates. The system prompt forces it to answer from retrieved chunks only — and cite sources. This is what makes RAG reliable.
Why same embedding model for questions and docs? If documents use model A and questions use model B, the vectors live in different "spaces" — similarity search is meaningless. Same model = same space = meaningful search.

🚀 Quick Start

# 1. Clone the repo git clone https://github.com/nagarjunak-pixel/rag-chat-docs.git cd rag-chat-docs # 2. Set up virtual environment python3 -m venv venv source venv/bin/activate pip install chromadb pymupdf requests fastapi # 3. Make sure Ollama is running with embedding model ollama pull nomic-embed-text ollama serve # if not already running # 4. Add your documents # Put .pdf or .txt files in the documents/ folder # 5. Ingest — embed documents into vector DB (one-time) cd app python main.py ingest # 6. Ask questions (CLI) python main.py ask "What is the revenue model?" # 7. Or launch the web app python web_app.py # Open http://localhost:8000 in browser

📋 Tech Stack

Embedding Model
Ollama nomic-embed-text — runs locally, free, produces 768-dimensional vectors
Vector Database
Chroma — file-based, zero infrastructure, stores vectors + metadata + text together
LLM
GLM-5.2 Cloud via OpenAI-compatible proxy — no API key needed, no rate limits
PDF Parsing
PyMuPDF (fitz) — fast, reliable, page-level text extraction with metadata
Web Framework
FastAPI — async, fast, built-in interactive API docs at /docs
Frontend
Single HTML file with CSS/JS — no React, no build step, easy for students to modify