Enterprise Document Chatbot

Developed a RAG chatbot capable of answering queries from enterprise documents (PDF, DOCX). Integrated Ollama for local LLM inference, eliminating reliance on external APIs.

Ollama
FastAPI
ChromaDB
React
Docker
View on GitHub
Project Overview

This project demonstrates the implementation of a Retrieval-Augmented Generation (RAG) chatbot capable of processing and answering queries from various enterprise document formats including PDF and DOCX files. The solution leverages Ollama for local LLM inference, eliminating dependencies on external APIs while maintaining high performance and data privacy.

Key Features & Achievements
  • Developed a RAG chatbot capable of answering queries from enterprise documents (PDF, DOCX).
  • Integrated Ollama for local LLM inference, eliminating reliance on external APIs (OpenAI/Claude).
  • Containerized services with Docker Compose and exposed REST endpoints for ingestion, health, and query.
  • Delivered a full-stack solution (React frontend + FastAPI backend) demonstrating scalable and modular AI deployment.
Technical Architecture

backend

FastAPI with Python for REST API endpoints

frontend

React.js for user interface

database

ChromaDB for vector storage and retrieval

llm

Ollama for local language model inference

containerization

Docker Compose for multi-service orchestration

Technical Challenges
  • Implementing efficient document parsing and chunking strategies for optimal retrieval
  • Managing vector embeddings and similarity search with ChromaDB
  • Optimizing local LLM performance while maintaining response quality
  • Creating a robust Docker-based architecture for easy deployment