RAG Support Agent

An agentic support assistant that answers from docs and live account data, running entirely on local models.

Type
AI agent · Open source
Role
Design and implementation
Context
Open source on GitHub
Links
GitHub repository
Support Assistant chat interface with suggested questions

01Overview

A support-chat system that can answer both kinds of question a customer asks: "how does the credit system work?" and "how many credits do I have left?". The first comes from documentation; the second comes from that customer’s own data.

Everything runs locally. The language model and the embedding model are served by Ollama on the same machine, so there are no API keys and no per-request cost.

02Architecture

ClientReact + TypeScriptChat UI · source chips
POST /api/chatFastAPISession memory · binds user_id
LocalOllamaLLM + embeddings, no API keys
LlamaIndexAgent workflowChooses tools per question
Tool · knowledge_baseVector indexBuilt from Markdown docs
4 × account toolsScoped by ContextVarModel can’t choose the user
SELECT onlyPostgreSQLRead-only database role
Request flow: the React UI posts to FastAPI, which runs a LlamaIndex agent backed by local Ollama models. The agent either queries a vector index built from Markdown docs, or calls four account tools scoped to the current user that read PostgreSQL through a read-only role.

03What I built

  1. Markdown files in a knowledge folder are indexed into a persisted vector index. Its query engine is wrapped as a tool so the agent decides when searching the docs is actually useful.
  2. Four read-only account tools (overview, credit activity, billing history and support tickets) query PostgreSQL, giving the agent access to the customer’s live data.
  3. A FastAPI backend exposes the agent over REST with per-session conversation memory, and returns the answer alongside the source files and tools it used. The React and TypeScript UI renders markdown answers with source chips and suggested questions.
PythonLlamaIndexFastAPIPostgreSQLOllamaReactTypeScriptVite

04Engineering notes

01The model can’t pick whose data it sees

The request’s user ID is bound into a ContextVar by the backend, and the account tools read it from there. The model chooses which tool to call, but it never controls the user scope, so a prompt can’t talk it into another customer’s account.

02Read-only by construction

Account tools connect as a dedicated database role that only has SELECT privileges. Safety is enforced by Postgres, not requested in a system prompt.

03Tool choice instead of a fixed pipeline

Rather than retrieving on every turn, the agent chooses between the knowledge base and account tools, and can combine both in one answer. The response reports which sources and tools were used, so answers stay inspectable.

→Outcome

A compact reference for agentic RAG done carefully: retrieval and live data in one assistant, with permissions enforced outside the model.

Next projectEmail Verifier

→Get in touch

Building something
interesting?

hi@ahmadthedev.com