Glossary
RAG Pipeline
RAG as an engineering system rather than a diagram. The stages from raw documents to a cited answer, where quality is won or lost, how to measure it, and which parts can run locally.
General definition
A RAG Pipeline is the full set of steps that make retrieval-augmented generation work in production. The idea of RAG is simple: look up relevant text, then let the model answer from it. The pipeline is where the simplicity ends, because each step has choices that decide whether answers are accurate, current and traceable. It has an indexing side that runs when content changes and a query side that runs on every question.
- Ingest: collect sources (web pages via a crawler, uploaded PDFs and documents, tickets, wikis, databases) and convert them to clean text with their metadata: source, title, section, date, who may see it
- Chunk: split text into passages sized for the embedding model and the answer, with overlap and structure awareness; see document chunking
- Embed: turn each chunk into a vector with an embedding model; the same model must be used at query time
- Store: write vectors plus metadata to a vector store such as Postgres with pgvector, with indexes for fast nearest-neighbour search
- Retrieve: embed the question, fetch the top-k nearest chunks, often combined with keyword search (hybrid retrieval) and filtered by metadata and permissions
- Rerank: score the candidates with a cross-encoder or a small LLM and keep the best few, which usually improves precision more than any other single change
- Generate: assemble a prompt from the system instructions, the chosen chunks and the conversation, within the model’s context window, and produce the answer
- Cite: return the sources used so the reader can verify, and so the system can be audited
Evaluation is what separates a demo from a product. Retrieval is measured on its own (did the right chunk appear in the top-k, using metrics such as recall and mean reciprocal rank against a labelled question set) and generation is measured on faithfulness to the retrieved text, answer relevance and citation accuracy, usually with a mix of human review and LLM-as-judge scoring. Most quality problems trace back to the indexing side: bad parsing of PDFs and tables, chunks that cut across sections, stale indexes, or permissions not enforced at retrieval so a user sees passages they should not.
Every stage can be hosted or local. A cloud embedding API and a managed vector database are quick to start; a local embedding model, pgvector in your own Postgres and a self-hosted LLM keep documents inside your perimeter, which is the usual requirement in healthcare and finance. The two sides can mix, but the embedding model is a fixed choice per index: changing it means re-embedding everything, so pick it with the deployment target in mind.
In the Ethora ecosystem
Ethora ships a RAG pipeline as part of the AI SDK rather than leaving it to the integrator. The ingestion side is a website crawler plus uploaded files, with per-agent knowledge and live web indexing so each agent can have its own sources; the storage side is Postgres with pgvector holding 256-dimension embeddings; and the generation side is whatever OpenAI-compatible model endpoint the deployment is configured with. An agent attached to a room or channel retrieves from its own knowledge before answering, and the MCP server exposes the knowledge base so Claude and other MCP clients can manage it.
In a self-hosted deployment the whole pipeline stays on your infrastructure: the crawler, the embedding model and the chat model can all be local, as the official Ollama example shows, with AI_EMBEDDING_MODEL set to a model that returns 256-dimension vectors. Because retrieval happens inside a chat platform, the answer also inherits the platform’s controls: the agent’s reply is a message in a room, subject to Trust and Safety moderation, the message archive and the audit trail like any other.
