Glossary
Document Chunking
Where RAG quality is usually won or lost. How documents are split into passages for embedding, the main strategies, the chunk size and overlap trade-offs, and why metadata travels with every chunk.
General definition
Document Chunking splits source documents into passages before they are embedded and indexed. It exists for two reasons. Embedding models accept a limited amount of text and produce one vector per input, so a 40-page PDF embedded whole becomes a single blurry point that matches everything and answers nothing. And the language model that writes the answer has a finite context window, so retrieval has to return a few precise passages rather than whole files. Chunking decides the unit of retrieval, and a poor choice cannot be fixed downstream by a better model.
- Fixed size: cut every N tokens or characters; simple and predictable, but splits sentences and tables mid-way
- Recursive: try to split on paragraphs, then sentences, then words until each piece fits the size limit; the common default in RAG frameworks
- Structure-based: follow headings, sections, list items, table rows or code blocks in Markdown, HTML and PDFs, so chunks match how the author organised the material
- Semantic: embed sentences and start a new chunk where the meaning shifts, producing variable-length chunks that stay on one topic at the cost of more compute
- Parent-child: retrieve on small chunks for precision, then hand the model the larger parent section for context
Chunk size is a trade-off between precision and context. Small chunks (100 to 300 tokens) match specific questions well but lose the surrounding meaning; large chunks (800 tokens and up) carry context but dilute the embedding and crowd the prompt. Many pipelines land between 200 and 600 tokens, with an overlap of 10 to 20 percent so a sentence that straddles a boundary appears in both neighbours. The right number depends on the embedding model’s input limit, the kind of question users ask and how many chunks the answer prompt can hold, so it should be set by measuring retrieval quality on real questions rather than by convention.
Every chunk should carry metadata: source URL or file, title, section heading, page number, last-modified date and who is allowed to see it. Metadata makes citations possible, lets retrieval filter by tenant, date or permission, and lets an index be refreshed document by document instead of rebuilt. A newer refinement, often called contextual chunking, prepends a short description of the document and section to each chunk before embedding, so a passage that says “the limit is 5,000” also knows which product and plan it describes. Parsing comes before all of this: tables, scanned pages and multi-column layouts break naive text extraction, and a RAG pipeline is only as good as the text its chunker receives.
In the Ethora ecosystem
In Ethora, chunking is part of the built-in ingestion in the AI SDK. Content enters through the website crawler, uploaded files and per-agent knowledge with live web indexing, is split into passages, embedded as 256-dimension vectors and stored in Postgres with pgvector, and each agent retrieves from its own knowledge base before it replies in a room or channel. The integrator supplies the sources; the pipeline handles the splitting and indexing.
The practical advice for teams loading knowledge into an agent follows from the definition. Give the crawler clean, well-structured pages with real headings rather than one long scroll; prefer text documents to scanned images; keep one topic per page or file; and keep a date or version in the content so the agent can be re-indexed when a policy or price changes. In a self-hosted deployment the chunked text and its vectors never leave your database, which is the point for clinical, financial and insurance material.
