Context Window

General definition

Every large language model has a context window, measured in tokens (roughly three to four characters per token in English). The window is the total amount of text the model can attend to at once: the system prompt, any conversation history, documents retrieved from a RAG pipeline, tool call results, and the model’s reply all count against the same limit.

Context window sizes have grown substantially: models in 2024 and 2025 commonly offer 128k to 1 million token windows, compared with 4k or 8k in earlier generations. A larger window allows more conversation turns, longer documents, and richer tool-call results to be included without truncation.

• Tokens that exceed the context limit are either truncated or require a chunking and summarisation strategy
• Processing a full context window is slower and more expensive than a short prompt because attention scales with sequence length
• Practical effective context (how well the model uses information deep in a long context) is often shorter than the advertised maximum
• RAG reduces context pressure by retrieving only the most relevant passages rather than loading entire documents

In the Ethora ecosystem

Ethora’s AI SDK and self-hosted LLM agent architecture are designed around configurable model endpoints. The context window of your chosen model determines how much conversation history and how many retrieved knowledge-base chunks you can include per turn, and you tune this per deployment rather than accepting a platform default.

In a healthcare or finance context, a larger context window allows a bot to hold a full patient intake or customer enquiry in a single pass, reducing the need for complex multi-turn state management. When running a self-hosted model, you also control the trade-off between context length, inference latency, and GPU memory, rather than paying per-token for the same capacity on a managed API.