Embedding Model

General definition

An Embedding Model is a neural network trained to map a piece of text to a list of numbers, called an embedding or vector, so that texts with similar meaning produce vectors that are close together. Unlike a chat model it does not generate words; it takes an input and returns, say, 768 numbers. Those numbers are what a vector database indexes and what semantic search compares with a distance measure such as cosine similarity.

• Open models you can run locally: nomic-embed-text, the BGE family from BAAI (including multilingual bge-m3), Microsoft’s e5 models, and small MiniLM models where speed matters most
• API models: OpenAI text-embedding-3-small and text-embedding-3-large, plus equivalents from Cohere, Voyage and Google
• Dimensions: typically 384 to 3,072. More dimensions carry more nuance but cost more storage and search time; models trained with Matryoshka representation learning (nomic-embed-text v1.5, text-embedding-3) can be truncated to fewer dimensions with little loss
• Multilingual models map the same meaning in different languages to nearby vectors, which matters for chat products with users in several countries

Three rules of thumb. Index and query with the same model: vectors from different models are not comparable, so changing the embedding model means re-embedding the whole corpus. Mind the input limit: most embedding models accept 512 to 8,192 tokens, so long documents are chunked first. And benchmark on your own data; the MTEB leaderboard is a useful shortlist, not a verdict.

In a RAG pipeline the embedding model runs twice: once over every chunk of the knowledge base when it is indexed, and once over each user question at query time; the nearest chunks are then handed to the chat model as context. Because the embedding model sees every document and every question, it decides whether private data leaves your network. A local embedding model is the usual first step toward a fully self-hosted AI stack, and it is cheap: embedding models are small and run well on CPUs.

In the Ethora ecosystem

Ethora’s RAG knowledge base (website crawler, uploaded files, per-agent knowledge with live web indexing) stores its vectors in Postgres with pgvector at 256 dimensions, and the AI service reads those vectors from any OpenAI-compatible embeddings endpoint. The model is chosen with AI_EMBEDDING_MODEL, one of the four environment variables (with AI_API_URL, AI_API_KEY and AI_CHAT_MODEL) that configure the self-hosted AI agent stack. Models that support shortened Matryoshka outputs, such as nomic-embed-text v1.5 or OpenAI’s text-embedding-3, can produce 256-dimension vectors.

The official ethora-ollama-example runs the embedding model locally in Ollama next to the chat model, so document indexing and every user question stay inside the network. The endpoint is service-wide: all agents share one embeddings endpoint and one embedding model, which also keeps the vector store consistent, since every chunk in pgvector is embedded the same way. Change the model and the knowledge base must be re-indexed, as with any RAG system.