Glossary
OpenAI-Compatible API
The de facto standard for talking to language models. One request shape, served by Ollama, LM Studio, vLLM, gateways and most cloud providers, so an application is not tied to any single vendor.
General definition
An OpenAI-Compatible API is any model server or provider that accepts requests in the shape OpenAI defined for its own API and returns responses in the same shape. It is not a formal standard; it became one because OpenAI published its format first, the official client libraries let you change the base URL, and every serving tool found it easier to copy the format than to invent a new one. A client sends a bearer token in the Authorization header, a model name and a list of messages with roles, and reads back a completion, optionally as a stream of server-sent events.
/v1/chat/completions: the main endpoint; takesmessages(system, user, assistant, tool),model, sampling parameters such astemperature, optionaltoolsfor tool calling, andstream: truefor token-by-token output/v1/embeddings: turns text into vectors for search and RAG; the response carries one embedding per input/v1/models: lists the model names the server can serve, which is how a client discovers what is available/v1/completions: the older prompt-in, text-out endpoint, still implemented for legacy clients
On the self-hosted side, Ollama, LM Studio, the llama.cpp server, vLLM, SGLang and Hugging Face Text Generation Inference all expose it. Gateways and routers such as LiteLLM and OpenRouter present it in front of many providers, and most hosted providers (Mistral, Groq, Together, DeepSeek, Google’s Gemini endpoint, Azure OpenAI with its own URL and header conventions) offer it natively. OpenAI itself has since added a newer Responses API, but chat completions remains the common denominator that everything else speaks.
The value is decoupling. An application written against this interface can move from a cloud model to a local one, or from one vendor to another, by changing a base URL, a key and a model name, which is the whole premise of an LLM gateway. The limits are in the corners: parameter support varies (JSON mode, log probabilities, vision inputs), tool-calling quality depends on the model more than the protocol, embedding dimensions differ between models so an index built with one cannot be queried with another, and rate limits and error codes are provider-specific. Treat the format as a stable contract and test each model behind it.
In the Ethora ecosystem
Ethora’s AI service targets exactly this interface. Agent replies, RAG over a knowledge base and tool calls in the AI SDK are all requests to an OpenAI-compatible endpoint. In the official Ollama example, moving a deployment from a cloud model to a local one is four environment variables: AI_API_URL, AI_API_KEY, AI_CHAT_MODEL and AI_EMBEDDING_MODEL. Nothing in the agents, rooms or widgets changes.
The endpoint is set once per deployment, and each agent then chooses a model by name from what that endpoint serves. That matches how the serving tools work: a vLLM or Ollama host, or a gateway in front of several providers, lists its models and the agents pick from the list. For RAG the embedding model behind /v1/embeddings must return 256-dimension vectors, because that is the width of the pgvector store, so switching embedding models means re-indexing the knowledge base. The self-hosted LLM AI agent page covers the full setup.
