Horizontal Scaling

General definition

Horizontal Scaling, or scaling out, grows capacity by running more instances of a service side by side, usually behind a load balancer, so that load is spread across them. Vertical scaling, or scaling up, keeps one instance and gives it more CPU, memory or faster disks. Scaling up is simpler and has no coordination cost, but it hits a ceiling at the largest machine available and leaves a single point of failure. Scaling out has no fixed ceiling and gives redundancy as a side effect, at the price of having to coordinate state across instances.

Stateless services scale out cleanly: any instance can serve any request, so a load balancer can distribute traffic and an autoscaler can add or remove instances on CPU or request metrics. Stateful services need more design:

• Persistent connections. A chat user holds one WebSocket or XMPP connection to one specific node for the life of the session. Load balancers must support long-lived connections, and messages for that user must be routed to the node that holds them
• Presence and routing. Nodes must share who is online and where. Chat servers such as ejabberd form a cluster in which each node knows about the sessions on the others; other designs use a shared registry or a message broker
• Group fan-out. A room with members on several nodes means every message crosses the cluster, so inter-node traffic grows with room size and activity
• Data. A database scales for availability with replica sets and for write capacity with sharding, which partitions data by a key such as app or room
• Media. Call media servers scale by adding SFU instances and assigning each call to one of them

Capacity planning for chat is about concurrent connections and message throughput per node, not requests per second. Those numbers come from load testing with realistic patterns (many idle connections, bursts in busy rooms, reconnection storms after a deploy) rather than from vendor sheets. The related goal of high availability falls out of the same design: if the system already runs on several nodes, losing one is a capacity event rather than an outage.

In the Ethora ecosystem

Ethora’s backend separates the pieces that scale differently. The Node.js API layer is stateless and runs as multiple instances behind a load balancer. The XMPP layer runs on ejabberd, which is built on Erlang and clusters natively, so nodes share session and presence state and a user can be routed to any node. MongoDB runs as a replica set for availability and can be sharded when write volume requires it, and SFU media servers for calls are added as call volume grows. Each customer App is an isolated container of users, rooms and files, which gives a natural partition key.

On the managed cloud that scaling is Ethora’s job. On a dedicated or self-hosted deployment the same architecture ships as Docker containers with Grafana and Prometheus monitoring and load-testing tools, so your team can measure a node’s real connection and message limits and add capacity on your own schedule. The self-hosted chat server page covers the deployment shape.