C++ Chat Server Architecture: Acceptor-Worker, Rooms, Routing [#50-1]

Introduction

Building chat at scale raises questions:

• One io_context or Acceptor-Worker?
• How to manage 100 channels and route messages?
• How to stop one slow client from stalling everyone?
• Connection storms and rate limits?

Core ideas: Acceptor-Worker (separate accept from I/O work), room / channel maps, message routing (broadcast per room, optional DM), connection limits and bounded write queues.

Environment: C++17+, Boost.Asio 1.70+.

---

Problem scenarios (summary)

Symptom	Cause	Direction
Accept overload	Single thread doing everything	Dedicated accept + worker pool or io_context pool
Crash on broadcast	Concurrent modify of participant list	Strand per room or shared_ptr lifetime + synchronized structures
Global lag	Unbounded per-session write queue	Cap queue, drop or disconnect slow clients
Mixed channels	One global room	RoomManager keyed by channel id
OOM under spike	Unlimited async_accept	Max connections + backlog policy

---

Acceptor-Worker

• Acceptor thread (or strand) only async_accept and hands sockets to workers.
• Workers run sessions (async_read / async_write)—often N ≈ cores threads each running io_context::run(), or round-robin assign accepted sockets to pooled io_contexts via post.

Alternative: single acceptor + post next io_context in a pool for load spread.

---

Room management

• RoomManager: channel_id → shared_ptr<ChatRoom>.
• Each ChatRoom holds sessions (e.g. weak_ptr or shared_ptr), join/leave serialized (strand or mutex).
• Broadcast: iterate members; skip sender if needed; async_write per session with bounded queue—if queue full, policy: drop, disconnect, or sample.

---

Message routing

• Parse client messages (JSON, line protocol, etc.).
• Dispatch: ROOM_MSG → room broadcast; DM → single session lookup.

---

Connection pool / limits

• Global atomic<int> active; reject or delay accept when active ≥ max.
• Optional rate limiter on connect or messages.

---

Production

• Metrics: connections, messages/sec, queue depth, slow disconnects.
• Health checks; graceful shutdown; TLS termination (often at LB).

---

Related posts

• WebSocket
• Chat server basics

---

Summary

Acceptor-Worker + per-channel rooms + serialized membership + bounded writes is the backbone of scalable C++ chat servers. Validate under load; use strands or careful locking for room state.