Chat system design
Design the backend that delivers chat messages between people in real time.
The problem
A chat app lets people send short messages to friends and to groups.
You type a message, press Send, and it shows on your friend’s phone a moment later.
If your friend’s phone is off, the message waits and shows when they come back.
Questions to ask the interviewer
What is in scope?
- Sending a message and showing it on the friend’s phone.
- Opening a chat shows its newest messages.
- Not in this problem: calls, photos and videos, accounts, encryption.
How fast must a message arrive?
Under 500 ms on average, for 99% of the messages, when the friend is online.
How many people use it at once?
- About 8,000 people have the app open at the same moment.
- Each one must hear about a new message at once, even when nothing happens for minutes.
How many messages?
- About 50 requests a second; 20% of them open a chat, the rest send a message.
- A message is short: about 100 characters.
What must never happen?
A message that never arrives. A slow message is bad; a lost one is worse.
What is the budget?
About $35 a month for the whole system.
Want to try it yourself first? Play, then come back for the approach.
PlayThe design in short
Chat feels instant to the user, so the backend has a hard job: it must tell a phone about a new message the moment it exists. A web page can wait to be asked. A chat app cannot.
The main trade-off is how a phone hears about news. It can ask the server every few seconds, or it can keep one connection open and let the server speak first. Asking is easy to build, but most of those questions come back empty, and the answer is only as fresh as the gap between them. An open line answers at once, but every person online now holds a connection on some server, all day, even while nothing happens. Count both costs before you choose.
People online is a number of its own. Requests a second tell you how busy the servers are. People connected now tell you how many connections they hold. A server can be calm on requests and still be full on connections. Size for both.
Pieces to place on the whiteboard:
- the machines holding the connections, and how many people each can keep
- what happens to a message when its friend is away, so it is still there when they come back
- how a chat shows its newest messages when someone opens it
Never lose a message. The brief says a slow message is bad and a lost one is worse. So a message must wait for a friend who is away and show up when they return, and you should think about what the sender sees if a step fails halfway.
Scale questions to ask yourself: what happens when the people online double, and which part fills first? Answer that for each part in turn. The game board lets you test each guess with real numbers.
Last updated 22 October 2026