Rate limiter system design
Design a rate limiter for our public API.
The problem
An API lets other programs read and write your data: apps, websites and scripts call it.
A call looks like this: https://api.weather.example/v1/forecast?city=Lisbon
Most clients call now and then. A client can also call far too often, by mistake or on purpose.
Questions to ask the interviewer
What do we limit, and by what?
- The calls of each client, whatever they ask for.
- The free plan allows 20 calls a second per client.
What does a client see when it is over the limit?
An answer at once: 429, too many requests. The call does no other work.
How many calls?
- About 1,500 calls a second; 90% read data. One client sends about 5,000 calls a second.
- Every call must take under 200 ms on average.
How big is a counter?
- One count and a time for each client: about 100 bytes.
- 100M clients × 100 B ≈ 10 GB: one machine’s memory holds it.
Must the limit be exact?
Close is fine: a client that gets a few calls more does no harm. One that gets many times its limit does.
What is the budget?
About $135 a month.
Want to try it yourself first? Play, then come back for the approach.
PlayThe design in short
A rate limiter protects an API from its own clients. Most of them call now and then. One greedy script can call thousands of times a second and use up the servers that everyone else needs. The limiter's job is to say no to that script, early and cheaply, while everyone else gets through.
Say no before the work. The whole point is lost if a call over the limit still reaches the app servers and the database. So the check belongs in front of them, where every call passes first. A refused call should cost almost nothing: a 429 answer (too many requests) and no other work.
A limit needs a count. For each client you keep a number and a time: about 100 bytes. The brief does the math for you: all clients together fit in one machine's memory. Reading and updating that count happens on every single call, so it must be fast. Think about what one check costs.
Things to place and size:
- the box in front of the API that checks every call
- the app servers and data behind it, sized for the calls you let in, not for the calls you refuse
- what the work behind the check looks like: how many calls read data, and how many write
The main trade-off is exact against cheap. The brief says close is fine: a few calls more do no harm, many times the limit does. So do not pay for a perfect count. Say what "close enough" means for this API, and why.
Reason about scale by splitting the traffic. There are the honest calls you must serve, and the extra calls you must refuse. Size the work behind the check for the first group. Size the check itself for both, because every call passes it.
Put the check on the game board and see for yourself which calls get through.
Last updated 22 October 2026