What a rate limit actually counts

Every rate limit has three parts: a key, a window and a threshold. The key is whose requests are being counted. Teams tune the window and the threshold and leave the key at its default.

That default is almost always the IP address. It is the easiest option: it is always available and requires no application logic.

Which is exactly why it is the weakest. Changing an IP is cheap for an attacker, and your real users often share one.

The first is a false positive. Thousands of subscribers sit behind a mobile carrier’s NAT. A shared threshold means an ordinary user is blocked having done nothing.

The second is a miss. A modern attack arrives from thousands of residential proxies at one or two requests per second each. Every address stays under the threshold while the aggregate takes the service down.

Both failures have one cause: an IP address belongs to a network, not to an attacker. What you want to govern is behaviour.

In practice there is not one key but several, working together. The most precise is the account identifier: for a signed-in user, changing it is expensive.

Next is the session or device identifier: it works for unauthenticated flows, though it is easy to clear. Third is the network block — not one IP but a whole subnet or autonomous system.

The fourth key is the one most often forgotten: the action itself. "How many SMS have gone to this phone number" is a rate limit too, and it holds regardless of who is sending.

There should be no single threshold. Fetching a static file, running a search and sending an SMS do not cost the same, so holding them to one number makes no sense.

A worked example: a static page weighs 1×, a search 10×, an SMS 100×. The budget is spent by that weight, so a user may open a hundred pages but run ten searches.

This matters most where the cost is external: SMS, mail, third-party APIs. There the limit is financial rather than technical.

A request over the threshold does not have to be refused. Slowing it is usually enough: the answer is delayed by 200 milliseconds and a real user does not notice.

For a script it is fatal, because its entire advantage was speed. A delay makes the attack pointless while, when it is wrong, costing you nothing more than a pause.

The steps build in order: slow down, then require a check, then refuse. Each is applied on the result of the one before it.

A fixed window is the simplest scheme and the easiest to defeat. If the counter resets every minute, an attacker sends double the threshold across the boundary — the last second of one minute and the first of the next.

So a sliding window or a token bucket is used instead. The bucket has an extra benefit: it permits a short burst while holding the average rate.

The difference is visible in practice. Loading a page fires dozens of requests — a normal burst, and a limit that cuts it breaks the site.

A rate limit works when the decision is separated from the enforcement. The decision is taken in application context; enforcement happens as low and as cheaply as possible.

If the limit runs inside the application, you still pay for the connection, the handshake and reading the request. At volume the limit itself becomes the load.

The second requirement is a shared counter. Without one, an attacker simply lands on a different instance each time and multiplies the threshold by the number of servers.

Three tests are enough. First: one account, requests from ten different IPs. If the threshold fires, the key really is the account.

Second: one IP, ten different accounts. If nothing is blocked, you are not punishing users for sharing a network.

Third: one request per second from a hundred sources. If nothing registers, your limit is source-based only, and a distributed slow attack walks straight through.