Evidlake
Menu

Rate Limiting

Also known as: throttling, request quota, HTTP 429.

Capping how many requests a client may make in a period. Servers apply it to protect themselves; responsible clients also impose it on themselves.

Updated Last reviewed

Synthetic dataExamples on this page use synthetic data created for illustration.

Definition

Rate limiting caps how many requests a client may make in a given period. Servers apply it to protect themselves and signal it with the HTTP 429 status, and responsible clients also limit themselves, spacing their requests so that data collection never becomes a burden on the source.

In plain terms

A limit is usually “N requests per window” or a daily or monthly quota. A common mechanism is the token bucket: a bucket holds up to a fixed number of tokens, refills at a steady rate, and every request takes one. A full bucket allows a short burst; after that the client is held to the refill rate.

When a server says “too many requests” (status 429), it may also say how long to wait (Retry-After). A polite client obeys, slows down, and does not wait for the server to push back before spacing its requests.

Example

A token bucket with room for 5 tokens that refills at 1 token per second.

MomentTokens availableWhat the client can do
Start5Send 5 requests at once (bucket now empty)
After 1 s1Send 1 request
After 10 s of steady sendingabout 0One request per second

In the first 10 seconds the client can send at most 5 + 10 = 15 requests, and over any long period no more than one per second on average.

Common misunderstandings

  • “A limit is a target to use up.” It is a ceiling; staying well below it is usually kinder and more reliable.
  • “After a 429, retry straight away.” That makes the problem worse. Wait as told, then back off (see Exponential Backoff).
  • “Limits only exist to stop abuse.” They are a normal part of shared services and apply to every client.
  • Exponential Backoff

    Waiting longer after each failed attempt, usually doubling the delay and adding randomness, so retries relieve a struggling service instead of piling on.

  • Circuit Breaker

    A guard that stops calling a failing service after repeated errors, waits, then tests carefully before resuming.

  • robots.txt

    A plain-text file at a site's root in which the owner states which paths automated clients are asked not to fetch. A convention, not a lock.

References

  1. M. Nottingham, R. Fielding. RFC 6585: Additional HTTP Status Codes (429 Too Many Requests) . IETF, 2012.
  2. R. Fielding, M. Nottingham, J. Reschke. RFC 9110: HTTP Semantics . IETF, 2022.