Why Rate Limiting Is the Cheapest Insurance Your API Will Ever Buy | Blog

# Why Rate Limiting Is the Cheapest Insurance Your API Will Ever Buy

TL;DR: API rate limiting is a handful of lines of middleware or a gateway config that stops one buggy client, one scraper, or one runaway retry loop from taking down a service that serves everyone else — and the three common algorithms (fixed window, sliding window, token bucket) trade off implementation simplicity against burst tolerance and memory cost, so pick based on what kind of traffic pattern you actually need to survive.

The failure mode this actually prevents

Rate limiting rarely gets attention until an incident makes the cost of skipping it obvious. A useful recent example: on September 12, 2025, Cloudflare's dashboard and a set of related APIs went down for about an hour. The trigger wasn't an attacker — it was a bug in a React useEffect dependency array that caused the dashboard to call an internal Tenant Service API far more often than intended, on every re-render. The Tenant Service sits in the authorization path for API requests, so when it buckled under the unplanned load, authorization checks failed and requests started returning 5xx errors across the board. Part of the fix was installing a global rate limit on the Tenant Service so a single noisy caller (in this case, Cloudflare's own dashboard) couldn't exhaust it. Cloudflare's engineering blog has the full timeline (Cloudflare, 2025).

That pattern — a well-behaved-looking client accidentally hammering a shared resource — is more common than malicious traffic in practice. It's also the exact scenario OWASP tracks as API4:2023 Unrestricted Resource Consumption in the API Security Top 10, the category that replaced the older "Lack of Resources & Rate Limiting" entry. The advice hasn't changed much: without caps on request volume, payload size, or execution time, any client — buggy, scraping, or actively hostile — can degrade the service for everyone else, or run up your infrastructure bill in the process (OWASP, API Security Top 10).

Rate limiting doesn't require anticipating every failure mode. It requires one property: no single caller can consume unbounded resources. That's cheap to build and expensive to have skipped.

Three algorithms, three trade-offs

There's no single "correct" rate limiting algorithm — the right one depends on whether you care more about simplicity, memory footprint, or burst smoothing.

Algorithm	How it works	Strength	Weakness
Fixed window	Count requests in a discrete interval (e.g., every 60s); reset the counter at the boundary	Trivial to implement, one counter per key	Boundary burst: a client can send 2x the limit across a window edge (e.g., last second of one window + first second of the next)
Sliding window	Track requests over a continuously moving time range, either via a weighted blend of two fixed windows or a log of timestamps	Smooths out boundary bursts; more accurate	Sliding window log is memory-intensive at scale (one entry per request); the weighted-counter variant is an approximation
Token bucket	A bucket holds up to B tokens, refilled at rate R per second; each request consumes a token; requests are rejected when the bucket is empty	Allows short bursts up to bucket capacity while enforcing a steady average rate; O(1) state per key	Burst allowance can mask a sustained attack if capacity is set too high

Fixed window

Simplest to reason about, and fine for coarse limits like "100 requests per user per day" where a boundary edge case isn't worth the complexity. It's a poor fit for anything where burst amplification at a window edge matters — login-attempt throttling, for instance, where an attacker deliberately timing requests around the reset can nearly double their effective rate.

Sliding window

The sliding window log (store every request timestamp, count how many fall in the last N seconds) is the most accurate of the three but costs memory proportional to request volume per key. A common middle ground is the sliding window counter: blend the current and previous fixed window counts, weighted by how far into the current window you are. It's an approximation, not exact, but it's O(1) memory and removes most of the boundary-burst problem.

Token bucket

This is the closest thing to a default recommendation for public APIs because it matches how real traffic behaves: mostly steady, occasionally bursty (a client retrying a batch of requests after reconnecting, a mobile app syncing on launch). You get to allow that burst without permanently raising the sustained rate. It's also the algorithm behind widely deployed tools like the Linux tc traffic shaper and most cloud API gateways' default limiter.

Token bucket in Node.js

Here's a minimal, dependency-free token bucket you could drop into an Express middleware. It refills lazily (computing elapsed tokens on each request rather than running a timer), which avoids a setInterval per client key:

class TokenBucket {
  constructor({ capacity, refillRatePerSec }) {
    this.capacity = capacity;
    this.refillRatePerSec = refillRatePerSec;
    this.buckets = new Map(); // key -> { tokens, lastRefill }
  }

  consume(key, cost = 1) {
    const now = Date.now();
    let bucket = this.buckets.get(key);

    if (!bucket) {
      bucket = { tokens: this.capacity, lastRefill: now };
      this.buckets.set(key, bucket);
    }

    const elapsedSec = (now - bucket.lastRefill) / 1000;
    const refilled = elapsedSec * this.refillRatePerSec;
    bucket.tokens = Math.min(this.capacity, bucket.tokens + refilled);
    bucket.lastRefill = now;

    if (bucket.tokens < cost) {
      return { allowed: false, retryAfterMs: ((cost - bucket.tokens) / this.refillRatePerSec) * 1000 };
    }

    bucket.tokens -= cost;
    return { allowed: true };
  }
}

const limiter = new TokenBucket({ capacity: 20, refillRatePerSec: 5 });

function rateLimitMiddleware(req, res, next) {
  const key = req.ip; // swap for API key / user ID in production
  const result = limiter.consume(key);

  if (!result.allowed) {
    res.setHeader('Retry-After', Math.ceil(result.retryAfterMs / 1000));
    return res.status(429).json({ error: 'Too many requests' });
  }
  next();
}

This works for a single process. In-memory buckets don't survive a restart and don't share state across horizontally scaled instances — each pod would enforce the limit independently, effectively multiplying your real limit by pod count. For anything running more than one instance, back the bucket with Redis instead. rate-limiter-flexible (v11.2.0 on npm as of this writing) implements token bucket, sliding window, and fixed window against Redis, Memcached, or plain Postgres, and is a reasonable default rather than hand-rolling the Lua script yourself:

const { RateLimiterRedis } = require('rate-limiter-flexible');
const Redis = require('ioredis');

const redisClient = new Redis();

const limiter = new RateLimiterRedis({
  storeClient: redisClient,
  points: 20,        // max requests
  duration: 4,        // per 4 seconds -> ~5 req/sec sustained, matching the example above
  keyPrefix: 'rl',
});

app.use(async (req, res, next) => {
  try {
    await limiter.consume(req.ip);
    next();
  } catch {
    res.status(429).json({ error: 'Too many requests' });
  }
});

If you'd rather not manage the algorithm at all, express-rate-limit (v8.6.2 on npm) covers fixed-window and sliding-window-counter strategies out of the box and is the more common starting point for a straightforward Express app that doesn't need bespoke burst handling.

Where to enforce it, and what to return

Enforcing limits at the application layer is fine for a single service, but if you're running several services behind a shared entry point, pushing rate limiting to an API gateway or reverse proxy (nginx, Envoy, Kong, or a managed gateway) means every backend inherits the protection without reimplementing it. This is one of the design decisions worth getting right early — retrofitting consistent rate limiting across a fleet of services after an incident is far more expensive than building it into the gateway layer from the start. It's the kind of architectural call we help clients think through in API integration work when a system is growing past a single service.

Whatever layer enforces the limit, return 429 Too Many Requests, not a generic 500, and include a Retry-After header so well-behaved clients know when to come back instead of retrying immediately and making things worse. There's also an IETF draft standard — draft-ietf-httpapi-ratelimit-headers, currently at revision 11 as of May 2026 and still in Standards Track review, not yet a published RFC — that defines RateLimit and RateLimit-Policy response headers so clients can see their quota before they hit it. It's not universally adopted yet, but it's worth using the same header names it proposes since tooling is starting to expect them.

FAQ

What's the difference between rate limiting and throttling?

They're often used interchangeably, but rate limiting typically means rejecting requests over a threshold (a hard 429), while throttling can mean deliberately slowing responses down to stay under a limit without outright rejecting the request. Most production systems implement rate limiting; throttling is more common in client SDKs that self-regulate their own outbound call rate.

Should rate limits be per-IP or per-API-key?

Per-API-key (or per-authenticated-user) whenever you have auth, because IP-based limits break down behind NAT or shared proxies where many legitimate users share one IP, and they're trivial to evade with rotating IPs. Use IP-based limiting as a coarser, secondary layer against unauthenticated abuse (login endpoints, signup forms).

What HTTP status code should a rate-limited request return?

429 Too Many Requests, defined in RFC 6585, with a Retry-After header telling the client how long to wait. Returning 503 or a generic error instead makes it harder for clients to distinguish "you're overloaded, back off" from "something broke."

Does rate limiting protect against DDoS attacks?

Partially. Application-level rate limiting stops resource exhaustion from excessive legitimate-looking request volume from a given key, but a large-scale volumetric DDoS is usually mitigated at the network/CDN layer before it ever reaches your rate limiter. Treat rate limiting as one layer of defense against abuse and bugs, not a substitute for DDoS mitigation infrastructure.