"Add rate limiting" is a one-line ticket that hides three different problems. Someone is hammering the login route from one IP: that's abuse, and you want it stopped before it costs you compute. An integration partner is calling the API too fast for their plan: that's fairness, and you want to slow them down and tell them why. A user is trying to place a thousand orders a minute: that's a business rule, and you want the application to say no in a way the product understands.
Each problem has a layer where it's cheap and correct to solve, and a layer where it's expensive or wrong. This is how we split it for a public API on AWS, with the numbers, the code, and the one place we deliberately don't rate limit.
The three layers
The edge sees IPs, paths and headers, and nothing about who the user is. It's the cheapest place to drop a request, because it happens before any compute you pay for, and the bluntest, because everyone behind a corporate NAT shares an IP. Its job is abuse: credential stuffing, scrapers, a misconfigured client in a retry loop.
The gateway sees an API key, if you issue them, and can enforce a rate per key. It knows which client is calling, not which user, and not what the request means. Its job is fairness between integrators: nobody's runaway script degrades the API for everybody else.
The application sees the authenticated user, their plan, the tenant they belong to and what the request is trying to do. It's the only layer that can say "you've placed 50 orders this hour, your plan allows 50, try again at 15:00". Its job is business limits, and it's the only layer that can return an error the product can explain.
Layer 1: the WAF rate-based rule
We covered the WAF configuration separately; the rate rule is the part of it that has stopped real attacks. Two rules, because one limit for the whole site is wrong:
// 2,000 requests per 5 minutes from one IP, across everything
{ name: 'rate-all', priority: 40,
statement: { rateBasedStatement: { limit: 2000, evaluationWindowSec: 300, aggregateKeyType: 'IP' } },
action: { block: { customResponse: { responseCode: 429 } } } },
// 100 per 5 minutes on authentication routes, where a "user" makes maybe 5
{ name: 'rate-auth', priority: 41,
statement: { rateBasedStatement: { limit: 100, evaluationWindowSec: 300, aggregateKeyType: 'IP',
scopeDownStatement: { byteMatchStatement: { fieldToMatch: { uriPath: {} }, positionalConstraint: 'STARTS_WITH', searchString: '/api/auth/', textTransformations: [{ priority: 0, type: 'LOWERCASE' }] } } } },
action: { block: { customResponse: { responseCode: 429 } } } },
Two details. The custom response makes the block a 429 rather than WAF's default 403, so clients that understand rate limiting behave correctly. And the window is five minutes with a limit of 100 on auth, which a real user never approaches and a credential-stuffing script hits in the first ten seconds. In six months this rule has blocked three such attempts and one scraper. It costs $1 a month.
What it can't do: distinguish a hundred users behind one office IP from one attacker. When a customer's whole company got blocked from login because they'd all arrived at 9:00, the fix wasn't raising the limit; it was adding a scope-down that excludes their known egress range, which is a two-line change and a conversation with their IT team.
Layer 2: API Gateway usage plans, and why we don't use them
If your API sits behind API Gateway and you issue keys to integrators, usage plans are the right tool: a burst and a steady-state rate per key, enforced by the gateway, returning 429 with no code written. It's what we'd use for a public API product with paying integrators on tiers.
We don't have that shape. Our API is called by our own front-ends and by a handful of partners, and it runs on App Runner, not behind API Gateway. Adding a gateway just for rate limiting would add a hop, a cost ($3.50 per million requests, more than the WAF) and a 29-second timeout. So the "fairness between clients" job moved into the application layer, keyed by the partner's identity rather than by an API key. If we had fifty integrators instead of five, the arithmetic would flip and we'd put the gateway in.
Layer 3: the application
The application limiter is keyed by whatever the business rule is about: the user, the tenant, the partner, sometimes the resource. It's a token bucket in DynamoDB, because that's a store we already have, it's atomic, and it's cheap at our request rate. One item per key, refilled on read:
// lib/rate-limit.ts
export async function take(key: string, plan: { capacity: number; refillPerSec: number }): Promise<Allow | Deny> {
const now = Date.now() / 1000;
const res = await ddb.update({
TableName: 'rate-limits', Key: { pk: key },
// refill up to capacity based on elapsed time, then take one token
UpdateExpression: 'SET tokens = :cap - :one, updatedAt = :now',
ConditionExpression: 'attribute_not_exists(pk) OR (tokens + (:now - updatedAt) * :refill) >= :one',
ExpressionAttributeValues: { ':cap': plan.capacity, ':one': 1, ':now': now, ':refill': plan.refillPerSec },
ReturnValues: 'ALL_NEW',
}).catch(e => e.name === 'ConditionalCheckFailedException' ? null : Promise.reject(e));
if (!res) return { allowed: false, retryAfterSec: Math.ceil(1 / plan.refillPerSec) };
return { allowed: true, remaining: res.Attributes.tokens };
}
The real version is a few lines longer, because DynamoDB's update expressions can't do the full refill arithmetic in one statement without a min(), so it's a read-compute-conditional-write with a retry on conflict. The point is the shape: one atomic operation per request, no Redis to run, items expire with a TTL so idle keys cost nothing.
The response on deny is what makes this layer worth its cost:
HTTP/1.1 429 Too Many Requests
Retry-After: 6
RateLimit-Limit: 50
RateLimit-Remaining: 0
RateLimit-Reset: 6
Content-Type: application/json
{ "error": "rate_limited", "message": "Your plan allows 50 orders per hour. Try again in 6 seconds, or upgrade to remove this limit.", "upgradeUrl": "/billing" }
A 429 the product can render. The WAF can't write that message, because it doesn't know what an order is.
Where we deliberately don't rate limit
Webhook receivers. A payment provider retrying a delivery is not abuse, it's the protocol working, and a burst of a hundred webhooks after their outage is exactly when you most need to accept every one. The webhook routes are authenticated by signature, they're exempt from the WAF rate rule by path, and the application doesn't bucket them. If they're a load problem, the fix is a queue behind the receiver, not a limit in front of it.
Health checks, for the same reason, and because a rate-limited health check is a health check that lies.
What it costs, and what it caught
| Layer | Monthly cost | What it stopped in six months |
|---|---|---|
| WAF rate rules (2) | ~$2 | 3 credential-stuffing runs, 1 scraper, 1 misconfigured monitoring bot |
| API Gateway usage plans | $0 (not used) | n/a |
| Application limiter (DynamoDB) | ~$1 in on-demand writes | 2 partner retry loops, ~40 users a day hitting plan limits, which is the product working |
Five dollars a month, one afternoon for the WAF rules, one day for the application limiter and its 429 body. The most valuable line is the last one: forty users a day seeing a message that says what the limit is and how to lift it, instead of a generic error, because the limit lives in the layer that knows what it means.
If you have one rate limit for everything and it's either too loose to stop abuse or too tight for real users, that's the sign it's in the wrong layer. We'll help you split it.