# Queues without Kafka: SQS, EventBridge, and the one place we actually needed a stream
Every platform reaches the moment where a request does too much. The checkout handler sends an email, updates the search index, notifies the warehouse, records an analytics event and, oh, also charges the card. It takes four seconds and fails if any of the five things is slow. The fix is a queue, and the first proposal is usually Kafka, because Kafka is what the big companies use and the conference talks are about Kafka.
We do not run Kafka. For a platform we operate for a US client, we run SQS for work, EventBridge for events, and one Kinesis stream for the single workload that needed ordering and replay. Here is how we split it, what each costs, and the test for whether you need a stream at all.
## Three different problems
"Queue" is doing a lot of work in most architecture conversations. There are three shapes hiding under it.
**Work to be done, once, by someone.** Send this email. Resize this image. Sync this order to the warehouse. The producer does not care who does it or when, only that it happens, and happens once. This wants a *queue*: an item is delivered to one consumer, acknowledged when done, retried if not, dead-lettered after too many failures.
**Something happened, and anyone interested should know.** An order was placed. A customer changed their email. The producer does not know who is listening and should not have to. This wants an *event bus*: one publish, many subscribers, each with their own queue behind it, added and removed without touching the producer.
**An ordered, replayable history.** Every change to this account, in order, that a consumer can read from any point and re-read after a bug. This wants a *stream*, and it is the only one of the three that Kafka is uniquely good at. It is also the rarest need.
Most platforms have a lot of the first, some of the second, and one or zero of the third. Kafka does all three, at the cost of running Kafka, or paying for a managed Kafka that starts at a few hundred dollars a month before you have sent a message.
## Queues: SQS
Eleven SQS queues, one per kind of work, each with a dead-letter queue and a Lambda or an App Runner worker consuming it. The checkout handler that took four seconds now writes the order, charges the card (the one thing that must be synchronous), publishes one event, and returns in 400 ms. Everything else is a queue consumer.
The settings that matter, in CDK:
```ts
const dlq = new sqs.Queue(this, 'EmailDlq', { retentionPeriod: Duration.days(14) });
const emailQueue = new sqs.Queue(this, 'EmailQueue', {
visibilityTimeout: Duration.seconds(90), // > 6 × the consumer's timeout
deadLetterQueue: { queue: dlq, maxReceiveCount: 5 },
encryption: sqs.QueueEncryption.SQS_MANAGED,
});
new lambda.EventSourceMapping(this, 'EmailConsumer', {
target: emailFn, eventSourceArn: emailQueue.queueArn,
batchSize: 10, reportBatchItemFailures: true, // partial batch success
maxConcurrency: 20, // protect the email provider
});
new cloudwatch.Alarm(this, 'EmailDlqAlarm', {
metric: dlq.metricApproximateNumberOfMessagesVisible(), threshold: 1, evaluationPeriods: 1,
});
```
Three things that were wrong in the first version and are right now. The visibility timeout was equal to the function timeout, so a slow message was redelivered while still being processed, and emails went out twice; it is now six times the function timeout, as the docs say and nobody reads. `reportBatchItemFailures` was off, so one bad message in a batch of ten failed all ten, and nine good emails were retried five times each before the batch hit the dead-letter queue. And the dead-letter queue had no alarm, so messages sat in it for a week before anyone looked; it now pages at one.
**FIFO or standard?** Standard, everywhere except the warehouse sync, which needs "cancel" to arrive after "create" for the same order. FIFO with the order id as the message group gives that, at roughly the same price, with a throughput ceiling per group we are nowhere near. Do not default to FIFO; it adds a deduplication and ordering contract that most work does not want and that makes a stuck message block everything behind it in its group.
## Events: EventBridge
One custom bus. Producers put events with a `detail-type` and a source; rules match and route to targets, which are almost always an SQS queue owned by the consuming service, with a Lambda behind it. The producer of `OrderPlaced` has no idea that five services subscribe to it, and when the sixth appears next quarter, it is a new rule and a new queue, with no change to checkout.
```ts
const bus = new events.EventBus(this, 'PlatformBus');
new events.Rule(this, 'OrderPlacedToSearch', {
eventBus: bus,
eventPattern: { source: ['platform.orders'], detailType: ['OrderPlaced'] },
targets: [new targets.SqsQueue(searchIndexQueue)],
});
new events.Rule(this, 'AllEventsToArchive', {
eventBus: bus, eventPattern: { source: [{ prefix: 'platform.' }] },
targets: [new targets.CloudWatchLogGroup(eventArchive)], // 30 days, for "what happened?"
});
```
The archive rule is the cheap trick that gives you most of what people want from a stream: a searchable, time-ordered record of every event, [in the same log store as everything else](/en/blog/your-logs-should-not-know-which-cloud), without a stream. It cannot replay into a consumer, but it can answer "did we emit OrderPlaced for order 4412?" in one query, which is the question that actually gets asked.
EventBridge's contract is at-least-once, unordered, with a 256 KB payload limit. Every consumer is idempotent, keyed on the event id, and any event bigger than a few KB carries a pointer to S3, not the payload. Those two rules cover every problem we have had with it.
## The stream: Kinesis, once
The ledger. Every financial movement on the platform, in order per account, projected into balances and reports by a consumer that must be able to be rebuilt from scratch if a projection bug is found. That is the stream-shaped problem: ordering per key, and replay from any point.
One Kinesis stream, one shard, 7-day retention, a Lambda consumer with a checkpoint. When a projection bug was found in month four, the fix was to correct the consumer, reset the checkpoint to the start of retention, and let it re-read three days of events into a fresh table. SQS cannot do that; a consumed message is gone. EventBridge cannot do that; the archive is a log, not a cursor.
## The test for "do you need a stream?"
Ask: *if a consumer had a bug last Tuesday, do you need to re-feed it Tuesday's messages in order?* If the honest answer is "we would re-run a batch job against the database", you need a queue and a database, not a stream. If the answer is "yes, and the database does not have the history", you need a stream, for that consumer. One stream, for one consumer, is not a reason to move the whole platform onto Kafka.
## What it costs
| | Monthly | Operations |
|---|---|---|
| SQS, 11 queues + 11 DLQs, ~4 M messages | ~$2 | Zero. Alarms on the DLQs |
| EventBridge, 1 bus, 9 rules, ~1 M events | ~$1 | Zero. Archive rule for "what happened" |
| Kinesis, 1 shard, 7-day retention | ~$15 | Checkpoint monitoring; shard count if it ever matters |
| **Total** | **~$18** | |
| Managed Kafka, smallest useful cluster | $300–600 | Topics, partitions, consumer groups, retention, a broker version to track |
Eighteen dollars and no broker. The platform handles a few million messages a month; SQS would handle a few billion at the same shape, with the bill scaling linearly and nothing to re-architect.
## The short version
Split "queue" into three problems. Work goes to SQS with a dead-letter queue and an alarm. Facts go to EventBridge with a queue per subscriber and an archive rule. History, if you truly have a consumer that must replay in order, goes to one Kinesis stream, for that consumer. Kafka is the right answer when you have many of the third problem, and it is fine to not have that problem.
If your checkout takes four seconds because it does five things, [we can take four of them off the request path in a week](/contact), for about two dollars a month.