A notification system delivers messages to users across many channels — mobile push (APNs/FCM), SMS, email, and an in-app inbox — from a single internal event. It sits behind almost every product: OTPs, order updates, social alerts, and marketing blasts all flow through it.
Three forces make it a genuine design problem:
- Fan-out at scale — one trigger can expand to millions of recipients (a broadcast), and each recipient may want several channels.
- Reliable delivery — external providers time out, throttle, and fail, yet an OTP must still arrive in seconds. At-least-once + idempotency is the spine.
- Don't spam — preferences, quiet hours, dedup, and frequency caps decide whether and when to send, not just how.
The whole design is a buffered, asynchronous pipeline: ingest → policy → per-channel fan-out → workers → providers, with retries, dead-letter queues, and delivery-status tracking throughout.