Publish–Subscribe (Pub/Sub) Pattern
Let’s face it. The client shouldn’t have to wait for all backend processing to finish.
Imagine uploading a video to YouTube. Should the client really sit and wait while YouTube processes the video, encodes it, scans it for copyright, and then sends notifications? Definitely not.
🧠 Why Pub/Sub?
Many modern systems are made of independent services working together. For example: UploadService → ParseService → CopyrightService → NotificationService.
If these services are directly calling each other in sequence, the system becomes tightly coupled and is hard to maintain or recover from failures. If one service fails, the entire chain could break. This approach also doesn’t scale well.
That’s where the Publish–Subscribe pattern comes in.
📦 How Pub/Sub Works
The client uploads a file (e.g., a video or PDF).
The server responds with a success status, and the client can disconnect.
The UploadService then publishes an event to a topic (e.g., "upload.video").
A message broker (like Kafka, RabbitMQ, etc.) notifies all subscribers who are interested in that topic.
Each subscriber performs its task (parsing, scanning, generating thumbnails, sending notifications...) and may publish further events to other topics.
Everything becomes decoupled, scalable, and much easier to maintain.
🧪 Real-Life Example from My Experience
I worked on a project where users uploaded PDF files. Here's what happened after the upload:
The server converted the PDF to text and stored it in a database.
A second service generated two images (a thumbnail and a high-res preview) from the first page.
Both the PDF and images were uploaded to Azure Blob Storage.
Only after the PDF was uploaded, the client got a success response and disconnected. If we had applied Pub/Sub, the client would’ve only waited for the PDF upload. Everything else, such as processing, converting, and uploading images, could’ve happened asynchronously in the background, handled by separate services via subscribed topics.