Asynchronous communication: queue or topic?

When we talk about asynchronous communication, it’s about a service communicating with another without expecting a response from the target service, or we expect the response may arrive from the target but in a different channel. Think about the postman delivering a mail into your mailbox, but they don’t wait for your response? When you go back and receive it, you start to process it till then. A message broker gets involved in this situation whose responsibility is to deliver your messages from the producer to the consumer.

But how to choose which one to use for delivering the message? Queue or Topic?

Single receiver pattern 

Usually, based on the name, we use Single-receiver when we only have one consumer. The broker offers message-delivery guarantees, such as at-least-once. The only thing the producer care about (and knows) is whether the message is sent to the broker (queue) or not. The same on the Consumer side, the broker makes sure the message is delivered based on your message-delivery policy.

Multiple receiver pattern 

What about if we have multiple consumers need to consume the same message? Think about you want to subscribe to a newsletter? Here is why we need multiple receiver pattern. In the real business world, more than one of the microservices might need to consume the same data, for example, when customers book a deal, the deal processor starts working, the deal-booked email could be triggered, and we may have another audit-record writer behind is working. In this case, we can publish a deal-booked topic event and those three microservice subscribe to it to get the notification and then trigger their process.

But be aware, that the delivery guarantees are not as strict as Single-receiver, which means if we have a scenario where one of the consumers is offline for a long time, the message would be lost. We can enable the durable subscription based on different broker solutions but keep in mind, it increases the load on the event bus. So Generally, if the use case is about to deliver “at-least-once”, you should consider using multiple queues for each consumer(Single-receiver).

How does AWS SNS deal with this problem? Based on SNS documentation, Amazon SNS introduces a message deliver retry policy which makes sure all the messages should be delivered before the retry policy ends. But few cases could cause the delivery to fail which includes the things like subscription metadata problems or infrastructure errors. Therefore, the best practice is to add a Dead Letter Queue to the subscription. It makes your application more resilient and durable.

By using this pattern, the data should be stored in a persistent place by the producer since it’s the source of truth. In addition, it’s important to keep the broker independent from the business logic, for example, the broker usually supports multiple features(ex. Filtering, Routing…), and if you use it much, you may end up coupling your logic into this infrastructure level and lose visibility to increase complexity. If you really need them, consider revisiting the design to make sure it’s reasonable.

Note: The banner is from my mother's painting: Splendid ocean