Difference between Queues and Message Brokers

Recently, it has been quite common for most organizations to use micro-service architecture to build their systems. Microservice architecture has many advantages like improved scalability, resilience and better CI/CD over monolithic architecture. Despite these advantages, microservice architecture requires a reliable communication layer between its different services to work as a single logical and business unit. There are two types of messaging protocols for communication between microservices:

1. Synchronous: In this type of messaging protocol, the caller waits for a response from the receiver.
2. Asynchronous: In this type of messaging protocol, a microservice sends a message and continues its execution without waiting for the receiving microservice(s) response.

We are primarily interested in Asynchronous communication; this is where message queues and brokers are most helpful (This is not the only use-case for message queues and brokers).

Message Queue:

A queue is a simple data structure such that the data that enters the queue first is the first one transmitted out of the queue. In a queue, the message can only be processed once out of the queue. In other words, if there are multiple consumer services to a queue, it is impossible for all the consumer services to process all the messages that come out of the queue.

Now that we know what a queue is, a message queue can be used for simple use-cases; for instance, when a user signs up for a website, a verification email should be sent to the user's email. We can assume that the organization has two microservices one is a user service which handles user signup, and another is a notification service which handles all the notifications between the system and its users. We could use synchronous communication like an HTTP API to achieve the above requirement, but if the notification service is unavailable, it will lead to situations where users will not get verification emails. If we introduce a simple message queue between the user and notification services, the queue will act as a buffer between services and ensure that all the messages from the user service will eventually get processed by the notification service.

Message Broker:

A message broker is a software service built on top of a message queue and other technologies that can validate, store, route and deliver messages to the relevant services. Message broker also offers message translation, including translation of the message between formal messaging protocols.

To understand the message broker concept, let's take a simple example of a user buying a product through an e-commerce website. As per my limited understanding of e-commerce website operations, if a user purchases a product, the inventory count corresponding to that product must be managed, and an invoice must be generated for that product. Basically, the requirement demands that the message published should be sent to multiple consumers. A message broker achieves this requirement with the concept of the topic. All the consumers are subscribed to a single topic; when a message is published to the topic, all the consumers will be notified.

I hope this article helps you understand the difference between message queues and message brokers. If I have missed any major points, let me know in the comments below.