Efficient event driven microservices with RabbitMQ
RabbitMQ is an open source message broker which was designed to solve the problem of Sphageti/Mesh architecture where all components are directly dependent.
Multiplexing
Some applications need multiple connections to the broker. However, it is undesirable to keep many TCP connections open at the same time because each connection creates 14 tcp packages and doing so consumes system resources and makes it more difficult to configure firewalls. AMQP 0-9-1 connections are multiplexed with channels that can be thought of as "lightweight connections that share a single TCP connection". Best practice is to reuse connections and multiplex a connection between threads with channels.
Throughput vs Resilience
If throughput is you goal while using RabbitMQ then use Transient queues, exchanges and messages
If Resilience is your goal while using RabbitMQ then use Durable Exchanges, queues and messages
Mirroring
Not all queues should be mirrored. Identify the queues for which you needs High Availability and then mirror them. Also do not mirror long queues as it will take time to sync across the mirrored queues.
Load balancing
Use load balancing where the traffic is high. Load balancing can be achieved using RabbitMQ plugins like Shovel and Federation.
Short queues
For better performance, queues should be as short as possible. Longer queues needs more processing.
Number of Master queues on a node
Queues are single threaded in RabbitMQ. So for better performance on a multi core system have queues equal to number cores on the node. On a cluster setup, distribute master queues on each node equal to number of cores on the node.
Prefetch
Prefetch count should be as less as possible. Unlimited prefetch count can result in crashing the client.
Quoram queues
RabbitMQ 3.8.x a new feature "Quorum Queues". It is a replicated queue to provide high availability and data safety. The idea is to replicate a queue across multiple servers so that in the event of a server crashing or being shut down, the queue continues to be available and without message loss. Quorum queues aim to resolve both the performance and the synchronization failings of mirrored queues. Quorum Queues uses a variant of the Raft protocol which is distributed consensus algorithm. It is both safer and achieves higher throughput than mirrored queues.