Configuring Haproxy to balance the load between two webservers using Ansible

Today Modern High traffic websites serves millions of requests concurrently and return images, videos ,files or data in a efficient, fast and reliable manner .To cost-effectively scale and meet this increasing demands the best practice is to generally increase the number the servers. If the number of servers increases the problem is how we can distribute the incoming user traffic to different servers in a effective and secure manner.

This is where the role of load balancer/haproxy comes in, it will act as a reverse proxy and will take the request of the client to the server(webserver in our case) and will return its response to the client. Let me explain this with the help of an example:

Here we have multiple users that are connected to the multiple servers via a Load Balancer(Haproxy). In this scenario we have given only one IP(of the Load Balancer) to the client. The client will send a request to this IP thinking that he is directly connected to the server but in fact he will send this request to the load balancer. Now the load balancer will further take the request from the client to a server. The server will then process this request and send the output to the load balancer IP which will further take the output to the client.

Now for creating the above architecture I have created ansible playbook which will create the entire setup on the running of the command. All the servers launched using the playbook are launched on AWS, the IP is then fetched using the concept of dynamic inventory and is used to configure the Haproxy server.

If you want to learn how launch and configure instances on AWS using ansible and its dynamic inventory concept refer my previous article.

First making sure that the configuration file of ansible has been configured for the use of dynamic inventory:

Now onto the playbook. I will explain the playbook in two parts. The first part is launching the instances on AWS and the second part will be how to configure the launched instances.

- hosts: localhost
  vars_files:
  - passwd.yml
  tasks:
  - name: start an instance and Add EBS
    amazon.aws.ec2:
     exact_count: 2
     count_tag:
             Type: Webserver
     region: ap-south-1
     vpc_subnet_id: subnet-202e446c
     instance_type: t2.micro
     image: ami-0a9d27a9f4f5c0efc
     key_name: "mykey"
     group_id: sg-02491ed86d2c7184c
     aws_access_key: "{{ access_key }}"
     aws_secret_key: "{{ secret_key }}"
     instance_tags:
             Type: Webserver


  - name: start an instance and Add EBS
    amazon.aws.ec2:
     exact_count: 1
     count_tag:
             Type: Haproxy
     region: ap-south-1
     vpc_subnet_id: subnet-202e446c
     instance_type: t2.micro
     image: ami-0a9d27a9f4f5c0efc
     key_name: "mykey"
     group_id: sg-07970abf3ac6fbc39
     aws_access_key: "{{ access_key }}"
     aws_secret_key: "{{ secret_key }}"
     instance_tags:
             Type: Haproxy


  - name: Refresh inventory to ensure new instances exist in inventory
    meta: refresh_inventory

The code used here is pretty self explanatory but the thing to be noted here is that, the tags that are given while launching the instance will be used by dynamic inventory to create groups like here the tag: Type: Webserver will create a group : tag_Type_Webserver. The group so created can be used in the playbook to refer to the IPs.

I have also used the 'refresh_inventory' to update the inventory with the newly launched instances.

Now for the second half of the playbook, Here I have used the concept of Roles in ansible to configure the target nodes. Roles are basically what a package is in python here it is a directory that contains another set of directory, in those directories we can write what we want ansible to do. The directory structure is something like:

Here for each property of the playbook there is a different directory like for tasks there is a separate directory, for vars a separate one and so on. To create a role use the following commands:

ansible-galaxy init 'role name'

These roles can also be uploaded to Ansible Galaxy which is a public repository for ansible roles. These roles can be downloaded and are ready to be used.

The rest of the playbook code:

- hosts: tag_Type_Webserver
  become: yes
  remote_user: ec2-user
  roles:
  - webserver
                                                                                                                                                                    
- hosts: tag_Type_Haproxy
  become: yes
  remote_user: ec2-user
  roles:
  - haproxy

Here each role will work on different groups which have been created by the dynamic inventory code as explained above.

Here two roles have been used webserver and haproxy. The tasks of each are as follows:

The code of each is self explanatory but these are the changes which are to be made in the config file of haproxy:

Here the thing to be noted is to access the haproxy server we will send the request on the 5000 port and a 'for' loop has been added to the file. The use of this for loop is to update the haproxy file with the IPs of the webservers. The following is syntax of jinja2 which is a templating language and is used to created dynamic content.

Execute the playbook using the following command:

ansible-playbook --ask-vault-password haproxy.yml

Output as seen on the terminal:

Output as seen on the AWS console:

Accessing the webserver via the haproxy server's IP:

As it can be seen there are two different private IPs of the servers launched which confirms that the haproxy server has launched successfully.

Github link for the code: https://github.com/ArjunRekhi/ansible-aws