Deploying High-Availability AWS Infrastructure using Terraform and Jenkins

In today's fast-paced digital environment, ensuring high availability and fault tolerance in cloud infrastructure is crucial for maintaining seamless user experiences. I'm excited to share my recent project where I implemented a high-availability infrastructure on AWS using Terraform and Jenkins. This project highlights the setup of essential AWS components like Auto Scaling Groups (ASG), Elastic Load Balancers (ELB), and more, all orchestrated through Terraform's robust infrastructure-as-code capabilities, complemented by Jenkins for continuous integration and continuous deployment (CI/CD)

Project Overview

The goal of this project was to create a resilient and highly available AWS infrastructure. By utilizing Terraform, I automated the deployment and management of several AWS resources, ensuring that they are scalable, secure, and efficient. Additionally, Jenkins was used for automating the CI/CD pipeline, enabling seamless integration and deployment of changes.

Key Features

• Auto Scaling Groups (ASG): Automatically adjusts the number of instances based on traffic patterns, optimizing resource use and ensuring reliability.
• Elastic Load Balancers (ELB): Evenly distributes incoming traffic across multiple instances, enhancing availability and eliminating single points of failure.
• Fault Tolerance: Designed to withstand component failures without downtime through redundancy and failover mechanisms.
• Virtual Private Cloud (VPC): Offers a dedicated virtual network for resources, isolated from other AWS accounts.
• Subnets: Hosts instances in different availability zones, enhancing redundancy.
• Internet Gateway (IG): Facilitates communication between the VPC and the public internet.
• Route Tables: Manages traffic flow within the VPC and to the IG.
• Jenkins for CI/CD: Installed Jenkins on a local host to automate the CI/CD pipeline. Jenkins is configured to trigger a job whenever changes are pushed to the GitHub repository, ensuring continuous integration and deployment.

Prerequisites

1. Install terraform and AWS CLI

#Install terraform with below steps on ubuntu
sudo apt-get update && sudo apt-get install -y gnupg software-properties-common

wget -O- https://apt.releases.hashicorp.com/gpg | \
gpg --dearmor | \
sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg > /dev/null

gpg --no-default-keyring \
--keyring /usr/share/keyrings/hashicorp-archive-keyring.gpg \
--fingerprint

echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] \
https://apt.releases.hashicorp.com $(lsb_release -cs) main" | \
sudo tee /etc/apt/sources.list.d/hashicorp.list

sudo apt update

sudo apt-get install terraform        

Install AWS cli using below commands
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip

sudo ./aws/install        

2. Jenkins Setup:

Install Java before installing jenkins

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key

echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null

sudo apt-get update
sudo apt-get install jenkins        

3. Create IAM users with required permissions

4. Configure AWS CLI using access and secret keys

aws configure
AWS Access Key ID [None]: YOUR_ACCESS_KEY_ID
AWS Secret Access Key [None]: YOUR_SECRET_ACCESS_KEY
Default region name [None]: us-east-1
Default output format [None]: json        

Terraform Configuration

The Terraform configuration files with(*.tf) in this repository define the AWS resources and settings required to create the high availability infrastructure.

Providers.tf

provider "aws" {
  region = "us-east-1"
}        

It configures the AWS provider to manage resources in the "us-east-1" region. The provider block specifies the cloud service provider, in this case, AWS, and sets the region where Terraform will execute its operations and manage resources.

vpc.tf

resource "aws_vpc" "vpc" {
  cidr_block       = var.vpc-cidr-block
  tags = {
    Name = var.vpc-name
  }
}        

It creates an AWS Virtual Private Cloud (VPC) with a CIDR block defined by the variable var.vpc-cidr-block. It tags the VPC with a name specified by var.vpc-name, allowing for flexible configuration of the network range and easy identification through tagging

subnets.tf

resource "aws_subnet" "web-subnet1" {
  vpc_id     = aws_vpc.vpc.id
  cidr_block = var.web-subnet1-cidr
  availability_zone = var.az-1
  tags = {
    Name = var.web-subnet1-name
  }
}

resource "aws_subnet" "web-subnet2" {
  vpc_id     = aws_vpc.vpc.id
  cidr_block = var.web-subnet2-cidr
  availability_zone = var.az-2
  tags = {
    Name = var.web-subnet2-name
  }
}        

It sets up two AWS subnets within a specified VPC. Each subnet is assigned a CIDR block to define its IP address range and is placed in a distinct availability zone to enhance redundancy and availability.

route-table.tf

resource "aws_route_table" "public-route-table" {
  vpc_id = aws_vpc.vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.internet-gw.id
  }
}

resource "aws_route_table_association" "pub-rt-asscociation-1" {
  subnet_id      = aws_subnet.web-subnet1.id
  route_table_id = aws_route_table.public-route-table.id
}

resource "aws_route_table_association" "pub-rt-asscociation-2" {
  subnet_id      = aws_subnet.web-subnet2.id
  route_table_id = aws_route_table.public-route-table.id
}        

• Route Table: Creates a route table for the VPC that includes a route directing all outbound traffic (0.0.0.0/0) to the internet via an Internet Gateway. This setup allows resources within the associated subnets to access the internet.
• Route Table Associations: Links the public route table to two subnets, enabling them to use the specified route for internet access. This configuration is essential for allowing instances in these subnets to communicate with the public internet.

internet-gw.tf

resource "aws_internet_gateway" "internet-gw" {
  vpc_id = aws_vpc.vpc.id
  tags = {
    Name = var.igw-name
  }
}        

It creates an AWS Internet Gateway and attaches it to a specified VPC identified by aws_vpc.vpc.id. The Internet Gateway enables the VPC to communicate with the public internet, allowing resources within the VPC to send and receive traffic.

load-balancer.tf

resource "aws_lb" "alb" {
  name               = var.alb-name
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.alb_security_group.id]
  subnets            = [aws_subnet.web-subnet1.id, aws_subnet.web-subnet2.id]
}        

It creates an AWS Application Load Balancer (ALB) that is internet-facing, allowing it to route external traffic to resources. It operates at the application layer, is associated with a specified security group for traffic control, and is deployed across multiple subnets to ensure high availability.

target-group.tf

resource "aws_lb_target_group" "target-group" {
  name     = "tg-1"
  port     = 80
  protocol = "HTTP"
  vpc_id   = aws_vpc.vpc.id
  health_check {
    path = "/"
    port = 80
    protocol = "HTTP"
    matcher = 200
  }
}

resource "aws_lb_listener" "alb_listener" {
  load_balancer_arn = aws_lb.alb.arn
  port              = "80"
  protocol          = "HTTP"
  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.target-group.arn
  }
}        

• Target Group: Defines a target group named "tg-1" for the load balancer, specifying that it will route HTTP traffic on port 80 to instances within the specified VPC. It includes a health check configuration to monitor the health of targets by sending HTTP requests to the root path (/) and expecting a 200 HTTP status code.
• Listener: Sets up a listener for the load balancer that listens for HTTP traffic on port 80. The listener's default action is to forward incoming requests to the previously defined target group, ensuring that traffic is directed to healthy instances.

load-balancer-sg.tf

resource "aws_security_group" "alb_security_group" {
  name        = "alb_security_group"
  description = "Allow inbound traffic and all outbound traffic"
  vpc_id      = aws_vpc.vpc.id

  tags = {
    Name = "alb_security_group"
  }
}

resource "aws_vpc_security_group_ingress_rule" "alb_ingress_rule" {
  security_group_id = aws_security_group.alb_security_group.id
  cidr_ipv4         = "0.0.0.0/0"
  from_port         = 80
  ip_protocol       = "tcp"
  to_port           = 80
}

resource "aws_vpc_security_group_egress_rule" "alb_egress_rule" {
  security_group_id = aws_security_group.alb_security_group.id
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "-1" # semantically equivalent to all ports
}        

• Security Group: Creates a security group named "alb_security_group" within a specified VPC, designed to allow inbound and all outbound traffic.
• Ingress Rule: Configures an ingress rule for the security group to allow incoming TCP traffic on port 80 from any IPv4 address (0.0.0.0/0). This enables public access to the ALB over HTTP.
• Egress Rule: Sets up an egress rule permitting all outbound traffic from the security group to any IPv4 address, using -1 for ip_protocol to signify all protocols and ports, allowing unrestricted outgoing connections.

asg.tf

resource "aws_launch_template" "launch_template" {
  name_prefix   = "web-launchtemplate"
  image_id      = "ami-0e2c8caa4b6378d8c"
  instance_type = "t2.micro"
  key_name = "ansible-key-pair"
  network_interfaces {
    associate_public_ip_address = true
    security_groups = [aws_security_group.alb_security_group.id]
  }
  tag_specifications {
    resource_type = "instance"

    tags = {
      Name = "test"
    }
  }
  user_data = filebase64("user-data.sh")
}

resource "aws_autoscaling_group" "asg" {
  name = "asg-high-avail"
  target_group_arns = [aws_lb_target_group.target-group.arn]
  desired_capacity   = 2
  max_size           = 3
  min_size           = 1
  vpc_zone_identifier = [aws_subnet.web-subnet1.id, aws_subnet.web-subnet2.id]

  launch_template {
    id      = aws_launch_template.launch_template.id
    version = aws_launch_template.launch_template.latest_version
  }
}        

• Launch Template: The launch template defines the setup for EC2 instances, including the AMI, instance type (t2.micro), and SSH key pair. It configures network settings to assign a public IP and associates instances with a designated security group. Additionally, it includes user data for startup scripts and tags instances for identification.

• Auto Scaling Group: The auto scaling group ensures high availability by maintaining a specified number of EC2 instances. It connects to a load balancer target group for efficient traffic distribution, enhancing fault tolerance. Instances are spread across multiple subnets, using the launch template for consistent configuration.

user-data.sh

#!/bin/bash
# Use this for your user data (script from top to bottom)
# Install Apache (Ubuntu 24.04 version)
apt update -y
apt install -y apache2
systemctl start apache2
systemctl enable apache2
echo "<h1>Hello World from $(hostname -f). changing file </h1>" > /var/www/html/index.html        

This script installs and enables appache2 server in EC2 instances

variables.tf

variable "vpc-cidr-block" {
  description = "CIDR Block for VPC"
}

variable "vpc-name" {
  description = "Name for Virtual Private Cloud"
}

variable "igw-name" {
  description = "Name for Internet Gateway"
}

variable "web-subnet1-cidr" {
  description = "CIDR Block for Web-tier Subnet-1"
}

variable "web-subnet1-name" {
  description = "Name for Web-tier Subnet-1"
}

variable "web-subnet2-cidr" {
  description = "CIDR Block for Web-tier Subnet-2"
}

variable "web-subnet2-name" {
  description = "Name for Web-tier Subnet-2"
}

variable "az-1" {
  description = "Availabity Zone 1"
}

variable "az-2" {
  description = "Availabity Zone 2"
}

variable "alb-name" {
  description = "load balancer name" 
}        

This file contains all variable declarations

terraform.tfvars

vpc-cidr-block = "10.0.0.0/16"
vpc-name = "vpc-high-avail"
igw-name = "igw-high-avail"
web-subnet1-cidr = "10.0.1.0/24"
web-subnet1-name = "web-subnet-1"
web-subnet2-cidr = "10.0.2.0/24"
web-subnet2-name = "web-subnet-2"
az-1 = "us-east-1a"
az-2 = "us-east-1b"
alb-name = "alb-high-avail"        

This file contains variable definitions

Application

terraform init
terraform plan
terraform apply        

• Initialize Terraform: Set up the Terraform working directory and download the necessary provider plugins with terraform init.
• Plan Execution: Generate and review an execution plan to see the changes Terraform will make using terraform plan.
• Apply Configuration: Implement the changes to reach the desired state of your infrastructure with terraform apply.

YAY!! High-availability one tier AWS architechture is deployed successfully

Copy the load balancer's dns to the browser and check how load balancer works

Jenkins Integration

Utilize Jenkins for continuous integration and deployment by following these steps:

• Install Jenkins: Set up Jenkins and create a freestyle project to manage your builds.
• Source Code Management: Configure Jenkins to clone your code from a GitHub repository.

• Build Triggers: Set up a polling strategy in the SCM configuration with "*****", enabling Jenkins to check for new changes in the repository every second and rerun the job if updates are detected.

This project not only showcases the power of AWS and Terraform in building scalable and reliable cloud infrastructure but also demonstrates the effectiveness of Jenkins in automating CI/CD processes.

Github Link: https://github.com/purnashanmugam/High_availability_AWS_architechture_using_terraform.git