🚀 Real Life Shell Scripting Project for Cloud Infrastructure (DevOps Must-Know!)

In today's cloud-driven world, tracking resource usage efficiently is critical — not just for cost optimization but also for managing operations at scale. If you're a DevOps engineer (or preparing for interviews), understanding real-world shell scripting projects is not optional — it's essential.

In this blog, I’ll walk you through a real-life Shell Scripting project example that mirrors what most organizations actually do to manage their cloud infrastructures (especially AWS). 🌟

📌 Why Move to the Cloud?

Organizations move to cloud mainly for two reasons:

• Manageability: Maintaining your own servers requires constant updates, patches, and a dedicated engineering team. Cloud service providers handle most of this, reducing maintenance overhead.
• Cost-effectiveness: In the cloud, you "pay as you use." Unlike traditional data centers where you pay for physical servers regardless of usage, cloud providers only bill for active resources.

🎯 Real-World Problem Statement

Imagine you work at an organization where developers can freely create AWS resources — EC2 instances, S3 buckets, Lambda functions, and IAM users.

Over time, without proper monitoring:

• Unused resources like unattached EBS volumes start piling up.
• Cloud bills skyrocket due to unused but chargeable resources.

Your Task: Everyday at 6 PM, automatically generate a report with:

• Active EC2 Instances
• List of S3 Buckets
• Active Lambda Functions
• Existing IAM Users

This task must run even if you are offline.

🛠️ How to Solve It: Step-by-Step

Step 1: Install and Configure AWS CLI

If AWS CLI is not installed:

Then configure AWS CLI:

Enter: Access Key, Secret Key, Region, Output format (json).

Step 2: Write the Script

1. Shebang

It tells the operating system to use the Bash shell to run this script.

2. Metadata (Documentation)

At the top of the script:

• Good practice in DevOps.
• Explains who wrote it, when, what the script does.

3.1. List S3 Buckets

3.2. List EC2 Instances

• aws ec2 describe-instances: Returns details of EC2 instances.
• jq -r '.Reservations[].Instances[].InstanceId': Extracts only the Instance IDs.

3.3. List Lambda Functions

3.4. List IAM Users

Final Message

Detailed Explanation of jq Command

jq is a lightweight command-line JSON processor. AWS CLI gives outputs in JSON format — but JSON is difficult to read manually.

Syntax Used:

• -r : Raw output (removes quotation marks).
• .Reservations[].Instances[].InstanceId: Navigate the JSON structure to extract only Instance IDs.

Similarly:

• .Functions[].FunctionName → Extracts Lambda function names.
• .Users[].UserName → Extracts IAM usernames.

4. Enable Debug Mode

set -x: Displays each command before execution (useful for debugging).

Step 3: Schedule with Cron Job

Once you have written and tested your AWS resource reporting script (aws_resource_report.sh), the next important step is to automate its execution — so that you get regular reports without manually running the script every time.

This is where crontab comes into play!

What is crontab?

• crontab stands for Cron Table.
• It is a time-based job scheduler in Unix/Linux systems.
• Using crontab, you can schedule tasks (cron jobs) to run automatically at specific intervals — like daily, weekly, monthly, or even every few minutes.

It is extremely useful in DevOps and Automation to ensure important tasks are executed without manual intervention.

1. Open the crontab file for the current user:

Add an entry to schedule your script.

Example: To run your AWS Resource Report script every day at 6 AM, you can add:

💬 Final Thoughts

Shell scripting is not just about "writing a few lines of code." It's about thinking practically and solving real-world operational challenges.

This project is a brilliant example of how a simple shell script can save companies thousands of dollars and tons of manual hours.