Deploy to EC2 from Jenkins Pipeline with Docker Compose

I'm excited to share one of my recent cloud engineering projects: CloudTask Pro — a production-grade task management platform deployed on AWS. The goal of this project was not only to build a web application, but to design and deploy it using a realistic production-style cloud architecture and DevOps workflow. 𝗞𝗲𝘆 𝗵𝗶𝗴𝗵𝗹𝗶𝗴𝗵𝘁𝘀 𝗼𝗳 𝘁𝗵𝗲 𝗽𝗿𝗼𝗷𝗲𝗰𝘁:  • Infrastructure as Code using Terraform modules  • CI/CD pipeline with Jenkins and GitHub webhooks  • Dockerized backend deployment with Docker Hub  • Frontend hosting using Amazon S3 and CloudFront  • Backend deployment on EC2 Auto Scaling Group behind an Application Load Balancer  • PostgreSQL database hosted on Amazon RDS  • Secrets management with AWS Secrets Manager  • Monitoring and logging using CloudWatch  • Public/private subnet separation inside a custom VPC  • Internal deployment automation using AWS Systems Manager (SSM) 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 𝗰𝗼𝗺𝗽𝗼𝗻𝗲𝗻𝘁𝘀 𝘂𝘀𝗲𝗱:  • VPC, public/private subnets, route tables, NAT Gateway  • EC2, Auto Scaling Group, Launch Template  • Application Load Balancer  • Amazon RDS PostgreSQL  • Amazon S3 and CloudFront  • IAM roles and security groups  • Jenkins, Docker, GitHub, Terraform 𝗙𝗼𝗿 𝗱𝗲𝘁𝗮𝗶𝗹𝘀, 𝗽𝗹𝗲𝗮𝘀𝗲 𝗰𝗵𝗲𝗰𝗸 𝗼𝘂𝘁 𝘁𝗵𝗲 𝗚𝗶𝘁𝗛𝘂𝗯 𝗿𝗲𝗽𝗼𝘀𝗶𝘁𝗼𝗿𝘆: https://lnkd.in/d6fiEi3m #AWS #DevOps #Terraform #Jenkins #Docker #CICD #CloudEngineer #AWSSolutionsArchitect #InfrastructureAsCode 

🔧 Lab Title: 10 - Deploy to EC2 server from Jenkins Pipeline - CI/CD Part 3 Project Steps PDF Your Easy-to-Follow Guide :https://lnkd.in/gkdbhTMU 🔗 GitLab Repo Code:https://lnkd.in/gbXhi8nS 🔗 DevsecOps Portfolio:https://lnkd.in/g6AP-FNQ 💼 DevOps Portfolio: https://lnkd.in/gT-YQE5U 🔗 Kubernetes Portfolio:https://lnkd.in/gUqZrdYh 🔗 GitLab CI/CD Portfolio:https://lnkd.in/g2jhKsts Summary:  Today, I automated a full CI/CD pipeline that builds a Java Maven app, containerizes it, and deploys it on an AWS EC2 instance using Jenkins and Docker Compose. This involved leveraging Jenkins Shared Libraries, dynamic versioning, and secure SSH to orchestrate multi-container deployments remotely. Tools Used: Jenkins: For orchestrating CI/CD stages and pipeline automation Docker & Docker Compose: Containerizing the app and managing multi-service deployments AWS EC2: Hosting the application in a cloud environment Maven: Building the Java application artifact Skills Gained: Modular Pipeline Design: Used Jenkins Shared Libraries to reuse and modularize pipeline code Dynamic Versioning: Automated image version increments in Jenkins pipeline for traceability Secure Deployment: Implemented secure SSH connections with sshagent for remote EC2 deployment Multi-Container Orchestration: Managed app and database services remotely using Docker Compose Challenges Faced: Handling SSH key permissions and StrictHostKeyChecking to avoid connection issues Automating version bumps in Maven and syncing image tags correctly with Jenkins build numbers Why It Matters:  This lab showcases real-world DevOps practices for scalable, automated delivery of containerized applications. Mastering these CI/CD techniques is essential for efficient and secure deployment pipelines in cloud-native environments, boosting development velocity and infrastructure reliability. 📌 hashtag#DevOps hashtag#Jenkins hashtag#DockerCompose hashtag#AWS hashtag#CI_CD hashtag#Automation hashtag#CloudDeployment hashtag#JavaMaven 🚀 Stay tuned!  Next project 11 - Introduction to AWS CLI - Part 1 coming soon.

From zero to a production-grade Kubernetes platform on AWS, built in a single day. I recently completed an end-to-end cloud infrastructure project where the goal wasn’t just to deploy an application, but to design and operate it the way a real production system would be built. Starting from a containerised stack (React, Flask, PostgreSQL), I provisioned and deployed a highly available, secure, and fully automated platform using Infrastructure as Code and GitOps principles. Key components of the system: • Multi-AZ AWS infrastructure provisioned with Terraform (VPC, segmented subnets, per-AZ NAT Gateways, Route53 DNS) • Private Kubernetes cluster built with Kops (3 control plane nodes and 3 worker nodes across availability zones, no public node exposure) • Application workloads deployed with proper scaling, health checks, and persistent storage (PostgreSQL on encrypted EBS, replicated API and frontend layers) • TLS and ingress management using NGINX Ingress Controller with cert-manager (automated certificate issuance and renewal) • Network segmentation using Calico with explicit NetworkPolicies (default deny with controlled service-to-service communication) • GitOps workflow implemented with ArgoCD (automated reconciliation, self-healing, and elimination of configuration drift) • Security best practices applied across layers (non-root containers, encrypted storage, IAM least privilege, no secrets in source control) What made this project valuable was not the individual tools, but the system design decisions: – Designing for high availability across availability zones – Enforcing private networking as a default – Treating Git as the single source of truth for deployments – Building with security and immutability from the start A few practical lessons reinforced along the way: • DNS delegation and propagation can become a critical path if not handled early • Running workloads as non-root affects service design and port management • GitOps significantly reduces operational overhead and configuration drift • Separating infrastructure and application concerns improves maintainability The platform is live: 🌐 https://lnkd.in/eQnDYmyY 🔎 https://lnkd.in/ez37P225 This project reflects how I approach cloud infrastructure: reliable by design, secure by default, and fully automated. Open to connecting with teams working on cloud-native platforms, DevOps, or infrastructure engineering.

Building a Scalable CI/CD Environment on AWS with Terraform, Jenkins, and S3 Building a modern AWS CI/CD pipeline requires the right combination of tools and infrastructure to handle growing development teams and complex deployment needs. This guide walks DevOps engineers, cloud architects, and development teams through creating a scalable CI/CD environment using Terraform infrastructure as code, Jenkins on AWS, and AWS S3 artifact storage. https://lnkd.in/ghZy7qCG Amazon Web Services (AWS) #AWS, #AWSCloud, #AmazonWebServices, #CloudComputing, #CloudConsulting, #CloudMigration, #CloudStrategy, #CloudSecurity, #businesscompassllc, #ITStrategy, #ITConsulting, #viral, #goviral, #viralvideo, #foryoupage, #foryou, #fyp, #digital, #transformation, #genai, #al, #aiml, #generativeai, #chatgpt, #openai, #deepseek, #claude, #anthropic, #trinium, #databricks, #snowflake, #wordpress, #drupal, #joomla, #tomcat, #apache, #php, #database, #server, #oracle, #mysql, #postgres, #datawarehouse, #windows, #linux, #docker, #Kubernetes, #server, #database, #container, #CICD, #migration, #cloud, #firewall, #datapipeline, #backup, #recovery, #cloudcost, #log, #powerbi, #qlik, #tableau, #ec2, #rds, #s3, #quicksight, #cloudfront, #redshift, #FM, #RAG

🚀 Scalable CloudOps Deployment Platform on AWS I recently built and deployed a production-ready 3-tier cloud architecture on AWS to host a full-stack application, focusing on scalability, automation, and real-world DevOps practices. 🔧 Tech Stack & Services Used AWS EC2, Auto Scaling Group, Application Load Balancer AWS RDS (MySQL), S3, CloudFront AWS Route 53, ACM (SSL), VPC (Public & Private Subnets) AWS CloudWatch (Monitoring & Alerts) Node.js (Backend), React.js (Frontend) Git & GitHub for version control ⚙️ What I Built Designed a 3-tier architecture with secure VPC networking Deployed backend on EC2 with Auto Scaling (min: 1, desired: 2, max: 3). Hosted frontend on S3 + CloudFront (CDN) for global delivery Configured custom domain with HTTPS (ACM + Route 53) Integrated backend API with RDS (MySQL) for persistent storage Implemented monitoring using CloudWatch dashboards & alarms 🚧 Challenges I Faced & Solved 🔹 Auto Scaling instances failing health checks → Fixed using proper User Data scripts + PM2 automation 🔹 Private subnet instances unable to install packages → Solved by configuring NAT Gateway + route tables 🔹 CloudFront frontend not calling backend → Resolved CORS + HTTPS mismatch issues 🔹 SSL certificate errors → Learned correct usage of ACM (region-specific for CloudFront vs ALB) 🔹 Instance debugging without public access → Used bastion host approach for private EC2 access 📊 Key Outcomes Achieved high availability & fault tolerance with Auto Scaling Reduced latency by ~30% using CloudFront CDN Improved reliability with automated deployment & monitoring alerts Built a real-world DevOps pipeline without manual intervention 💡 This project helped me deeply understand cloud architecture, networking, debugging real production issues, and DevOps practices. Would love feedback or suggestions from the community! 🙌

🚀 Strengthening DevOps Practices with Terraform on AWS Over the past few days, I revisited and implemented a complete infrastructure setup using Terraform — focusing on clean design, security, and real-world debugging scenarios. ⸻ 🧱 Infrastructure Overview: ✔️ EC2 Layer • Multiple instances (backend & frontend) • SSH access restricted via specific CIDR blocks • Public web access enabled (80/443) ✔️ Database Layer (RDS – MySQL 8) • Controlled public access for testing use cases • Access restricted using IP-based rules • Internal VPC communication enabled ✔️ Storage Layer (S3) • Environment-based naming • Designed for scalability and reuse ⸻ ⚙️ Terraform Approach: • Parameterized infrastructure using variables • Used dynamic blocks for managing multiple ingress rules • Structured code for better readability and maintainability • Avoided hardcoding to support multi-environment usage ⸻ 🔐 Security Considerations: • SSH access restricted (no open access) • Database access controlled via CIDR + VPC • Clear separation between public and internal traffic ⸻ 🛠️ Real-World Debugging: Handled practical challenges like: • Key pair configuration issues • AWS credential/token errors • RDS password and naming constraints • Resource dependency and configuration issues ⸻ 🧠 Key Takeaways: • Clean IaC design improves maintainability • Security should be part of design, not afterthought • Debugging builds real confidence in DevOps ⸻ 🚀 What’s Next: • Kubernetes (EKS) implementation • CI/CD pipeline integration • End-to-end application deployment ⸻ 💬 Always refining the fundamentals while moving towards scalable and production-ready architectures. ⸻ #DevOps #Terraform #AWS #CloudArchitecture #InfrastructureAsCode #SRE #LearningNeverStops

🚀 Automated AWS Infrastructure with Terraform & Jenkins – Static Website Hosting I worked on a DevOps project to provision AWS infrastructure and host a static website fully automatically using Terraform and Jenkins, demonstrating Infrastructure as Code (IaC), CI/CD, and cloud automation. 1️⃣ Project Overview The goal was to create a one-click, fully automated solution that provisions AWS resources, launches application containers, and hosts a static website using Terraform. The project integrates state management, automation, and CI/CD, making it a real-world example of DevOps practices. 2️⃣ Tools & Technologies Terraform – for Infrastructure as Code Jenkins – for CI/CD automation AWS – EC2, S3, DynamoDB Docker – to run application containers S3 & DynamoDB – for state management and locking 3️⃣ Key Steps & Workflow Step 1: Setup & Prerequisites Configured AWS account, Terraform, Jenkins, and Docker Ensured all tools were properly installed and connected Step 2: Terraform Scripts Wrote Terraform scripts to provision AWS resources including: EC2 instances for application containers S3 bucket for hosting the static website DynamoDB table for Terraform state locking Introduced user data scripts to automatically run tasks like launching two application containers per EC2 instance Step 3: State Management S3 bucket: securely stores Terraform state files DynamoDB table: enables locking, preventing multiple users from corrupting infrastructure state Step 4: Jenkins Integration Created Jenkins pipeline to automate Terraform provisioning Pipeline includes steps for: Initializing Terraform Validating and planning infrastructure changes Applying Terraform scripts Ensuring state files are securely stored and locked Step 5: Automation & Deployment Launch multiple application containers automatically using Terraform user data scripts Static website hosted on S3, fully automated from Jenkins pipeline Provides repeatable, consistent deployments 4️⃣ Benefits & Learnings Demonstrates Infrastructure as Code principles in practice Shows how to integrate Terraform with CI/CD pipelines Enables secure and scalable state management using S3 + DynamoDB Provides experience in end-to-end cloud automation for production-ready infrastructure hashtag#DevOps hashtag#Terraform hashtag#Jenkins hashtag#AWS hashtag#CI/CD hashtag#InfrastructureAsCode hashtag#Automation hashtag#S3 hashtag#DynamoDB hashtag#Docker hashtag#Cloud hashtag

Built and automated AWS infrastructure using Terraform — from event-driven monitoring to a serverless API ☁️🚀 Over the past few weeks, I’ve been working on two cloud projects focused on Infrastructure as Code and DevOps automation. 🔹 EC2 State Change Monitoring System 🔹 Serverless API (API Gateway + Lambda + DynamoDB) Instead of treating these as separate builds, I focused on end-to-end system design and automation. ⚙️ What was implemented Event-driven monitoring using EventBridge → Lambda → SNS API-based backend using API Gateway → Lambda → DynamoDB Infrastructure defined and deployed using Terraform CI/CD pipeline using GitHub Actions 🧠 Where Terraform fits Terraform acted as the control layer, handling: Infrastructure provisioning Service dependencies Lambda packaging (archive_file) Consistent deployments across environments 🔄 DevOps workflow Code → GitHub → CI/CD → Terraform → AWS Remote state stored in S3 State locking via DynamoDB Automated deployments from GitHub 🔍 Key troubleshooting lessons Fixed CI/CD failures caused by malformed AWS credentials Resolved Lambda runtime issues using CloudWatch logs Handled Terraform state conflicts by implementing remote backend Debugged resource duplication due to missing shared state Corrected handler and packaging issues in Lambda 🔥 Key takeaway Building infrastructure is one thing — making it automated, consistent, and reliable is what defines DevOps. #Terraform #AWS #DevOps #AWS #Terraform #DevOps #CloudComputing #InfrastructureAsCode #Serverless #GitHubActions #CloudEngineering #DevOpsEngineer #BuildInPublic

🚀 New Project: Multi-Environment Terraform Deployment with GitLab CI/CD One thing every DevOps engineer encounters early on: how do you manage dev, staging, and prod infrastructure without duplicating code or risking state conflicts? Here's what I built to solve exactly that What the project does: A fully automated IaC pipeline that provisions isolated AWS environments (develop + prod) from a single Terraform codebase, triggered automatically by GitLab CI/CD on every push. How it works: → Push to develop → pipeline runs → staging EC2 deployed (manual approval required) → Merge to main → pipeline runs → prod EC2 deployed (automatic) → Each environment gets its own isolated Terraform state in S3 → State locking prevents concurrent pipeline runs from corrupting infrastructure Stack: • Terraform Workspaces: one codebase, multiple isolated environments • AWS S3: remote backend for shared, versioned state storage • GitLab CI/CD: 3-stage pipeline: validate → plan → apply • AWS EC2 + Security Groups: environment-tagged resources • IAM: least-privilege service account for the pipeline Key lessons learned: • TF_WORKSPACE is a reserved Terraform variable, naming your CI variable the same breaks workspace selection silently (fun one to debug 🙃) • GitLab Protected variables are only injected into protected branches, unprotect them if your pipeline runs on feature/develop branches • Terraform 1.10+ native S3 locking (use_lockfile) replaces the DynamoDB dependency, simpler and cleaner • Manual approval gates in CI aren't just a safety net, they're standard practice in real teams Why this matters for interviews: Remote state, workspace isolation, and branch-based deployment strategies are questions I now get asked about, and can answer from real hands-on experience, not just theory. Full project with README guide on GitHub: https://lnkd.in/dgNT_NTe #DevOps #Terraform #GitLabCI #AWS #InfrastructureAsCode #CloudEngineering #IaC #Berlin #OpenToWork

🚀 Scalable CI/CD Pipeline for Node.js Application using AWS DevOps Services Excited to share my project where I designed and implemented a scalable CI/CD pipeline for a Node.js application using AWS DevOps services. 🔹 What I Did Built an automated CI/CD pipeline that integrates source code from GitHub, builds the application using AWS CodeBuild, stores artifacts in S3, and deploys them to EC2 instances using CodeDeploy. 🔹 Tools & Technologies Used AWS CodeBuild (Build & Test Automation) AWS S3 (Artifact Storage) AWS CodeDeploy (Deployment Service) AWS EC2 (Compute Infrastructure / Agent Setup) IAM (Roles & Permissions Management) GitHub (Source Code Management) VS Code (Development Environment) 🔹 Key Highlights ✔️ Implemented end-to-end CI/CD pipeline on AWS ✔️ Automated build, storage, and deployment processes ✔️ Used S3 for secure artifact management ✔️ Configured CodeDeploy for seamless application deployment ✔️ Managed secure access using IAM roles ✔️ Reduced manual intervention and improved deployment speed 🔹 Workflow GitHub → CodeBuild → S3 (Artifacts) → CodeDeploy → EC2 Deployment 🚀 This project strengthened my understanding of AWS DevOps services, automation pipelines, and scalable deployment strategies. #AWS #CICD #DevOps #CodeBuild #CodeDeploy #S3 #EC2 #NodeJS #CloudComputing #Automation #Learning