Automation with Bash scripting for Log Rotation and Backup

Day 19 of #90DaysOfDevOps 💻🔥 Today I built my first real-world automation scripts using Shell Scripting. ✔ Created a log rotation script (cleanup + compression) ✔ Built a server backup script using .tar.gz ✔ Automated tasks using crontab scheduling 💡 Biggest learning: Automation isn’t just writing scripts — it’s about making systems run without manual effort. ⚡ Real-world DevOps use: These concepts are used in log management, server backups, and scheduled maintenance in production systems. From learning → to building → to automating 🚀 #DevOps #Linux #ShellScripting #Automation #Crontab #90DaysOfDevOps

Bash Scripting for DevOps — Part 11/? Till now, I was passing values using environment variables. That worked. But I realized something. Sometimes, I don’t want to set variables separately.I just want to pass values directly when running the script. That’s where arguments come in. In Bash, we can pass values like this: ./deploy.sh staging Inside the script, we can access it using: echo "Deploying to $1 environment" Here, $1 means the first argument. So if I run: ./deploy.sh prod It becomes: Deploying to prod environment This makes scripts much more flexible. Instead of editing the script or setting variables, I can just pass what I need at runtime. This is used a lot in real DevOps workflows: • passing environment names • passing versions or tags • controlling script behavior dynamically Small change. But now the script feels more like a real tool, not just a fixed set of commands. #DevOps #BashScripting #Linux #Automation #DevOpsJourney #LearningInPublic

Day 19 of learning and practicing DevOps 🔁 Today all about building actual automation scripts Worked on: • Log rotation script to compress and clean old logs • Backup script to create timestamped archives • Scheduling jobs using crontab • Combining everything into a maintenance script Important part: Understanding why log rotation and backups matter — without them, logs can fill up disk space and break systems. learning today --> automation + scheduling Instead of manually managing logs and backups, scripts + cron can handle everything in the background. This one is actually used in production environments. Here are my notes: https://lnkd.in/gwQUKK8b 📍 #DevOps #Linux #ShellScripting #Automation #Crontab #LearningInPublic #90DaysOfDevOps #TrainWithShubham

DevOps Zero to Job-Ready – Day 4/180 | Linux Text Processing Today I focused on something that’s actually used a lot in real DevOps work — text processing. What I explored: Searching logs using grep Case-insensitive search (-i) Finding exact lines (-n) Using pipes (|) to combine commands What I realized: In real environments, logs can be huge. You can’t manually scroll and find issues. You need to search smartly. Debugging failed deployments Investigating production issues Sharing today’s notes as well for anyone learning along. I’m also organizing all structured notes, real-world scenarios, and project-based learning content on www.engidock.com — useful if you want a more guided DevOps learning path. Next: Bash scripting basics Trying to stay consistent with this series — one step at a time. #EngiDock #DevOps #Linux #LearningInPublic #Debugging

Day 06 of #90DaysOfDevOps 🚀 Linux File I/O: A Core DevOps Skill Today I revisited the basics — this time thinking like an operator, not a student. Reading and writing files is part of daily DevOps work: → Tailing logs during an incident at 2 AM → Updating configs in deployments → Capturing script output for debugging → Managing YAML, Dockerfiles, and shell scripts What I practiced today: ✅ `echo "..." > file.txt` — overwrite with intention (one wrong `>` can wipe a config file) ✅ `echo "..." >> file.txt` — safe append for logs and configs ✅ `echo "..." | tee -a file.txt` — write AND display simultaneously ✅ `cat`, `head`, `tail` — inspect file contents at different granularities 💡 Command I'll use often: `tee` Perfect for scripts where output needs to be both visible and logged at the same time. 🔑 Key Insight: Every config, log, Dockerfile, and manifest is just text. Engineers who handle text efficiently troubleshoot faster and ship faster. #90DaysOfDevOps #DevOpsKaJosh #TrainWithShubham #Linux #DevOps

💻 Exploring Shell Scripting: Small Commands, Big Impact Another step forward in my DevOps journey 🚀 Shell scripting is more than just writing commands — it’s about: ✔️ Automating repetitive tasks ✔️ Improving efficiency ✔️ Building scalable workflows 🔑 Key areas I worked on: • Bash scripting & execution • Variables and arguments • Control structures (if, for, while) • Automating daily tasks 💡 Why it matters? Because automation is the backbone of DevOps — saving time, reducing errors, and ensuring consistency. “The best way to predict the future is to automate it.” #ShellScripting #DevOps #Automation #Linux #ContinuousLearn

Ever typed the same Linux commands again and again and thought, “There has to be a better way”? That is exactly where shell scripting starts to make sense. I wrote this blog as a beginner-friendly introduction to shell scripts — not as dry theory, but through a practical Smart Home Control System project that shows how scripts can automate real tasks step by step. In this blog, you’ll learn: 1) What a shell script actually is 2) Why #!/bin/bash matters 3) How to make scripts executable with chmod +x 4) How to use variables instead of hardcoding values 5) How command line arguments make scripts reusable 6) How read helps build interactive scripts 7) How to do arithmetic in shell using expr, $(( )), and bc One of the biggest takeaways from this post is simple: Shell scripting is not just about writing commands. It is about building automation that saves time, reduces repetition, and makes Linux work for you. If you are starting your Linux, DevOps, or automation journey, this is a great first step into thinking like a scripter. Read the full blog here: https://lnkd.in/grgWN5vB What should I write about next? Feel free to comment below & I’ll try to create a post on your suggestion within a day. I can cover topics like: Git, Ansible, Jenkins, Groovy, Terraform, AWS, Networking, Linux, DevOps practices, Cloud architecture, CI/CD pipelines, Infrastructure as Code, or anything related. If you find the content useful, please share it with your network and drop a like 👍 it really helps these posts reach more Linux, DevOps, and Cloud folks. Your likes and shares are what keep me motivated to keep writing consistently. Thanks in advance for your ideas and support! #Linux #ShellScripting #Bash #DevOps #Automation #LinuxBasics #LearningLinuxBasics #Scripting #SystemAdministration

Hi All, Let's understand and learn more about the Cronjobs in Linux In real systems, not every task needs to run all the time. Some tasks need to run at specific times. For example, taking backups at night or cleaning logs daily. Cron jobs help us do this automatically. A cron job is a scheduler. You tell it when to run and what command to run. The format looks like this: 𝗺𝗶𝗻𝘂𝘁𝗲 𝗵𝗼𝘂𝗿 𝗱𝗮𝘆-𝗼𝗳-𝗺𝗼𝗻𝘁𝗵 𝗺𝗼𝗻𝘁𝗵 𝗱𝗮𝘆-𝗼𝗳-𝘄𝗲𝗲𝗸 𝗰𝗼𝗺𝗺𝗮𝗻𝗱 Each field means: minute → 0 to 59 hour → 0 to 23 day of month → 1 to 31 month → 1 to 12 day of week → 0 to 7 (Sunday) Examples: 0 2 * * * /home/sai/backup sh Runs every day at 2 AM */5 * * * * /home/sai/health_check sh Runs every 5 minutes Basic commands: 𝗰𝗿𝗼𝗻𝘁𝗮𝗯 -𝗲 → 𝗲𝗱𝗶𝘁 𝗰𝗿𝗼𝗻 𝗷𝗼𝗯𝘀 𝗰𝗿𝗼𝗻𝘁𝗮𝗯 -𝗹 → 𝗹𝗶𝘀𝘁 𝗰𝗿𝗼𝗻 𝗷𝗼𝗯𝘀 𝗰𝗿𝗼𝗻𝘁𝗮𝗯 -𝗿 → 𝗿𝗲𝗺𝗼𝘃𝗲 𝗰𝗿𝗼𝗻 𝗷𝗼𝗯𝘀 In production, cron jobs are used for 𝗯𝗮𝗰𝗸𝘂𝗽𝘀, 𝗹𝗼𝗴 𝗰𝗹𝗲𝗮𝗻𝘂𝗽, 𝗱𝗮𝘁𝗮 𝘀𝘆𝗻𝗰, 𝗮𝗻𝗱 𝗿𝗲𝗽𝗼𝗿𝘁 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻. But they need monitoring. If a cron job fails, you may not notice immediately. A failed backup or cleanup can create problems later. Another common issue is overlapping runs. If a job takes longer than expected and runs again, it can cause duplicate work or conflicts. The idea is simple. Cron jobs run small tasks at the right time and keep systems running smoothly without manual effort. refer the below links for more information: https://lnkd.in/gZJHps67 https://lnkd.in/gSKFhqza https://lnkd.in/gpEp_grE #cronjobs #crontab #automation #scheduledtasks #devops #sre #K8s #backups #cloudengineer #devopengineer #DevOps #DevOpsEngineer #linux #linuxadmin #cronscheduler #infracommunity #devopscommunity #linuxcommunity #redhat #developers #cicd

Managing logs on Linux sounds simple… until it isn’t. While working on systems, I noticed how quickly things can go wrong: * Disk fills up without warning * Thousands of small files eat space silently * Manual cleanup feels risky * And there’s no clear visibility into what actually changed So I built something to solve this properly. I created a **production-style Linux log cleanup tool in Bash** — not just to delete logs, but to make the process safe, visible, and automated. Here’s what it does: 🔹 Runs in **dry-run mode by default** (no accidental deletions) 🔹 Cleans logs using **time + size based strategies** 🔹 Handles both **journalctl logs and custom directories** 🔹 Uses a **lock mechanism** to prevent concurrent runs 🔹 Sends **Slack & Email notifications** after execution 🔹 Supports **cron automation** for scheduled cleanup 🔹 Provides a clear summary of what changed But the real learning came from the challenges: * Handling permissions for `/var/log` * Dealing with limitations of `journalctl` * Debugging email setup with `msmtp` * Designing everything around **safety first** This project helped me understand something important: 👉 Writing scripts is easy. 👉 Building something that behaves safely in production is not. If you’re into DevOps or system engineering, I’d love your feedback. Link in comment #DevOps #Linux #Bash #Automation #SRE #CloudEngineering #OpenSource #SystemDesign #LearningInPublic #Engineering

Day 18 of #90DaysOfDevOps 💻🔥 Today I leveled up my Shell Scripting skills by learning how to write cleaner and reusable scripts. ✔ Created and used functions ✔ Worked with return values & local variables ✔ Learned strict mode (set -euo pipefail) for safer scripts ✔ Built intermediate scripts for real scenarios 💡 Biggest learning: Using strict mode helps catch errors early and makes scripts production-ready. ⚡ Real-world DevOps use: Functions + strict scripting are used in automation scripts, CI/CD pipelines, and system monitoring tools. Slowly moving from basic commands to real automation 🚀 #DevOps #Linux #ShellScripting #Automation #90DaysOfDevOps