Log Analyzer and Report Generator in Bash Scripting

Day 20 of learning and practicing DevOps 🔁 Worked on scripting project — building a log analyzer and report generator Worked on: • Reading and validating log file input • Counting errors (ERROR, Failed) using grep • Extracting critical events with line numbers • Finding top 5 recurring errors using awk, sort, uniq • Generating a structured report file • Archiving processed logs automatically Important part: Instead of manually reading logs, I built a script that analyzes everything and gives a summary in seconds. Learning today--> logs tell the story turning raw logs into useful insights. Here are my notes: https://lnkd.in/ga8xUT6U 📍 #DevOps #Linux #ShellScripting #Automation #LogAnalysis #LearningInPublic #90DaysOfDevOps #TrainWithShubham

⚡ Day 16 – Shell Scripting Fundamentals in DevOps Today’s focus was on making my scripts more interactive and reliable. I worked through: 🔹 User input with read — capturing numbers, strings, and service names directly from the terminal. 🔹 Conditional logic (if-else) — comparing numbers, checking strings, and handling multiple cases with elif. 🔹 File checks (-f) — writing a script that asks for a filename and confirms if it exists. 🔹 Service checks (systemctl) — building a script that asks for a service name and reports if it’s active or not. 🔹 Troubleshooting syntax errors — learning why spaces around [ ] and using the right operators (= for strings, -eq for numbers) are critical in Bash. 💡 Key takeaway: Shell scripting is unforgiving about syntax, but that’s what makes it powerful. Every space, operator, and quote matters — and mastering these details builds the foundation for automation in DevOps. #90DaysOfDevOps #Day15 #DevOpsKaJosh #TrainWithShubham #Linux #ShellScripting #LearnInPublic

Day 19 of #90DaysOfDevOps 💻🔥 Today I built my first real-world automation scripts using Shell Scripting. ✔ Created a log rotation script (cleanup + compression) ✔ Built a server backup script using .tar.gz ✔ Automated tasks using crontab scheduling 💡 Biggest learning: Automation isn’t just writing scripts — it’s about making systems run without manual effort. ⚡ Real-world DevOps use: These concepts are used in log management, server backups, and scheduled maintenance in production systems. From learning → to building → to automating 🚀 #DevOps #Linux #ShellScripting #Automation #Crontab #90DaysOfDevOps

## 🐧 Decoding Linux Pipes: Anonymous vs. Named Ever wondered how data flows seamlessly between processes in Linux? It’s all about the **Pipe**. Whether you're a DevOps engineer or a curious dev, understanding Inter-Process Communication (IPC) is a game-changer for system performance. Here is a quick breakdown of the two main types: ### 1. Anonymous Pipes (The "Quick & Dirty") These are the unsung heroes of the command line. When you run ls | grep .txt, you’re using an anonymous pipe. * **Scope:** Limited to parent-child processes. * **Lifespan:** Temporary; they vanish the moment the execution finishes. * **Setup:** No file entry—it’s all happening in the kernel's memory. ### 2. Named Pipes (The "FIFO" Method) Need two completely unrelated processes to talk? Enter the Named Pipe, created via mkfifo. * **Scope:** Any two processes can communicate. * **Lifespan:** Persistent. It exists as a special file in your filesystem until you manually delete it. * **Visibility:** You’ll see it marked with a p type when running ls -l. **Pro Tip:** Use Anonymous pipes for simple, linear data transformations and Named pipes when building more complex, modular systems that require asynchronous communication. **Which one do you find yourself using more often in your workflows? Let's discuss below! 👇** #Linux #DevOps #SystemArchitecture #Programming #CodingTips #BackendDevelopment #LinuxKernel #TechEducation

🚀 Day 18 – Shell Scripting Level Up! Today I focused on writing cleaner, safer, and reusable shell scripts — a big step from basic scripting to real-world usage 💻 What I learned: ✔️ Writing and calling functions for reusable code ✔️ Using set -euo pipefail for safer scripts ✔️ Handling return values & local variables ✔️ Building a complete system info script One important takeaway: Using set -euo pipefail makes your scripts more reliable and production-ready by preventing silent failures. Key Learnings: Functions = Cleaner & reusable code Strict mode = Safer & error-free scripts Check out my work:https://lnkd.in/g4TvriXU #90DaysOfDevOps #DevOpsKaJosh #TrainWithShubham #DevOps #ShellScripting #Linux #Automation #Scripting #LearningInPublic #TechJourney #Cloud #Programming #CareerGrowth #ITJobs #Developers #CodeNewbie

🚀 Most Used Linux Commands Every Developer Should Know If you’re working in backend, DevOps, or AI… Linux isn’t optional. It’s your daily toolkit. Here’s a quick breakdown of the commands that actually matter 👇 📂 File Handling Navigate and manage files like a pro → ls, cd, pwd, mkdir, rm 📖 File Viewing Read logs and files efficiently → cat, less, head, tail 🔍 Text Processing (Game Changer) Find and manipulate data fast → grep, awk, sort, find ⚙️ Process Management Control running applications → ps, top, kill, pkill 🌐 Networking Debug APIs & connect to servers → curl, ping, ssh, scp 💾 System Monitoring Know what’s happening inside your machine → df, du, free, uname 📦 Package Management Install tools in seconds → apt, dnf, yum 🔐 Permissions Control access and security → chmod, chown 🧰 Pro Tip: Don’t try to memorize everything. Think in actions: ◾ Search → grep ◾ Navigate → cd ◾ Debug → top Master these, and you can handle 90% of real-world tasks in Linux. 🔥 Reality check: 90% of dev work = just ~10 commands used daily. 💬 Which Linux command do you use the most? 🎯 Follow Virat Radadiya 🟢 for more..... #Linux #DevOps #BackendDevelopment #SoftwareEngineering #Programming #Developers #Coding #CloudComputing #TechSkills #LearnToCode

💻 Exploring Shell Scripting: Small Commands, Big Impact Another step forward in my DevOps journey 🚀 Shell scripting is more than just writing commands — it’s about: ✔️ Automating repetitive tasks ✔️ Improving efficiency ✔️ Building scalable workflows 🔑 Key areas I worked on: • Bash scripting & execution • Variables and arguments • Control structures (if, for, while) • Automating daily tasks 💡 Why it matters? Because automation is the backbone of DevOps — saving time, reducing errors, and ensuring consistency. “The best way to predict the future is to automate it.” #ShellScripting #DevOps #Automation #Linux #ContinuousLearn

Bash got sharper today. Not more complex… just more precise. I spent time on 🅂🄴🄳 and 🄵🄸🄽🄳, and it changed how I look at working with files and data. Before this, I was mostly reading and running commands. Now it feels like I can actually 𝐦𝐨𝐝𝐢𝐟𝐲 𝐚𝐧𝐝 𝐥𝐨𝐜𝐚𝐭𝐞 𝐭𝐡𝐢𝐧𝐠𝐬 𝐰𝐢𝐭𝐡 𝐢𝐧𝐭𝐞𝐧𝐭. 🅂🄴🄳 made it clear how powerful in place editing can be. Not just viewing data, but transforming it instantly. And 🄵🄸🄽🄳 showed me something simple but powerful: If you know how to search properly, you stop wasting time guessing. Thats the shift. From “where is this file?” to “𝐈 𝐤𝐧𝐨𝐰 𝐞𝐱𝐚𝐜𝐭𝐥𝐲 𝐡𝐨𝐰 𝐭𝐨 𝐥𝐨𝐜𝐚𝐭𝐞 𝐚𝐧𝐝 𝐚𝐜𝐭 𝐨𝐧 𝐢𝐭.” Still early, but Bash is starting to feel less reactive… and more controlled. Whats one command that made your workflow faster instantly? 👇 #Linux #Bash #DevOps #AIOps #Automation #LearningInPublic

Day 4 of 90 — My Bash scripts can now think, repeat, and run themselves. Today I learned the 3 things that turn basic scripts into real automation: 1️⃣ Variables — store values once, use everywhere 2️⃣ Conditions — if disk > 80% then alert, else OK 3️⃣ Loops — check 10 servers in 3 lines of code And then I learned Cron Jobs — Linux's alarm clock. Now my server health script runs automatically every 5 minutes. Zero manual work. I wrote smart-monitor.sh from scratch today: ✅ Checks disk usage against a threshold ✅ Verifies nginx is running — alerts if not ✅ Loops through services and reports status ✅ Saves output to a dated log file automatically This is what DevOps automation actually looks like. Not clicking buttons — writing scripts that work while you sleep. Code on GitHub 🔗 https://lnkd.in/gmka7Vk7 Beginner tip: Cron syntax looks scary at first. Remember: MIN HOUR DAY MONTH WEEKDAY */5 * * * * = every 5 minutes. That's it. Day 5 tomorrow: Git deep dive 🌿 #DevOps #Linux #Bash #Automation #CronJobs #90DayChallenge #LearningInPublic

Day 19 of #90DaysOfDevOps - Log Rotation, Backup & Crontab Today I worked on a real-world Shell Scripting project where I built automation scripts 🔧 What I built: • Log Rotation Script – compress old logs & clean up storage • Backup Script – create timestamped backups automatically • Crontab Setup – schedule jobs like a real server • Maintenance Script – combined everything into one automation ⏰ Automated tasks: • Log rotation → daily at 2 AM • Backup → every Sunday at 3 AM • Health check → every 5 minutes • Full maintenance → daily at 1 AM Checkout my work: https://lnkd.in/gJkC8_5Z #90DaysOfDevOps #DevOps #Linux #ShellScripting #Automation #Cloud #AWS #LearningInPublic #BuildInPublic #DevOpsJourney #Programming #Coding #Tech #CareerGrowth #Consistency #DevOpsKaJosh #TrainWithShubham