Linux Pipes: Anonymous vs Named

Managing logs on Linux sounds simple… until it isn’t. While working on systems, I noticed how quickly things can go wrong: * Disk fills up without warning * Thousands of small files eat space silently * Manual cleanup feels risky * And there’s no clear visibility into what actually changed So I built something to solve this properly. I created a **production-style Linux log cleanup tool in Bash** — not just to delete logs, but to make the process safe, visible, and automated. Here’s what it does: 🔹 Runs in **dry-run mode by default** (no accidental deletions) 🔹 Cleans logs using **time + size based strategies** 🔹 Handles both **journalctl logs and custom directories** 🔹 Uses a **lock mechanism** to prevent concurrent runs 🔹 Sends **Slack & Email notifications** after execution 🔹 Supports **cron automation** for scheduled cleanup 🔹 Provides a clear summary of what changed But the real learning came from the challenges: * Handling permissions for `/var/log` * Dealing with limitations of `journalctl` * Debugging email setup with `msmtp` * Designing everything around **safety first** This project helped me understand something important: 👉 Writing scripts is easy. 👉 Building something that behaves safely in production is not. If you’re into DevOps or system engineering, I’d love your feedback. Link in comment #DevOps #Linux #Bash #Automation #SRE #CloudEngineering #OpenSource #SystemDesign #LearningInPublic #Engineering

Hi All, Let's understand and learn more about the Cronjobs in Linux In real systems, not every task needs to run all the time. Some tasks need to run at specific times. For example, taking backups at night or cleaning logs daily. Cron jobs help us do this automatically. A cron job is a scheduler. You tell it when to run and what command to run. The format looks like this: 𝗺𝗶𝗻𝘂𝘁𝗲 𝗵𝗼𝘂𝗿 𝗱𝗮𝘆-𝗼𝗳-𝗺𝗼𝗻𝘁𝗵 𝗺𝗼𝗻𝘁𝗵 𝗱𝗮𝘆-𝗼𝗳-𝘄𝗲𝗲𝗸 𝗰𝗼𝗺𝗺𝗮𝗻𝗱 Each field means: minute → 0 to 59 hour → 0 to 23 day of month → 1 to 31 month → 1 to 12 day of week → 0 to 7 (Sunday) Examples: 0 2 * * * /home/sai/backup sh Runs every day at 2 AM */5 * * * * /home/sai/health_check sh Runs every 5 minutes Basic commands: 𝗰𝗿𝗼𝗻𝘁𝗮𝗯 -𝗲 → 𝗲𝗱𝗶𝘁 𝗰𝗿𝗼𝗻 𝗷𝗼𝗯𝘀 𝗰𝗿𝗼𝗻𝘁𝗮𝗯 -𝗹 → 𝗹𝗶𝘀𝘁 𝗰𝗿𝗼𝗻 𝗷𝗼𝗯𝘀 𝗰𝗿𝗼𝗻𝘁𝗮𝗯 -𝗿 → 𝗿𝗲𝗺𝗼𝘃𝗲 𝗰𝗿𝗼𝗻 𝗷𝗼𝗯𝘀 In production, cron jobs are used for 𝗯𝗮𝗰𝗸𝘂𝗽𝘀, 𝗹𝗼𝗴 𝗰𝗹𝗲𝗮𝗻𝘂𝗽, 𝗱𝗮𝘁𝗮 𝘀𝘆𝗻𝗰, 𝗮𝗻𝗱 𝗿𝗲𝗽𝗼𝗿𝘁 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻. But they need monitoring. If a cron job fails, you may not notice immediately. A failed backup or cleanup can create problems later. Another common issue is overlapping runs. If a job takes longer than expected and runs again, it can cause duplicate work or conflicts. The idea is simple. Cron jobs run small tasks at the right time and keep systems running smoothly without manual effort. refer the below links for more information: https://lnkd.in/gZJHps67 https://lnkd.in/gSKFhqza https://lnkd.in/gpEp_grE #cronjobs #crontab #automation #scheduledtasks #devops #sre #K8s #backups #cloudengineer #devopengineer #DevOps #DevOpsEngineer #linux #linuxadmin #cronscheduler #infracommunity #devopscommunity #linuxcommunity #redhat #developers #cicd

🚀 Day 13: Linux Internals for DevOps Engineers (Advanced) 👉 Debugging APIs & Services Like a Real Engineer Earlier, I learned how to check if a service is running. Today, I went one level deeper: 👉 Understanding HOW the service responds. 📌 What I explored: 🔹 curl -v to see full request/response flow 🔹 Sending POST requests using curl 🔹 Using netcat (nc) to test ports and simulate connections 🔹 Manually sending HTTP requests (raw debugging) 💡 Real Scenario: Frontend is not working… But is the problem in frontend or backend? 👉 Using curl -v, I saw a 500 Internal Server Error Now I know: ✔ Backend issue ✔ Not a network problem This kind of debugging saves hours. 🧠 Question for you: Have you ever used curl to debug an API issue? What did you find? 👇 Let’s discuss! 🎯 Learning Goal: To move from basic checks → deep debugging of services and APIs. 📅 Day 14 Tomorrow: HTTP/HTTPS Deep Dive (Headers, Status Codes, SSL) Let’s keep going deeper 🚀 #DevOps #Linux #Networking #APIDebugging #SRE #CloudComputing #SoftwareEngineering #TechLearning #LearningInPublic #ITCareers #EngineeringMindset #CareerGrowth

Ever wondered what powers most of the command-line magic in Linux? It’s the shell, the interface between you and the operating system. Let’s break down the most popular ones 👇 🔹 What is a Shell? A shell is a command-line interpreter that allows users to interact with the OS by executing commands, running scripts, and automating tasks. 🔸 1. sh (Bourne Shell) The original Unix shell and the foundation for many others. ✔️ Simple and lightweight ✔️ Highly portable across systems ❌ Limited features (no advanced scripting capabilities) Example: #!/bin/sh echo "Hello from sh" 👉 Used in: System scripts, POSIX-compliant environments 🔸 2. csh (C Shell) Designed with a syntax similar to the C programming language. ✔️ Easier for C programmers ✔️ Supports aliases and history ❌ Not ideal for scripting (less predictable behavior) Example: #!/bin/csh echo "Hello from csh" 👉 Used in: Legacy systems, interactive use (rare today) 🔸 3. bash (Bourne Again Shell) The most widely used shell in Linux today. ✔️ Powerful scripting capabilities ✔️ Command history, tab completion ✔️ Backward compatible with sh ✔️ Huge community support Example: #!/bin/bash name="Linux" echo "Hello from $name" 👉 Used in: DevOps, automation, scripting, default shell in most Linux distros 🔸 4. zsh (Z Shell) An advanced shell built on top of bash with enhanced features. ✔️ Better auto-completion ✔️ Plugin & theme support (Oh My Zsh ❤️) ✔️ Smarter navigation ✔️ Highly customizable Example: #!/bin/zsh echo "Hello from zsh" 👉 Used in: Modern developer environments, productivity-focused workflows Why Shells Matter? Automate repetitive tasks Manage systems efficiently Write powerful deployment scripts Core skill for DevOps, Cloud, and SRE roles 🚀 My Take: If you're starting → go with bash If you want productivity & customization → explore zsh If you’re dealing with legacy systems → you might still see sh/csh Mastering shells is not just about commands — it's about thinking in automation. #Linux #DevOps #ShellScripting #Automation #Cloud #Engineering

🚀 Linux Commands Every Developer Should Know Linux is no longer “nice to have” — it’s a must-have skill for developers, DevOps engineers, and backend builders. Here are the command categories you should master 👇 📂 Navigation & Filesystem pwd, ls, cd → move through directories like second nature 📄 File Operations cp, mv, rm, touch → manage files with confidence 🔐 Permissions chmod, chown → control access and security ⚙️ Process Management ps, top, kill → monitor and handle running processes 🌐 Networking ping, curl, netstat → debug connectivity issues quickly 💽 Disk & Storage df, du → understand and manage system usage 🔎 Search & Text Processing grep, awk, sed → filter and transform data like a pro 👤 User Management useradd, usermod, passwd → manage users securely 💡 Tip: Don’t just memorize commands—use them daily. Build, break, fix… repeat. That’s how real learning happens. If you're working with scalable systems, Linux isn’t optional—it's your foundation. #Linux #DevOps #BackendDevelopment #Programming #SoftwareEngineering #SysAdmin #TechSkills

🚀 Most Used Linux Commands Every Developer Should Know If you’re working in backend, DevOps, or AI… Linux isn’t optional. It’s your daily toolkit. Here’s a quick breakdown of the commands that actually matter 👇 📂 File Handling Navigate and manage files like a pro → ls, cd, pwd, mkdir, rm 📖 File Viewing Read logs and files efficiently → cat, less, head, tail 🔍 Text Processing (Game Changer) Find and manipulate data fast → grep, awk, sort, find ⚙️ Process Management Control running applications → ps, top, kill, pkill 🌐 Networking Debug APIs & connect to servers → curl, ping, ssh, scp 💾 System Monitoring Know what’s happening inside your machine → df, du, free, uname 📦 Package Management Install tools in seconds → apt, dnf, yum 🔐 Permissions Control access and security → chmod, chown 🧰 Pro Tip: Don’t try to memorize everything. Think in actions: ◾ Search → grep ◾ Navigate → cd ◾ Debug → top Master these, and you can handle 90% of real-world tasks in Linux. 🔥 Reality check: 90% of dev work = just ~10 commands used daily. 💬 Which Linux command do you use the most? 🎯 Follow Virat Radadiya 🟢 for more..... #Linux #DevOps #BackendDevelopment #SoftwareEngineering #Programming #Developers #Coding #CloudComputing #TechSkills #LearnToCode

🚀 Day 20 of 30 – Debugging in Terraform (TF_LOG) When I first started learning Terraform, one big question I had was: 👉 How do you actually see what Terraform is doing under the hood? Turns out — Terraform has a powerful built-in logging system you can enable with a single environment variable. 🔹 What is TF_LOG? TF_LOG is an environment variable that controls Terraform’s logging verbosity. ✔ Helps debug failed plans ✔ Understand provider behavior ✔ Identify state-related issues ✔ No code changes required 🔹 Log Levels (highest → lowest) TRACE ← most detailed DEBUG INFO WARN ERROR ← least detailed 🔹 Enable Logging (Linux / Mac) export TF_LOG=INFO terraform plan 👉 Logs will be printed directly in your terminal 🔹 Store Logs to a File export TF_LOG=INFO export TF_LOG_PATH=terraform.txt terraform plan 👉 Logs will be saved in terraform.txt 👉 Useful for debugging & sharing with teams 🔹 Practical Example resource "local_file" "foo" { content = "foo!" filename = "${path.module}/foo.txt" } Run: export TF_LOG=DEBUG terraform apply 👉 You can see: • How path.module is resolved • File creation steps • Internal Terraform execution flow 🎯 Key Takeaway When Terraform behaves unexpectedly: 👉 Don’t guess 👉 Don’t assume Check the logs first — TF_LOG is your best friend 📅 Tomorrow: Terraform format #30DaysOfTerraform #Terraform #DevOps #CloudEngineering #AWS

Day 19 of #90DaysOfDevOps - Log Rotation, Backup & Crontab Today I worked on a real-world Shell Scripting project where I built automation scripts 🔧 What I built: • Log Rotation Script – compress old logs & clean up storage • Backup Script – create timestamped backups automatically • Crontab Setup – schedule jobs like a real server • Maintenance Script – combined everything into one automation ⏰ Automated tasks: • Log rotation → daily at 2 AM • Backup → every Sunday at 3 AM • Health check → every 5 minutes • Full maintenance → daily at 1 AM Checkout my work: https://lnkd.in/gJkC8_5Z #90DaysOfDevOps #DevOps #Linux #ShellScripting #Automation #Cloud #AWS #LearningInPublic #BuildInPublic #DevOpsJourney #Programming #Coding #Tech #CareerGrowth #Consistency #DevOpsKaJosh #TrainWithShubham

Ever typed the same Linux commands again and again and thought, “There has to be a better way”? That is exactly where shell scripting starts to make sense. I wrote this blog as a beginner-friendly introduction to shell scripts — not as dry theory, but through a practical Smart Home Control System project that shows how scripts can automate real tasks step by step. In this blog, you’ll learn: 1) What a shell script actually is 2) Why #!/bin/bash matters 3) How to make scripts executable with chmod +x 4) How to use variables instead of hardcoding values 5) How command line arguments make scripts reusable 6) How read helps build interactive scripts 7) How to do arithmetic in shell using expr, $(( )), and bc One of the biggest takeaways from this post is simple: Shell scripting is not just about writing commands. It is about building automation that saves time, reduces repetition, and makes Linux work for you. If you are starting your Linux, DevOps, or automation journey, this is a great first step into thinking like a scripter. Read the full blog here: https://lnkd.in/grgWN5vB What should I write about next? Feel free to comment below & I’ll try to create a post on your suggestion within a day. I can cover topics like: Git, Ansible, Jenkins, Groovy, Terraform, AWS, Networking, Linux, DevOps practices, Cloud architecture, CI/CD pipelines, Infrastructure as Code, or anything related. If you find the content useful, please share it with your network and drop a like 👍 it really helps these posts reach more Linux, DevOps, and Cloud folks. Your likes and shares are what keep me motivated to keep writing consistently. Thanks in advance for your ideas and support! #Linux #ShellScripting #Bash #DevOps #Automation #LinuxBasics #LearningLinuxBasics #Scripting #SystemAdministration

If you want to work in DevOps, you need to be comfortable with Linux tools. Here are essential tools every engineer uses: 🔹 File & Text Processing ✔ grep ✔ awk ✔ sed 🔹 System Monitoring ✔ top ✔ htop ✔ vmstat 🔹 Networking ✔ ping ✔ netstat / ss ✔ curl 🔹 Process Management ✔ ps ✔ kill ✔ nice 🔹 Logs & Debugging ✔ tail -f ✔ journalctl ✔ dmesg These tools help you: ✔ Debug production issues ✔ Monitor systems ✔ Fix problems fast Most engineers ignore these. And struggle in real jobs. Tools don’t make you strong. Understanding does. Start mastering these today. Save this list for later. Follow for daily DevOps & Cloud content. #Linux #DevOps #CloudComputing #CareerGrowth #Engineering