Python Log Monitoring with Timestamp & Alerts for DevOps

🚀 Python for DevOps – Log Level Automation Project Today I built a practical DevOps script using Python to analyze logs and separate them based on log levels. 📂 Problem: Manually checking logs is time-consuming. Needed a way to automatically filter and organize logs. 💻 Solution (Python Script): with open("app.log") as f, \ open("error.log", "w") as err, \ open("warning.log", "w") as warn, \ open("info.log", "w") as info: for line in f: if "ERROR" in line: err.write(line) elif "WARNING" in line: warn.write(line) elif "INFO" in line: info.write(line) ####################################### Output: ubuntu@satheesha:~/python$ python3 multiple-log_level.py ubuntu@satheesha:~/python$ ls -ltr error.log warning.log info.log -rw-r--r-- 1 ubuntu ubuntu 18 Apr 21 07:45 warning.log -rw-r--r-- 1 ubuntu ubuntu 44 Apr 21 07:45 info.log -rw-r--r-- 1 ubuntu ubuntu 17 Apr 21 07:45 error.log ubuntu@satheesha:~/python$ cat app.log INFO: Service startes WARNING: High CPU INFO: Service startes ERROR: Disk full ubuntu@satheesha:~/python$ cat error.log ERROR: Disk full ubuntu@satheesha:~/python$ cat warning.log WARNING: High CPU ubuntu@satheesha:~/python$ cat info.log INFO: Service startes INFO: Service startes ######################################### does: Reads app.log Filters logs into: error.log warning.log info.log 📊 Outcome: Faster troubleshooting Organized logs for better monitoring Reduced manual effort 🔥 Real DevOps Use Cases: Production log monitoring CI/CD pipeline validation Incident detection and alerting 💡 Key Learning: Python is a powerful tool for automation in DevOps, especially for handling logs and system data. 📈 Next Step: Enhancing this script to: Count log levels Trigger alerts (email/Slack) Monitor logs in real-time (tail -f style) #Python #DevOps #Automation #Scripting #Cloud #Learning #100DaysOfCode

🚀 Python Basics for DevOps Engineers (Practical Examples) Python is a powerful tool for automation in DevOps. Here’s a quick guide to essential data types with real-world use cases 👇 🔹 1. String (str) Used for text (server names, logs, messages) server = "web-server-1" print(server) 💡 DevOps Example: log = "ERROR: Disk full" if "ERROR" in log: print("Issue found") 🔹 2. Integer (int) Used for numbers (CPU, memory, ports) cpu = 75 print(cpu) 💡 DevOps Example: cpu = 85 if cpu > 80: print("Alert: High CPU") 🔹 3. Boolean (True / False) Used for status (running/stopped, success/failure) is_running = True if is_running: print("Service is running") 💡 DevOps Example: deployment_success = False if not deployment_success: print("Rollback required") 🔹 4. List (list) Used to store multiple values (servers, services) servers = ["web1", "web2", "web3"] print(servers[0]) 💡 DevOps Example: services = ["nginx", "docker", "jenkins"] for service in services: print(service) 🔹 5. Combine All (Real Example) servers = ["web1", "web2"] cpu_usage = 85 status = True if cpu_usage > 80: print("Alert: scale up needed") if status: for s in servers: print(f"{s} CPU: {cpu_usage}") 🔹 6. Quick Practice services = ["web1", "web2"] status = True cpu_usage = 85 # fixed variable name if status: for s in services: print(f"Server {s} CPU {cpu_usage}") if cpu_usage > 80: print(f"Alert: CPU {cpu_usage}") Out put: >>> services = ["web1", "web2"] >>> status = True >>> cup_usage = 85 >>>  >>> if status: ...  for s in services: ...    print(f"server {s} CPU {cpu_usage}") ...  server web1 CPU 85 server web2 CPU 85 >>> if cpu_usage > 80: ...  print(f"Alert: CPU {cpu_usage}") ...  Alert: CPU 85 💡 Key Takeaway: Mastering these basics helps automate monitoring, alerts, and system management in real DevOps environments. #DevOps #Python #Automation #Scripting #Learning #AWS #Kubernetes

🚀 Python for DevOps – API Monitoring with requests Practiced using Python’s requests library to check API health, a common real-world DevOps task. 📂 Use Case: In production, services depend on APIs. We need to continuously verify if APIs are reachable and healthy. 💻 Python Script: import requests url = "https://api.github.com" try: res = requests.get(url, timeout=5) if res.status_code == 200: print("✅ GitHub API is UP") else: print("⚠️ GitHub API issue:", res.status_code) except requests.exceptions.RequestException as e: print("🚨 API call failed:", e) Output: Status_code: 200 Response: {'current_user_url': 'https://lnkd.in/guvkNT7k', 'current_user_authorizations_html_url': 'https://lnkd.in/gx-65ERd', 'authorizations_url': 'https://lnkd.in/gzcehbTu', 'code_search_url': 'https://lnkd.in/gQU8cghE', 'commit_search_url': 'https://lnkd.in/g62A-__n', 'emails_url': 'https://lnkd.in/gXaZyEkK', 'emojis_url': 'https://lnkd.in/gp3Scn2Y', 'events_url': 'https://lnkd.in/grbt4NNg', 'feeds_url': 'https://lnkd.in/gCBk-eSN', 'followers_url': 'https://lnkd.in/gQvSEXqB', 'following_url': 'https://lnkd.in/grh4YDpJ', 🔍 What this does: Sends HTTP request to API Uses timeout to avoid hanging Checks response status Handles failures gracefully 🔥 Why this matters in DevOps: Monitor service availability Validate endpoints in CI/CD pipelines Detect outages early Automate health checks 💡 Key Learning: APIs are everywhere in DevOps, and Python makes it easy to integrate, monitor, and automate systems. 📈 Next Steps: Send alerts (Slack/Email) if API fails Combine with log monitoring scripts Build a full monitoring + alerting system #Python #DevOps #API #Automation #Monitoring #Scripting #Cloud #Learning #100DaysOfCode

🚀 Python for DevOps – Stop Learning, Start Automating Most people learn Python… But very few use it the right way in DevOps. Here’s the truth 👇 👉 You don’t need deep theory. 👉 You need practical automation skills. 🔹 What to Focus On (DevOps Style) ✔ Variables, loops, conditions ✔ Functions ✔ File handling (logs, configs) ✔ Error handling (try/except) ✔ Key modules: os → system operations subprocess → run shell commands json / yaml → config management 🔧 Real Example: Run Linux Command Using Python import subprocess result = subprocess.run(["df", "-h", "/"], capture_output=True, text=True) lines = result.stdout.strip().split("\n") if len(lines) > 1:     parts = lines[1].split()     usage = int(parts[4].replace("%", ""))     if usage > 80:         print(f"🚨 ALERT: Disk usage is {usage}%")     else:         print(f"✅ OK: Disk usage is {usage}%") else:     print("❌ Unexpected output:", result.stdout) Output: ubuntu@satheesha:~/python$ python3 Disk-Usage.py ✅ OK: Disk usage is 3% 💡 What This Shows ✔ You can interact with the OS ✔ You can parse real-time system data ✔ You can build automation scripts ✔ You think like a DevOps Engineer 🎯 How I Explain This in Interviews “I use Python’s subprocess module to execute system commands, parse outputs, and automate monitoring tasks like disk usage alerts.” 🔥 Pro Tip Take it one step further: Send alerts to Slack/Email 📩 Schedule with cron ⏰ Integrate with Jenkins 🔁 💬 If you're learning DevOps, stop just writing scripts… Start solving real problems. #DevOps #Python #Automation #Linux #SRE #Cloud #Jenkins #Learning

🚀 Python for DevOps: Real-Time System Monitoring Script (CPU + Memory + Disk) One of the most practical skills in DevOps is automating system monitoring. Instead of manually checking servers, I built a simple Python script that: ✅ Monitors CPU usage ✅ Tracks Memory consumption ✅ Checks Disk utilization ✅ Triggers alerts when thresholds are exceeded 💻 Full Script import psutil import shutil # Thresholds CPU_THRESHOLD = 80 MEM_THRESHOLD = 80 DISK_THRESHOLD = 80 def check_cpu(): cpu = psutil.cpu_percent(interval=1) if cpu > CPU_THRESHOLD: print(f"ALERT: High CPU usage: {cpu}%") else: print(f"OK: CPU usage: {cpu}%") def check_memory(): mem = psutil.virtual_memory() usage = mem.percent if usage > MEM_THRESHOLD: print(f"ALERT: High Memory usage: {usage}%") else: print(f"OK: Memory usage: {usage}%") def check_disk(): disk = shutil.disk_usage("/") usage = (disk.used / disk.total) * 100 if usage > DISK_THRESHOLD: print(f"ALERT: High Disk usage: {usage:.2f}%") else: print(f"OK: Disk usage: {usage:.2f}%") def main(): print("===== System Monitoring =====") check_cpu() check_memory() check_disk() if __name__ == "__main__": main() Output: ubuntu@satheesha:~/python$ python3 full-monitor-script.py ===== System Monitoring ===== OK: CPU usage: 1.0% OK: Memory usage: 30.5% OK: Disk usage: 2.74% ⚙️ How I Used It Installed dependency using: sudo apt install python3-psutil Ran the script to get real-time system health Can be scheduled using cron for continuous monitoring 🔥 Why This Matters in DevOps 👉 Helps detect issues before outages 👉 Reduces manual effort 👉 Can be extended to send alerts (Email / Slack / SNS) 👉 Foundation for tools like monitoring agents 🎯 Key Learning "Don’t just run commands like top or df -h — automate them using Python and build intelligent monitoring." 🚀 Next Steps I’m planning to: Integrate this with Jenkins pipeline Send alerts to Slack Push metrics to monitoring tools 💬 How do you monitor your servers in real-time? #DevOps #Python #Automation #Monitoring #SRE #Cloud #Linux #Jenkins #Learning #100DaysOfCode

🚀 Python Basics for DevOps Engineers (With Practical Examples) Python is a must-have skill for DevOps. Here are some basic concepts with real-time examples 👇 🔹 1. Variables name = "server1" cpu_usage = 75 is_running = True print(name) print(cpu_usage) 💡 DevOps Example: server = "web-server" status = "running" print(server, status) 🔹 2. Data Types String → "hello" Integer → 10 Boolean → True / False List → ["app1", "app2"] services = ["nginx", "docker", "jenkins"] print(services[0]) # nginx 🔹 3. Conditions (if-else) Used for decision-making in automation cpu = 85 if cpu > 80: print("High CPU usage") else: print("Normal CPU") 💡 DevOps Example: disk = 90 if disk > 80: print("Alert: Disk almost full") else: print("Disk is normal") 🔹 4. Loops Used to repeat tasks (like checking multiple servers) 👉 for loop: servers = ["web1", "web2", "web3"] for s in servers: print(s) 👉 while loop: i = 1 while i <= 5: print(i) i += 1 🔹 5. Functions Reusable code (very important for scripting) def check_cpu(cpu): if cpu > 80: print("Alert: High CPU") else: print("Normal CPU") check_cpu(85) 🔹 6. Real DevOps Example servers = ["web1", "web2", "web3"] def check_status(server): print(f"Checking {server}...") for s in servers: check_status(s) 🔹 7. Mini Practice cpu = 70 if cpu > 80: print("Alert: scale up server") else: print("Server is stable") 💡 Key Takeaway: Python helps automate repetitive tasks like monitoring, alerts, and server management in DevOps. #DevOps #Python #Automation #Scripting #Learning #AWS #Kubernetes

🚀 Python for DevOps – Triggering Jenkins Jobs via API Explored how to automate CI/CD by triggering Jenkins jobs using Python and APIs — a key real-world DevOps capability. 📂 Use Case: Instead of manually clicking “Build Now” in Jenkins, we can trigger jobs programmatically using APIs. 💻 Python Script: import requests jenkins_url = "http://your-jenkins-url/job/your-job-name/build" username = "your-username" api_token = "your-api-token" response = requests.post(jenkins_url, auth=(username, api_token)) if response.status_code == 201: print("✅ Jenkins job triggered successfully") else: print("❌ Failed to trigger job:", response.status_code) 🔐 Handling CSRF (Crumb Token): crumb_url = f"{jenkins_url}/crumbIssuer/api/json" crumb_data = requests.get(crumb_url, auth=(username, api_token)).json() headers = { crumb_data['crumbRequestField']: crumb_data['crumb'] } ⚙️ Trigger Job with Parameters: params = { "ENV": "prod", "VERSION": "1.0" } response = requests.post( "http://your-jenkins-url/job/your-job-name/buildWithParameters", params=params, auth=("user", "token") ) 🔍 What this enables: Automate CI/CD pipelines Trigger builds from scripts or monitoring tools Integrate Jenkins with other systems Reduce manual intervention 🔥 Why this matters in DevOps: Automation is the backbone of DevOps. Using APIs, we can connect tools and build end-to-end automated workflows. 💡 Key Learning: Jenkins + APIs + Python = powerful combination for pipeline automation and integration. 📈 Next Steps: Trigger Jenkins from log monitoring script Send build status to Slack Integrate with cloud deployments (AWS) #DevOps #Jenkins #Python #Automation #CICD #API #Cloud #Scripting #Learning #100DaysOfCode

🚀 Why YAML Validation with Python Matters in the DevOps World YAML has become the backbone of modern DevOps workflows. From CI/CD pipelines to infrastructure-as-code and Kubernetes manifests, YAML keeps our configurations clean, readable, and structured. But here’s the catch — a single indentation mistake can break an entire deployment. That’s where Python-based YAML validation becomes a game changer. 🔎 Why Validate YAML with Python? ✅ Early Error Detection Catch syntax and structure issues before they reach production. No more failed deployments due to simple spacing or formatting mistakes. ✅ Automation Friendly Python scripts can be easily integrated into CI/CD pipelines to automatically validate YAML files on every commit or pull request. ✅ Custom Validation Rules Beyond syntax checks, Python allows you to enforce business logic — such as required fields, allowed values, or environment-specific configurations. ✅ Improved Reliability Validated YAML means more stable pipelines, fewer rollbacks, and higher confidence in automated deployments. 🛠️ Why This Matters in DevOps In the DevOps ecosystem, YAML powers tools like deployment pipelines, container orchestration, monitoring setups, and infrastructure definitions. Using Python as a validation layer ensures: Cleaner configuration management Safer deployments Faster debugging cycles Better collaboration across teams 💡 Takeaway: Treat YAML validation as a first-class step in your DevOps workflow. A lightweight Python validator today can save hours of troubleshooting tomorrow. GITHUB: https://lnkd.in/dTDADrzS #DevOps #Python #YAML #Automation #InfrastructureAsCode #CICD #SRE #CloudComputing

🚀 Python for DevOps – Log Analysis with Metrics Today I enhanced my log automation script to not only separate logs by level but also count them for quick insights. 📂 Problem: Manually analyzing logs is slow and inefficient. 💻 Solution (Python Script): error = warning = info_count = 0 with open("app.log") as f, \ open("error.log", "w") as err, \ open("warning.log", "w") as warn, \ open("info.log", "w") as info: for line in f: if "ERROR" in line: err.write(line) error += 1 elif "WARNING" in line: warn.write(line) warning += 1 elif "INFO" in line: info.write(line) info_count += 1 print("ERROR:", error) print("WARNING:", warning) print("INFO:", info_count) 🔍 What this does: Reads app.log Splits logs into separate files Counts each log level 📊 Sample Output: ERROR: 1 WARNING: 1 INFO: 2 Output: ubuntu@satheesha:~/python$ python3 multiple-log_level.py ERROR: 1 WARNING: 1 INFO_COUNT: 2 ubuntu@satheesha:~/python$ ls -ltr error.log warning.log info.log -rw-r--r-- 1 ubuntu ubuntu 18 Apr 21 08:20 warning.log -rw-r--r-- 1 ubuntu ubuntu 44 Apr 21 08:20 info.log -rw-r--r-- 1 ubuntu ubuntu 17 Apr 21 08:20 error.log ubuntu@satheesha:~/python$ cat error.log ERROR: Disk full ubuntu@satheesha:~/python$ cat warning.log WARNING: High CPU ubuntu@satheesha:~/python$ cat info.log INFO: Service startes INFO: Service startes 🔥 Why this matters: Quick visibility into system health Helps prioritize issues (ERROR > WARNING > INFO) Reduces manual troubleshooting time 💡 Key Learning: Python can be used not just for automation, but also for real-time insights and monitoring in DevOps. 📈 Next Step: Add alerting when ERROR count exceeds threshold Integrate with monitoring tools Build real-time log monitoring (tail -f in Python) #Python #DevOps #Automation #Scripting #Monitoring #Cloud #Learning

🚨 I used to overcomplicate Python in DevOps… until real CI/CD pipelines taught me something simple. When I started working with automation, I thought I needed heavy frameworks and advanced Python structures to build “real DevOps scripts”. But in production environments, I realized something very different: 👉 DevOps automation is not about complexity 👉 It’s about using the right simple tools reliably In most CI/CD and cloud automation work, I ended up using only a small set of Python standard library modules: os → environment variables, system interaction subprocess → running real commands (docker, kubectl, terraform) json → APIs, Kubernetes configs, pipeline responses logging → production-grade observability pathlib → clean file and artifact handling datetime → deployment tracking & audit logs sys → CLI control and pipeline exit handling shutil → backups and artifact management Real example from DevOps work: Instead of building complex tools, I often use Python scripts to: automate deployment steps execute validation commands capture logs from CI/CD pipelines interact with cloud APIs The biggest lesson I learned: 👉 In DevOps, simplicity always wins over complexity. Because in production, reliability matters more than clever code. What Python modules do you find yourself using the most in DevOps automation? #DevOps #Python #CloudComputing #CI/CD #Automation #SRE