Log File Analysis with Python for DevOps

🚀 Python for DevOps – Log Analysis with Metrics Today I enhanced my log automation script to not only separate logs by level but also count them for quick insights. 📂 Problem: Manually analyzing logs is slow and inefficient. 💻 Solution (Python Script): error = warning = info_count = 0 with open("app.log") as f, \ open("error.log", "w") as err, \ open("warning.log", "w") as warn, \ open("info.log", "w") as info: for line in f: if "ERROR" in line: err.write(line) error += 1 elif "WARNING" in line: warn.write(line) warning += 1 elif "INFO" in line: info.write(line) info_count += 1 print("ERROR:", error) print("WARNING:", warning) print("INFO:", info_count) 🔍 What this does: Reads app.log Splits logs into separate files Counts each log level 📊 Sample Output: ERROR: 1 WARNING: 1 INFO: 2 Output: ubuntu@satheesha:~/python$ python3 multiple-log_level.py ERROR: 1 WARNING: 1 INFO_COUNT: 2 ubuntu@satheesha:~/python$ ls -ltr error.log warning.log info.log -rw-r--r-- 1 ubuntu ubuntu 18 Apr 21 08:20 warning.log -rw-r--r-- 1 ubuntu ubuntu 44 Apr 21 08:20 info.log -rw-r--r-- 1 ubuntu ubuntu 17 Apr 21 08:20 error.log ubuntu@satheesha:~/python$ cat error.log ERROR: Disk full ubuntu@satheesha:~/python$ cat warning.log WARNING: High CPU ubuntu@satheesha:~/python$ cat info.log INFO: Service startes INFO: Service startes 🔥 Why this matters: Quick visibility into system health Helps prioritize issues (ERROR > WARNING > INFO) Reduces manual troubleshooting time 💡 Key Learning: Python can be used not just for automation, but also for real-time insights and monitoring in DevOps. 📈 Next Step: Add alerting when ERROR count exceeds threshold Integrate with monitoring tools Build real-time log monitoring (tail -f in Python) #Python #DevOps #Automation #Scripting #Monitoring #Cloud #Learning

🚀 Python for DevOps – Real-Time Error Monitoring Practiced building a simple real-time log monitoring script using Python to detect errors instantly. 📂 Use Case: In production, logs are continuously generated. We need a way to detect errors in real time instead of manually checking files. 💻 Python Script: import time with open("app.log", "r") as f: f.seek(0, 2) # move to end of file while True: line = f.readline() if not line: time.sleep(1) continue if "ERROR" in line: print("Alert:", line.strip()) Output: ubuntu@satheesha:~/python$ python3 real-time_log-montr.py Alert ERROR: Test Wed Apr 22 08:05:25 UTC 2026 Alert ERROR: Test Wed Apr 22 08:05:27 UTC 2026 Alert ERROR: Test Wed Apr 22 08:05:29 UTC 2026 Alert ERROR: Test Wed Apr 22 08:05:31 UTC 2026 🔍 What this does: Reads log file in real time Starts from end (like tail -f) Continuously checks for new entries Prints alert when ERROR is detected 🔥 Why this matters: Helps detect issues instantly Reduces downtime Automates monitoring tasks 💡 Key Learning: Python can be used to build lightweight monitoring tools, similar to real-world DevOps systems. 📈 Next Step: Add timestamps Handle multiple log levels Send alerts (email/Slack) Auto-restart services on failure #Python #DevOps #Automation #Monitoring #Logging #Scripting #Learning #100DaysOfCode

Day 2 of building my skills in IT Automation with Python. 🐍 Still early… but something is already shifting. I won't lie; when I started today I wasn't sure what to expect. But by the end of it, I genuinely felt like I was starting to think differently. Not just writing lines of code… but actually reasoning through problems. That's a different feeling. Here's what today looked like: 🔹 Hello, World! Every developer starts here. It's simple, almost too simple; but when that output shows up on your screen for the first time? There's something about it. Like the machine just said "yeah, I hear you." Small win. Big feeling. 🔹 User Input This is where things got interesting. Your program stops being a monologue and starts becoming a conversation. Code that responds to people; that's where the real utility begins. 🔹 Python as a Calculator Numbers, operations, logic. Clean and minimal on the surface but there's a lot of power sitting underneath. The more I explored it the more I realized Python doesn't need to shout to be impressive. 🔹 The Command Line Okay this one felt different. Running scripts straight from the terminal, no cajoling , no buttons; just you and the machine talking directly. I'm not going to pretend I didn't feel a little like a developer in a movie 😄. But real talk, this is a skill that's going to matter a lot(Yup, it does matter, and its cool). 🔹 JupyterLab & Notebooks This tool is genuinely a game changer for learning. Instead of running an entire script every time you want to test something, you just run a block. Fast feedback, faster learning. If you're just starting out - Usually used for scripting tasks when my PC is slow, or when I just want to do something quick Python is backed by one of the biggest developer communities in the world. That last part matters more than people think. Because it means the moment you get stuck- and you will get stuck (believe you me) -there's always a doc, a thread, or a developer somewhere who's already been exactly where you are. You're never really alone in this. Still Day 2. Still learning. Still making mistakes and figuring things out. But the mindset is shifting_ from "how do I write this code" to "how do I think through this problem." That's the real progress. Grateful to Mentor Me Collective and special Thanks to Chanel Power 💡🌍 for sponsoring this journey and making sure I have the structure and resources to actually grow; not just consume content, but build real skills. More coming. Let's keep going. 🚀 #Python #ITAutomation #BuildInPublic #LearningInPublic #MentorMeCollective #TechJourney #Cameroon

If you’re learning Python for tech roles, here’s the progression that covers most use cases: 1. Basics Start with the fundamentals: variables, operators, data types, loops, functions, and control flow. → This stage builds the foundation for everything else. 2. Advanced Python Move into concepts like OOP, decorators, generators, regex, lambda functions, and multithreading. → This is where you understand what’s happening behind the scenes and how Python actually works. 3. Data Structures & Algorithms (DSA) Arrays, strings, linked lists, trees, recursion, and complexity analysis help strengthen problem-solving and coding interview skills. → Great for sharpening problem-solving and interview prep, but depth here can vary depending on your career path. 4. Testing Debugging, unit testing, writing test cases, mocking, and measuring code coverage. → Essential for writing reliable, production-grade Python. 5. Libraries & Databases Learn core libraries like NumPy, Pandas, and Matplotlib, and work with databases such as MySQL, SQLite, or PostgreSQL. → This is where Python starts becoming useful for real-world applications. 6. Specialize by Role From here, the path branches depending on your goals: * Web Dev: Django, Flask, REST APIs * DevOps: Docker, CI/CD, AWS Boto3, GCP SDK, Pulumi, automation * Data Science: Data cleaning, EDA, visualization, statistics * Machine Learning: Scikit-learn, model evaluation, deep learning, AI projects → Choose the direction that aligns with the problems you want to solve. Python isn’t one career path ~ it’s a gateway to many. The key is building the core layers first, then specializing. Which direction are you using Python for right now? Image Credits: yourclouddude _________________________________________ 𝐄𝐧𝐫𝐨𝐥 𝐧𝐨𝐰! 𝐃𝐞𝐯𝐎𝐩𝐬 𝐂𝐨𝐡𝐨𝐫𝐭 𝟒 𝐢𝐬 𝐧𝐨𝐰 𝐨𝐩𝐞𝐧. If you're serious about becoming a world-class DevOps engineer in 2026, this is your path. This isn't another bootcamp. This isn't a tutorial hell with a certificate at the end. This is systems-based training for engineers ready to go from good to exceptional. WHAT YOU'LL BUILD Not toy projects. Not "hello world" apps. Real production-grade systems: → Multi-environment CI/CD pipelines with DevSecOps → Infrastructure as Code that scales across 3+ environments → Production observability with Prometheus, Grafana, and OpenTelemetry Join today 👉 https://lnkd.in/eS3t5NwE

BLOG 03 OF 19 · PYTHON Python for Cloud & DevOps: The Glue That Holds Everything Together By Aditya Girish Padhye · ~5 min read Topic: Python | Series: Cloud & DevOps Learning Journey Python isn't just a programming language in the DevOps world — it's the universal glue. It connects tools, automates workflows, talks to APIs, processes logs, and drives infrastructure. If Linux is the OS of the cloud, Python is its scripting language. “Every cloud engineer I admire writes Python. Not because it's trendy — because it genuinely solves problems faster than anything else.” Why Python Specifically? AWS Lambda runs Python natively. Boto3, the AWS SDK, is a Python library. Ansible playbooks extend with Python. The cloud ecosystem has quietly made Python its first-class citizen, and learning it opens all those doors. Core Language Fundamentals • Data Types & Collections: Lists, dictionaries, sets, and tuples are the building blocks. In DevOps scripts, you're constantly manipulating JSON responses from APIs — and JSON maps directly to Python dicts. • Control Flow: Decision making with if/elif, iteration with for/while loops — what makes automation scripts intelligent rather than just sequential. • Functions & OOPs: Writing reusable functions and understanding classes makes your scripts maintainable. Modular code is the difference between a tool and a mess. • File & Exception Handling: Reading config files, writing logs, handling errors gracefully — production scripts must not crash silently. Advanced Capabilities: Django, Flask & API Handling Flask is lightweight and perfect for building simple REST APIs or internal tooling dashboards. Django provides a full-featured framework for more complex applications. In a DevOps context, Flask is commonly used to build webhook receivers, monitoring endpoints, or automation trigger APIs. • API Handling: Using the requests library to call REST APIs, parse JSON responses, handle auth tokens — a daily DevOps activity. • Advanced Libraries: boto3 for AWS automation, paramiko for SSH automation, subprocess for running shell commands, schedule for job automation. A Real Automation Example One of the most satisfying scripts I wrote used boto3 to automatically tag untagged EC2 instances with their owner and creation date — pulling data from CloudTrail events. That script ran in Lambda on a schedule and eliminated a manual compliance task entirely. That's the power of Python in the cloud: turn a repetitive human task into a zero-maintenance automated process. Start with the basics, build a few automation scripts, then connect them to real AWS services via boto3. The compounding effect on your productivity is remarkable. #Python #DevOps #CloudAutomation #Boto3 #Flask #Django #AWSLambda #CloudEngineer #LearningInPublic #FortuneCloud Aditya Girish Padhye · AWS Cloud & DevOps Engineer ·

🚀 Python for DevOps – Automating Disk Monitoring with subprocess As part of my DevOps learning, I worked on automating system monitoring using Python’s subprocess module. Instead of manually checking disk usage, I built a simple script to monitor it and trigger alerts. 🔧 Here’s the code: import subprocess Run disk usage command result = subprocess.run(["df", "-h", "/"], capture_output=True, text=True) Print full output print(result.stdout) Parse and check usage lines = result.stdout.split("\n") for line in lines[1:]: if line: usage = line.split()[4] # Extract usage % print(f"Disk Usage: {usage}") if int(usage.replace("%","")) > 80: print("⚠️ ALERT: Disk usage is above 80%!") Output: ubuntu@satheesha:~/python$ python3 import-subprocess.py Disk Usage: 3% 💡 Key Learnings: ✔️ subprocess helps automate Linux commands using Python ✔️ Useful for real-time monitoring and automation ✔️ Can be extended to trigger alerts, emails, or restart services 🚀 Real DevOps Use Cases: System health monitoring Auto-alerting when resources are high Integrating with cron jobs Automating routine checks Small automation like this can save a lot of manual effort in production environments. #Python #DevOps #Automation #Linux #Scripting #Cloud #Learning #Monitoring

Boosting Developer Productivity with the OpenAI Agents Python SDK: The OpenAI Agents Python SDK is a lightweight framework for building multi‑agent workflows in Python, with built‑in support for tools, guardrails, sessions, and tracing. It works with OpenAI models and 100+ other LLMs, so teams can stay flexible on model choice while standardizing their agent architecture. Why it matters for productivity? Instead of hand‑rolling custom logic for every AI feature, teams get ready‑made building blocks: - Agents with clear roles, tools, and safety policies. - Sessions that keep conversation and task state without extra wiring. - Tracing that makes debugging and optimization much faster. This turns “glue code” into reusable infrastructure and frees developers to focus on product logic rather than orchestration. Here is a concrete productivity example: Imagine a team maintaining a large Python codebase. With the SDK’s Sandbox Agents, you can spin up an agent in a local Unix sandbox, point it at your Git repo, and let it: - Inspect files (e.g., README, modules, test folders). - Propose changes, apply patches, and run commands in a controlled environment. - For example, you could ask a sandbox agent: “Scan this repository, identify duplicate utility functions, refactor them into a single module, and update imports.” The agent uses the sandbox to read the code, suggest edits, and apply them, turning what used to be a multi‑hour manual refactor into a guided, review‑able change set that takes minutes. Over time, these agent‑driven workflows compound into meaningful time savings across code reviews, documentation updates, and repetitive maintenance tasks. If you’re experimenting with agents or scaling AI features in production, the OpenAI Agents Python SDK is a practical way to turn LLMs into reliable, productive teammates. Please find the GitHub repository here: https://lnkd.in/gpgFvQGj #OpenAI #Agents #Python #MultiAgentSystems #AIEngineering #LLM #MLOps #AgenticAI #AIInfrastructure #SoftwareEngineering

Most people try to learn Python by watching tutorials. and forget everything in 24 hours. Here’s what actually works (and nobody talks about it): 👉 Handwritten Python notes. Sounds old school? Good. That’s why it works. When you write things down: • Your brain processes concepts deeply • You stop passive scrolling and start active learning • You build your own cheat sheet for revision I started making handwritten notes for: – Variables & Data Types – Loops & Conditions – Functions – OOP concepts And suddenly… Concepts started sticking. Not because Python is hard But because typing ≠ learning. If you're struggling with Python, try this for 7 days: ✍️ Write, don’t just watch 📌 Keep it simple (no fancy notes needed) 🔁 Revise daily for 10 minutes (𝗚𝗼𝗼𝗴𝗹𝗲) 𝗖𝗼𝘂𝗿𝘀𝗲𝘀 𝘆𝗼𝘂 𝘄𝗶𝗹𝗹 𝗿𝗲𝗴𝗿𝗲𝘁 𝗻𝗼𝘁 𝘁𝗮𝗸𝗶𝗻𝗴 𝗶𝗻 𝟮𝟬𝟮6. 🔃10,000+ courses from industry leaders : imp.i384100.net/dy9oR7 Here are Top Free Courses : 1. Introduction to Generative AI → imp.i384100.net/zzxe66 2. Foundations of Project Management → imp.i384100.net/qWW22j 3. Google Project Management → imp.i384100.net/MKmYNo 4. Python for Data Science, AI & Development → imp.i384100.net/4aPEQ9 5. Google Digital Marketing & E-commerce → imp.i384100.net/k40Pod 6. Google IT Support → imp.i384100.net/rEQ123 7. Google Data Analytics → imp.i384100.net/yZ2xmW 8. Machine Learning Specialization → imp.i384100.net/dyajBK 9. Deep Learning Specialization → imp.i384100.net/1GrOWa 10. Google Cybersecurity → mp.i384100.net/qW4G2L 11. Google UX Design → imp.i384100.net/Pzyq6Q 12. Web Applications for Everybody → imp.i384100.net/Pzr0zN 13. Microsoft Python Programming → imp.i384100.net/GbmVPr 14. Google Advanced Data Analytics → imp.i384100.net/vDPZn 15. IBM Full Stack Software Development → imp.i384100.net/B5zX5W 16. Introduction to Web Development → imp.i384100.net/0GxJGE 17. IBM Front-End Developer → imp.i384100.net/rEa2n3 18. IBM Back-End Development → imp.i384100.net/L0KZPM

Understanding Tuples and Sets in Python is a small step that makes a big difference in DevOps 🚀 Tuple = immutable (safe & fast) List = mutable (flexible) Set = no duplicates (optimized loops & clean data) In real DevOps workflows, using sets can reduce unnecessary iterations and improve script performance. If you're working with automation or CI/CD, these basics matter more than you think. Blog Link: https://lnkd.in/d8_md7WK , https://lnkd.in/dKm-mpnG #Python #DevOps #Automation #DataStructures #CICD

Day 33- 🐍 Understanding Python Data Structures: Array, List, Tuple, Set & Dictionary As I continue strengthening my Python fundamentals, I revisited one of the most important concepts — Data Structures. Choosing the right data structure makes your code more efficient, readable, and powerful. Let’s quickly break them down: 🔹 Array Used to store elements of the same data type. Efficient for numerical operations (commonly used with libraries like NumPy). 🔹 List • Ordered • Mutable (can be changed) • Allows duplicate values Example: my_list = [1, 2, 3, 4] 🔹 Tuple • Ordered • Immutable (cannot be changed) • Allows duplicates Example: my_tuple = (1, 2, 3) 🔹 Set • Unordered • No duplicate values • Mutable Example: my_set = {1, 2, 3} 🔹 Dictionary • Key–Value pairs • Ordered (Python 3.7+) • Mutable Example: my_dict = {"name": "DevOps", "level": "Beginner"} ⸻ 💡 When to Use What? ✔ Use List when you need flexibility ✔ Use Tuple when data should not change ✔ Use Set to remove duplicates ✔ Use Dictionary for structured key-value data ✔ Use Array for numeric-heavy operations Mastering these basics builds a strong foundation for advanced concepts like automation, DevOps scripting, and data handling.