Adeel Rehman’s Post

🚀 Day 9/10 — Optimization Series Config-Driven Pipelines (Avoid Hardcoding) 👉 Basics are done. 👉 Now we move from working code → optimized code. You build a pipeline… It works perfectly… But you hardcode everything 😐 file_path = "data/sales_2024.csv" api_url = "https://lnkd.in/gsfHEDWP" 👉 Looks simple… but becomes a problem later. 🔹 The Problem Hard to update values ❌ Not reusable ❌ Breaks across environments ❌ 🔹 What is Config-Driven Approach? 👉 Move all dynamic values to a config file 🔹 Example (config.json) { "file_path": "data/sales_2024.csv", "api_url": "https://lnkd.in/gsfHEDWP" } 🔹 Use in Python import json with open("config.json") as f: config = json.load(f) file_path = config["file_path"] api_url = config["api_url"] 🔹 Why This Matters Easy to update 🔄 Reusable pipelines ♻️ Environment-friendly 🌍 🔹 Real-World Use 👉 Dev / Test / Prod configs 👉 Data pipelines 👉 API integrations 💡 Quick Summary Config-driven = flexible + scalable pipelines 💡 Something to remember If your values change often… they don’t belong in your code. #Python #DataEngineering #LearningInPublic #TechLearning

New blog post. You've finished developing an ML model with {tidymodels}, and you're ready to automate it in Dagster. You hand things off to data engineering. Their reply: "Sorry, we need this rewritten in Python to deploy." But the model pipeline code is solid. It's wrapped in an R package; there's good test coverage, a {pkgdown} website documenting everything, the works. It's just written in R. Do we really need to do all of that work all over again? Not anymore. I built the R package {dagsterpipes} to solve this problem. It implements Dagster's Pipes Protocol for the R language, allowing you to run R code inside of Dagster without losing its logging and observability features. Walkthrough with a working example in the post: https://lnkd.in/gfxjadQy #rstats

🧠 Stop Sending Manual Emails — Let Python Do It Every Friday, I used to manually email a report to multiple stakeholders. Open Outlook → attach file → write subject → send. Repeat. Every week. Now Python does it. </> Python import smtplib from email.mime.multipart import MIMEMultipart from email.mime.base import MIMEBase from email import encoders msg = MIMEMultipart() msg['Subject'] = 'Weekly Claims Summary — Auto Report' msg['From'] = 'you@company.com' msg['To'] = 'team@company.com' with open('weekly_summary.xlsx', 'rb') as f:   part = MIMEBase('application', 'octet-stream')   part.set_payload(f.read())   encoders.encode_base64(part)   msg.attach(part) with smtplib.SMTP_SSL('smtp.gmail.com', 465) as server:   server.login('you@company.com', 'app_password')   server.send_message(msg) </> Python 💡 What changed 👉 no manual emails 👉 no missed reports 👉 consistent communication 👉 runs automatically in the background Combine this with a scheduled script → and the report sends itself before you even log in. Automation isn’t just about analysis. ✅ It’s about removing repetitive steps around it. 👉 What’s one manual step in your workflow you’d automate if you could? #ActuaryWhoCodes #PythonForActuaries #Automation #Productivity #DataAnalytics #Analytics

🚀 Day 3/20 — Python for Data Engineering Functions: Writing Reusable & Clean Code As we start working with more data, writing the same logic again and again becomes messy. 👉 That’s where functions come in. 🔹 What is a Function? A function is a reusable block of code that performs a specific task. 🔹 Why Functions Matter in Data Engineering In real-world scenarios, we: clean data transform data apply repeated logic 👉 Instead of rewriting code, we reuse functions 🔹 Simple Example def clean_name(name): return name.strip().title() print(clean_name(" alice ")) 👉 Output: Alice 🔹 With Multiple Inputs def calculate_total(price, tax): return price + (price * tax) print(calculate_total(100, 0.1)) 👉 Output: 110.0 🔹 Where You’ll Use Functions Data cleaning Data transformation Reusable pipeline steps Automation scripts 💡 Quick Summary Functions help you: write cleaner code avoid repetition build reusable logic 💡 Something to remember Good code is not just working code. It’s reusable and maintainable. #Python #DataEngineering #DataAnalytics #LearningInPublic #TechLearning #Databricks

Recursion used to confuse me a lot… until I started thinking of it like a corporate manager delegating work. 😅 🌳 Maximum Depth of Binary Tree (LeetCode 104 — Easy | Blind 75) Whenever I saw a Binary Tree problem, I used to panic. Keeping track of depth while moving through the tree felt overwhelming. The question was always: 👉 How do I keep track of everything at once? I realized something simple but powerful: 👉 I don’t need to track everything myself — I can ask my subtrees Think of each node as a Manager. The manager asks the left subtree: “What’s your max depth?” Then asks the right subtree: “What’s your max depth?” Takes the maximum of both and adds 1 (for itself) That’s exactly what this does: 👉 1 + max(left_depth, right_depth) 🔑 Key Learnings: ✔️ Base Case is your foundation If there’s no node, return 0 → Like an employee saying: “No team under me.” ✔️ Trust Recursion Don’t try to track every path manually → Assume subproblems are solved correctly ✔️ Recognize the Pattern This is a classic Depth-First Search (DFS) approach (bottom-up) ⚙️ Complexity Time Complexity: O(N) — visit each node once Space Complexity: O(H) — recursion stack (tree height) Trees used to be my biggest fear… Now they’re becoming one of my favorite topics Curious to hear from you 👇 What’s a data structure or concept that finally “clicked” for you recently? #LeetCode #BinaryTrees #Blind75 #DataStructures #Recursion #DFS #ProblemSolving #CodingJourney #Python #TechCareers

𝗪𝗵𝘆 𝗘𝘃𝗲𝗿𝘆 𝗣𝘆𝘁𝗵𝗼𝗻 𝗗𝗲𝘃𝗲𝗹𝗼𝗽𝗲𝗿 𝗡𝗲𝗲𝗱𝘀 𝘁𝗼 𝗠𝗮𝘀𝘁𝗲𝗿 𝗣𝗮𝗻𝗱𝗮𝘀   Raw Python loops on tabular data are slow, unreadable, and honestly just painful to maintain. The moment your dataset grows beyond a few hundred rows, you feel it — both in runtime and in your code quality.   𝗣𝗮𝗻𝗱𝗮𝘀 solves this. It gives you a complete, expressive toolkit for data manipulation, cleaning, reshaping, and analysis — all built on top of NumPy with deep integration into the entire Python data science ecosystem. Here are 3 things that make Pandas genuinely powerful: - 𝗩𝗲𝗰𝘁𝗼𝗿𝗶𝘇𝗲𝗱 𝗢𝗽𝗲𝗿𝗮𝘁𝗶𝗼𝗻𝘀 — instead of writing loops, you perform arithmetic and logic across entire columns at once. df['A'] + df['B'] beats manual iteration every single time — faster execution, cleaner code. - 𝗙𝗹𝗲𝘅𝗶𝗯𝗹𝗲 𝗗𝗮𝘁𝗮 𝗖𝗹𝗲𝗮𝗻𝗶𝗻𝗴 — .isna(), .fillna(), .dropna(), .drop_duplicates(), and .astype() handle all the messy real-world data problems without custom functions or boilerplate code. - 𝗣𝗼𝘄𝗲𝗿𝗳𝘂𝗹 𝗚𝗿𝗼𝘂𝗽𝗶𝗻𝗴 & 𝗠𝗲𝗿𝗴𝗶𝗻𝗴 — .groupby() lets you split, apply, and combine data in one line. pd.merge() brings SQL-style joins directly into your Python workflow. Conclusion:-   Pandas is not just a library — it is the foundation of practical data work in Python. Once you move from raw loops to vectorized operations, method chaining, and expressive querying, you stop wrestling with your data and start actually understanding it. If you are serious about Python, Pandas is non-negotiable. Special thanks to my mentor Mian Ahmad Basit for the continued guidance. #MuhammadAbdullahWaseem #Nexskill #Pandas #PythonDeveloper #Ceasefire #IslamabadTalks

I built a complete 𝗨𝘀𝗲𝗱 𝗖𝗮𝗿 𝗣𝗿𝗶𝗰𝗲 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗼𝗿 from scratch, creating a full end-to-end pipeline that handles everything from raw data to a live application. Instead of relying on a pre-built dataset, I identified a unique problem and built my own data source using web scraping. My goal was to move beyond tutorials and mimic a real-world 𝗱𝗮𝘁𝗮 𝘀𝗰𝗶𝗲𝗻𝗰𝗲 workflow.  • 𝗦𝗰𝗿𝗮𝗽𝗶𝗻𝗴: Automated data collection to get real-time market prices.  • 𝗣𝗿𝗲𝗽𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴: Cleaning messy web data into a machine-learning-ready format.  • 𝗠𝗼𝗱𝗲𝗹𝗶𝗻𝗴: Training a robust regressor to find the patterns.  • 𝗗𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁: Building a Flask web app to make the model accessible to anyone. The Workflow: 𝗦𝗰𝗿𝗮𝗽𝗲 𝗗𝗮𝘁𝗮 → 𝗖𝗹𝗲𝗮𝗻 & 𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺 → 𝗧𝗿𝗮𝗶𝗻 𝗠𝗼𝗱𝗲l → 𝗗𝗲𝗽𝗹𝗼𝘆 #MachineLearning #DataScience #Python #Flask #WebScraping #PortfolioProject Check out the full documentation and code on GitHub: https://lnkd.in/gAZp4iKq

🚀 Day 5/20 — Python for Data Engineering Error Handling (try / except) When working with real-world data, things don’t always go as expected. 👉 Files may be missing 👉 Data may be corrupted 👉 APIs may fail If your code crashes every time something goes wrong, that’s not data engineering. 🔹 What is Error Handling? Error handling allows your program to: 👉 handle unexpected situations 👉 continue running without crashing 🔹 Basic Syntax try: # code that might fail except: # code to handle error 🔹 Example try: df = pd.read_csv("data.csv") print(df.head()) except: print("File not found") 👉 If the file is missing, your program won’t crash 🔹 Handling Specific Errors (Better Practice) try: value = int("abc") except ValueError: print("Invalid number") 👉 More precise and professional 🔹 Why This Matters in Data Engineering Prevent pipeline failures Handle bad data gracefully Improve reliability Build production-ready systems 💡 Quick Summary Error handling makes your code: safer more stable production-ready 💡 Something to remember Good engineers don’t just write code that works… They write code that doesn’t break. #Python #DataEngineering #DataAnalytics #LearningInPublic #TechLearning #Databricks

🚀 Introducing ALGO_TRACKER.AI – Bridging Machine Learning with Static Code Analysis for Python. As software systems scale, quantifying Technical Debt and maintainability becomes crucial. Traditional rules-based linters often miss the complex interplay of metrics that define genuine code risk. To address this, I built ALGO_TRACKER.AI, an intelligent auditor that moves beyond rigid rules. It leverages a trained XGBoost model to analyze static code metrics (LOC, Cyclomatic Complexity, Halstead Metrics) recursively fetched from any public Python repository via the GitHub API. The goal is simple: Provide developers and tech leads with a predictive, probability-based "Bullish" (Clean/Maintainable) or "Bearish" (High Technical Debt) rating for their codebase. Key Features: 🔹 Deep recursive scanning of Python (.py) files using GitHub’s /git/trees API. 🔹 Static Metric Extraction (Radon/Lizard) to quantify complexity. 🔹 Intelligent Risk Prediction using an optimized XGBoost classifier. Tech Stack (High Performance & Scalable): ⚛️ Frontend: React, Tailwind CSS (Deployed on Netlify) ⚡ Backend: FastAPI (Python), (Deployed on Railway) 🤖 Machine Learning: Scikit-learn & XGBoost Check out the working prototype here: https://lnkd.in/g2tVERcH #MachineLearning #SoftwareEngineering #Python #FastAPI #ReactJS #FullStack #ArtificialIntelligence #Innovation

🚀 𝐏𝐲𝐭𝐡𝐨𝐧 𝐑𝐨𝐚𝐝𝐦𝐚𝐩 𝟐𝟎𝟐𝟔 — 𝐅𝐫𝐨𝐦 𝐁𝐞𝐠𝐢𝐧𝐧𝐞𝐫 𝐭𝐨 𝐏𝐫𝐨 🐍 Most people start Python… But very few follow a structured roadmap 😶 If you want to become a Data Engineer / Data Scientist / Developer, follow this 👇 📌 Step-by-Step Python Roadmap: 🔹 Basics → Syntax, Variables, Data Types, Functions 🔹 Advanced → List Comprehensions, Generators, Decorators 🔹 DSA → Arrays, Trees, Recursion, Sorting 🔹 OOP → Classes, Inheritance, Methods 📊 Specialize Based on Your Goal: 📈 Data Science → NumPy, Pandas, Matplotlib, Scikit-learn 🌐 Web Development → Django, Flask, FastAPI ⚙️ Automation → Web Scraping, File Handling, Scripts 🧪 Testing → Pytest, Unit Testing, TDD 💡 Pro Tip: Don’t just learn — build projects at every stage. That’s what makes your profile stand out. 🔥 Why Python? ✔ Beginner-friendly ✔ High demand in 2026 ✔ Used in Data, AI, Web, Automation 📌 Save this roadmap 🔁 Share with your network #Python #PythonRoadmap #LearnPython #DataEngineering #DataScience #MachineLearning #WebDevelopment #Automation #Coding #Programming #TechCareers #CareerGrowth #2026Goals