Mastering Lambda Functions in Python

🔷A simple train test split is not always enough. I learned this the hard way when my model looked great on paper and struggled on real data. 📌Here is what nobody tells you about splitting data properly. The basic split gives you two sets. Training and testing. That works for simple projects. But what if you need to tune your model? You test different settings, pick the best one, and evaluate on the test set. The problem is that you have now indirectly used the test set to make decisions. It is no longer a fair judge. This is where a three way split becomes important. 🔹X_train, X_temp, y_train, y_temp = train_test_split( X, y, test_size=0.3, random_state=42 ) 🔹X_val, X_test, y_val, y_test = train_test_split( X_temp, y_temp, test_size=0.5, random_state=42 ) Now you have three sets. Training set. The model learns here. 70 percent of your data. Validation set. You tune and compare models here. 15 percent. Test set. You evaluate the final model here. Once. Never again. 15 percent. The test set is sacred. You look at it exactly one time at the very end. One more thing that most people miss. Always stratify your split when your target column is imbalanced. 🔹train_test_split(X, y, stratify=y, test_size=0.2) stratify=y makes sure both sets have the same proportion of each class. Without it you might end up with a training set that barely sees the minority class and a model that has no idea it exists. The split is not a formality. It is a decision that shapes every result that follows. Get it right before you touch anything else. ❓What split ratio do you use for your projects and why? #DataScience #MachineLearning #Python

Mistakes are part of the process Day 7 – #100DaysOfCode ⏰ Time Spent: 2 hours ⚒️ What I Did: * Yesterday I have learned one way to read scatter plots , today I Practiced that . * Modified my function to make it reusable * Plotted relationships between complaints and aggregated features I observed only these two trends: * log(x) vs y → logarithmic trend [ y = a · log(x) + b ] * log(x) vs log(y) → power law [ y = k · xᵃ ] But then I realized something important… I was plotting a sum on the x-axis, which naturally increases the values which created misleading patterns. So I switched to mean,but the trends disappeared. Which implies no relation but I'll experiment with few other transformations before I conclude that --- 🚪 Links: * Repo: [https://lnkd.in/g7zsMygp) --- 🧠 Learning: Bad feature choice can create fake patterns. 📌 Closing: Should try to work on these things when I am not tired ( Mornings / After a nap ) #DataScience #DataAnalytics #Python #CodingJourney

Shallow Copy vs Deep Copy — The 2 AM Bug Trap 🛑 Most developers think they understand copying objects, until their original data mysteriously changes. That’s not a bug, that’s memory behavior biting you. → Shallow Copy Creates a new container, but nested objects are still shared (by reference) 👉 Change nested data → both copies change. Best for: Flat, simple data. → Deep Copy Creates a completely independent clone, everything is copied recursively. 👉 Change anything → original stays untouched Best for: Complex, nested structures. 💡 Rule of Thumb Shallow → when you only need a surface-level copy Deep → when you need true isolation ⚠️ The real trap: Most bugs aren’t syntax errors. They come from not understanding how data behaves in memory. If you’ve ever spent hours debugging only to realize it was a shallow copy issue. Welcome to the club 😄 #Python #Python3 #Programming #SoftwareEngineering #CleanCode #Debugging #TechTips #PythonDeveloper  #BackendDevelopment

🚀 Day 7/100: Leveling Up with Advanced Pandas! 🐼 🚀 One full week into my #100DaysOfML challenge! Today, I transitioned from just exploring data to actively modifying and manipulating it. I covered some advanced Pandas techniques that are absolute game-changers for feature engineering and data cleaning. Here is a breakdown of today’s learning: ➕ Adding Columns: Learned how to inject new data using both the direct assignment method and the precise .insert() method for specific column placement. 🎯 Targeted Updates: Explored how to update specific, isolated values within a DataFrame without messing up the rest of the data. ⚡ Bulk Column Operations: This is where Pandas shines! I learned how to update an entire column at once (like applying a 5% increase across an entire 'Salary' column in one line of code!). 🧹 Refining Data: Reinforced my concepts on handling missing values and safely removing/dropping unnecessary columns. Theory is great, but muscle memory is better. Tomorrow is going to be 100% dedicated to hands-on practice with these methods in VS Code! 💻✨ #100DaysOfML #MachineLearning #Pandas #DataScience #DataManipulation #Python #100DaysOfCode #TechJourney #Day7

I had a funny moment while coding today. 😁 Everything was going great. Data was ready. Then I typed the usual code: X = df.drop("price", axis=1) y = df["price"] And my brain just stopped: "Wait... why do we always use X and y? Who made this a rule?" 🤨😭 I looked it up, and it is actually just simple math 📐: Capital X = A big group of data (many columns). Small y = Just one thing (one column). Big data gets a big letter. Small data gets a small letter. 🤯 Do I have to use them? No. I could use normal names like features and price. But am I going to do that? Nope! Tomorrow I will use X and y again. It is just a habit now! 🌚 It is funny how in ML, the biggest questions come from the smallest things. 😅 Be honest: Do you use normal names, or do you also just use X and y? 👇 #MachineLearning #Python #DataScience #CodingLife #SimpleCode

I wrote this piece because I was tired of seeing data scientists (myself included) waste the first two hours of a project writing the same boilerplate code. We've all been there: df.head(), df.isnull().sum(), squinting at correlation heatmaps, and writing yet another snippet to check distributions. It's plumbing, not science. ydata-profiling changed my workflow completely, and I wanted to share exactly how I use it and, just as importantly, when I don't use it. If you're in the Python/data science world and haven't given this library a spin yet, I hope this gives you back some of your mental bandwidth. Let me know what your go-to EDA tool is in the comments! #DataScience #Python #EDA #MachineLearning #ydata #DataAnalytics #OpenSource #Productivity #TechWriting #DataQuality

🚀 Machine Learning Algorithms – Hands-on Implementation (GitHub Project) I’ve built and documented multiple core Machine Learning algorithms from scratch using Python and Jupyter Notebooks. This project focuses on understanding concepts deeply rather than just using libraries blindly. 🔍 What’s included: Decision Tree (ID3 Algorithm) K-Nearest Neighbors (KNN) Logistic Regression Simple Linear Regression (Salary Prediction) Confusion Matrix Correlation Matrix Normal Distribution Data Manipulation using DataFrames Decision Tree Pruning 💡 What I focused on: Clear understanding of algorithms Step-by-step implementation Real dataset usage Practical visualization and analysis 📂 This repository reflects my journey of moving from theory → practical execution in Machine Learning. 🔗 Check out the project here:https://lnkd.in/dkvv8Jwd

My model hit 89% accuracy. I was proud of it. Then I tested it on different data. It dropped to 71%. Just like that. Same model. Same code. Totally different result. I had no explanation. The problem wasn't the model. It was how I was testing it. I was splitting my data once, 80% train, 20% test, trusting whatever number came out. My model wasn't learning real patterns. It was memorising that one specific slice of data. Cross-validation changed how I think about this completely. Instead of trusting one number, you get five. But here's what nobody told me early on: The standard deviation matters more than the mean. Mean: 0.87 │ Std: 0.02 → Stable. Trust it Mean: 0.87 │ Std: 0.12 → Fragile. Dig deeper Both look identical on a single split. Cross-validation exposes the truth. A single accuracy number isn't a result. It's a guess. I now run this before trusting any model, because a model that only works on the data you showed it isn't a model. It's just an expensive lookup table. Have you ever confidently presented a model that later turned out to be wrong? 👇 #MachineLearning #Python #DataScience #CrossValidation #LearningInPublic

👉 We use data types every day…but rarely ask why they’re even needed. 💡 At first, they look simple: numbers, strings, booleans…Just categories, right? But imagine programming without them. What would this mean? # Numbers → calculation print(10 + 5) # 15 # Strings → concatenation print("10" + "5") # "105" Looks fine so far… But now this 👇 print(10 + "5") # ❌ TypeError Should it become 15? Or "105"? There’s no clear answer. And that’s the problem. Data types are not just technical labels… They define meaning. They tell the computer: “This is a number — you can calculate it.” “This is text — you can combine it.” “This is a boolean — you can decide with it.” Without these rules… Programming wouldn’t just be difficult — it would be ambiguous. 💡 And here’s the bigger idea: When we don’t define what something is…we create confusion. Clarity isn’t extra. It’s the foundation. Are you learning concepts deeply… or just accepting them as they are? #Python #LearnPython #ProgrammingConcepts #DataTypes #CodingForBeginners #ComputerScience #TechEducation #LearnWithMe #CodingJourney #CSStudents

Pandas is about to get replaced. Not tomorrow. But in 2 years, half of you will have switched to Polars. And the other half will be wondering why their scripts are still slow. Polars is: → 5-30x faster than Pandas (on real benchmarks) → Memory-efficient (no more OOM errors on 10GB datasets) → Written in Rust (lazy evaluation, query optimization built in) → Has a cleaner, more consistent API than Pandas → Native support for streaming data (no chunking required) My free notebook walks through the fundamentals: → Polars DataFrames — creation, inspection, indexing → The expressions API (the thing that makes Polars fast) → Filtering, selecting, sorting — the Pandas equivalents → group_by with expressions (way cleaner than agg) → Lazy evaluation — query optimizer explained → Side-by-side Pandas vs Polars benchmarks If you've never heard of Polars, you're about to. Get ahead of the curve. https://lnkd.in/gDXKkV75 Day 2/7. #Polars #Python #DataEngineering #DataAnalytics #Pandas #Rust #DataFrames #OpenSource