Logistic Regression 101: Yes or No Decisions

My model scored 100% accuracy....Yey But I didn't celebrate. Something felt wrong. A model that perfect on real-world messy data isn't a success it's a warning sign. So I went looking. Turns out I had been importing a dataset I previously worked on. It was corrupted. The model had essentially memorized answers it had already seen. The score was meaningless. I restarted from a clean dataset. Ran everything again properly. Restart & Run All, no shortcuts. This time the numbers were honest: 83.8% cross-validation accuracy. 88.7% ROC-AUC. Less impressive on the surface. Far more valuable in reality. Here's what the actual model pipeline looks like: I tested several algorithms on the same feature set. Logistic regression outperformed the others on this problem, binary classification, structured tabular data, limited sample size. Simple models often win when the problem fits them. This one did. The pipeline: → StandardScaler for numerical features (Age, Fare, FamilySize) → One-hot encoding for passenger title (Miss, Mr, Mrs, Rare) → FamilySize engineered as a composite feature → LogisticRegression wrapped in a scikit-learn Pipeline → Serialised with joblib for API serving The title encoding decision matters more than it looks. My first instinct was label encoding assigning integers to each title. That was wrong. Label encoding implies an order: Mr=1, Mrs=2, Miss=3. There is no such order. One-hot encoding treats each title as an independent binary flag. That's the correct representation. Catching that distinction early saved the model from learning a relationship that doesn't exist. I also found a bug during integration testing. The FamilySize feature was off by one, the player themselves wasn't being counted in their own family. A small error, but in a model where every feature matters, small errors compound. I documented it as a known issue rather than quietly patching it with a guess. Known bugs you can explain are better than hidden bugs you can't. This is post 2 of a series documenting how I built an AI-driven simulation game powered by a real ML pipeline. Next post: the FastAPI backend how the model went from a notebook to a live prediction endpoint. #MachineLearning #Python #ScikitLearn #MLEngineering

🚀 Car Price prediction project using ML – Part 2: Model Building & Evaluation Continuing from yesterday’s update on Data Cleaning & ETL, today I focused on the next critical phase of the ML pipeline 👇 🔹 Data Splitting Divided the dataset into training and testing sets to ensure unbiased model evaluation. 🔹 Model Training Experimented with multiple algorithms: • Linear Regression • Random Forest Regressor • XGBoost Regressor 🔹 Hyperparameter Tuning Applied **GridSearchCV** to optimize model performance and find the best parameters. 🔹 Results & Insights After comparing all models, Linear Regression performed the best for this dataset—simple, interpretable, and effective. 💡 Key takeaway: Sometimes simpler models outperform complex ones depending on the data. Looking forward to taking this further with model evaluation metrics and deployment 🚀 #MachineLearning #DataScience #MLProjects #Python #AI #LearningInPublic

𝐎𝐮𝐭𝐥𝐢𝐞𝐫𝐬 -- 𝐎𝐧𝐞 𝐩𝐫𝐨𝐛𝐥𝐞𝐦 𝐈 𝐤𝐞𝐞𝐩 𝐟𝐚𝐜𝐢𝐧𝐠 𝐰𝐡𝐢𝐥𝐞 𝐛𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐦𝐨𝐝𝐞𝐥𝐬… While working on a recent dataset before model building, I ran into a common issue ---- outliers. We all know: "Outliers are unusual data points that behave very differently from the rest of the data." But what I realized practically is: Outliers are not always “bad”. 𝐖𝐡𝐞𝐫𝐞 𝐨𝐮𝐭𝐥𝐢𝐞𝐫𝐬 𝐜𝐫𝐞𝐚𝐭𝐞 𝐩𝐫𝐨𝐛𝐥𝐞𝐦𝐬 Some ML algorithms are sensitive to outliers: 1. Linear Regression 2. Logistic Regression 3. AdaBoost 4. Deep Learning models These models can get biased because a few extreme values pull the learning in the wrong direction. 𝐁𝐮𝐭 𝐬𝐨𝐦𝐞𝐭𝐢𝐦𝐞𝐬 𝐰𝐞 𝐍𝐄𝐄𝐃 𝐨𝐮𝐭𝐥𝐢𝐞𝐫𝐬 Example: Fraud Detection Fraud transactions = outliers Removing them = removing the actual problem So decision depends on business context, not just data. 𝐇𝐨𝐰 𝐈 𝐡𝐚𝐧𝐝𝐥𝐞𝐝 𝐨𝐮𝐭𝐥𝐢𝐞𝐫𝐬 𝐢𝐧 𝐦𝐲 𝐰𝐨𝐫𝐤𝐟𝐥𝐨𝐰 There are mainly two approaches: 1. Trimming (Removing Outliers) --> Completely removing extreme values 2. Capping (Winsorization) --> Limiting values to a threshold instead of removing Method depends on distribution 1. 𝐍𝐨𝐫𝐦𝐚𝐥 𝐃𝐢𝐬𝐭𝐫𝐢𝐛𝐮𝐭𝐢𝐨𝐧 --> 𝐙-𝐒𝐜𝐨𝐫𝐞 Rule: Mean ± 3 * Standard Deviation 2. 𝐒𝐤𝐞𝐰𝐞𝐝 𝐃𝐚𝐭𝐚 --> 𝐈𝐐𝐑 𝐌𝐞𝐭𝐡𝐨𝐝 Outliers are not just noise They can be signal depending on the problem #datascience #machinelearning #modelbuilding #outlier #python #Statistics #dataanalyst

Most ML models don’t fail because of bad algorithms. They fail because of bad data preparation. Feature engineering is the step most beginners skip or rush. But it’s often the difference between a model that works and one that actually performs. Here are 3 things I always check before training any model: 𝟭. 𝗠𝗶𝘀𝘀𝗶𝗻𝗴 𝗩𝗮𝗹𝘂𝗲𝘀 Missing data is not the end of the world. You can fill gaps using simple statistics like mean or median (univariate imputation), or go smarter with KNN imputation which looks at similar data points to estimate what’s missing. 𝟮. 𝗢𝘂𝘁𝗹𝗶𝗲𝗿𝘀 Outliers can silently wreck your model. I use the IQR method to catch them: anything below Q1 - (1.5×IQR) or above Q3 + (1.5×IQR) gets flagged. For normally distributed data, Z-scores do the job just as well. 𝟯. 𝗜𝗺𝗯𝗮𝗹𝗮𝗻𝗰𝗲𝗱 Data If your dataset has 95% of one class and 5% of another, your model will just learn to ignore the minority. Fix it by downsampling the majority class or upweighting the minority. Both work. Pick based on your data size. Get these three right and your model has a real shot. What part of feature engineering do you find most tricky? Drop it below 👇 #MachineLearning #DataScience #Python #MLEngineering #FeatureEngineering

📊 Bayesian Regression Dashboard – Capturing Predictions & Uncertainty Turning uncertainty into insight with Bayesian regression I recently built a Bayesian Regression Dashboard focused on one of the most valuable aspects of predictive modeling: understanding uncertainty, not just point estimates. Unlike traditional frequentist methods, Bayesian regression provides a full posterior distribution over predictions. This dashboard visualizes that power in three key ways: 1️⃣ Regression with 95% Credible Interval The top plot shows the observed data, the posterior mean of the regression fit, and a 95% credible interval (CI) around the predictions. 🔍 Interpretation: There is a 95% probability that the true regression function lies within this band — a much more intuitive statement than a confidence interval. 2️⃣ Predictions with Uncertainty at Specific X Points The middle panel highlights predictions (with credible intervals) at chosen X locations. 📌 Use case: Decision-makers can see not just what Y is predicted to be, but how confident we are in that prediction — critical for risk assessment. 3️⃣ Uncertainty vs. Confidence Level Trade-off The bottom chart shows the average width of the credible interval as a function of the confidence level (e.g., 50%, 80%, 95%, 99%). 📈 Insight: Wider intervals give higher confidence but less precision. This trade-off visualization helps select an appropriate confidence level for business or scientific decisions.                                                                         #BayesianInference #DataScience #MachineLearning #Python #PyMC #Statistics #UncertaintyQuantification #DataVisualization

This is the only machine learning algorithm you can explain to your grandmother. A decision tree makes predictions exactly the way humans make decisions. It asks a series of yes or no questions until it reaches an answer. Is the customer's monthly income above 50,000? 👉 Yes. Have they missed any payments in the last year?     👉 No. Approve the loan.     👉 Yes. Decline the loan. 👉 No. Decline the loan. Every split in the tree is a question. Every leaf at the bottom is a decision. Why data scientists love it. ✅ Completely transparent. You can see every decision the model made. ✅ Handles both numbers and categories without preprocessing ✅ Requires almost no data preparation ✅ Easy to visualise and explain to non-technical stakeholders The honest downside. 🚨 A single decision tree overfits easily. It memorises the training data instead of learning the pattern. This is exactly why Random Forest was invented. It builds hundreds of decision trees and combines their answers. More on that in the next post. Use a decision tree when you need a quick, explainable baseline before trying anything more complex. 📌 It will not always be your best model. But it will always help you understand your data better. #DataScience #MachineLearning #Python

Machine Learning/Artificial Intelligence  Day 27  Today I built my first logistic regression model. Took me so loooong🤧 but slow and steady……..  I created a simple dataset with cibil_score, loan_amount, and approval_status. I didn’t want to use a large or complex dataset at first so I could fully grasp how linear regression works.  Then I trained a model to predict loan approval.The code I ran:```pythonimport pandas as pdfrom sklearn.model_selection import train_test_splitfrom sklearn.linear_model import LogisticRegressionfrom sklearn.metrics import accuracy_scoredata = {    'cibil_score': [750, 680, 720, 800, 650, 700, 780, 620],    'loan_amount': [500000, 300000, 450000, 600000, 250000, 400000, 550000, 200000],    'approval_status': [1, 0, 1, 1, 0, 1, 1, 0]}df = pd.DataFrame(data)X = df[['cibil_score', 'loan_amount']]y = df['approval_status']X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)model = LogisticRegression()model.fit(X_train, y_train)y_pred = model.predict(X_test)accuracy = accuracy_score(y_test, y_pred)print(f"Accuracy: {accuracy}")```RESULTAccuracy: 1.0 My observations;1. The code ran without errors, though I made errors before this, especially typographical errors. 2. The model worked. 100% accuracy on a small fake dataset is not real life. Real data will never be that clean.My code is correct. Now I need to run it on a real dataset to see true performance. Next step:Take my real loan dataset and replace the fake data.Learning step by step, staying consistent every day.#M4ACE LearningChallenge#LearningInPublic#30DaysOfAIML#Python #LogisticRegression

Day 2: Mastering the "Engine" Behind Data Science The journey into Data Science 2.0 & Agentic AI continues! After setting the stage yesterday, Day 2 was all about getting under the hood to understand how Python actually talks to our hardware. If you want to build high-performance AI agents, you have to understand memory and environment management. Here’s the breakdown of today’s deep dive: 1. The Hardware-Software Handshake We explored the lifecycle of a variable. It’s not just code; it’s a physical reality in your RAM. The Chain: Hardware \rightarrow OS \rightarrow Python \rightarrow VS Code. Memory Mapping: When you define a = 12, Python isn't just "remembering" a number; it’s requesting a specific address in your RAM to store that value. RAM vs. Disk : We clarified why code execution happens in the RAM (8GB/16GB) while our scripts and installers sit on the HDD/SSD. 2. Environment Precision with UV Managing multiple Python versions is a nightmare without the right tools. We utilized UV to pin specific versions (like Python 3.12) to our projects. Notebooks vs. Scripts: Learned when to use .ipynb for rapid experimentation and when to transition to .py for production-ready scripts. 3. Data Types: The Building Blocks Data Science is only as good as the data you feed it. We broke down: Integers, Floats, and Strings: Understanding why 12 (int) is fundamentally different from 12.0 (float) in memory. Booleans: The binary foundation of "True/1" and "False/0" that drives all logic. 4. The "Action" Symbols (Operators) We categorized the tools that allow us to manipulate data: Arithmetic & Relational: For math and comparisons. Logical & Bitwise: The core of complex decision-making for AI agents. Today's Challenges: Type Casting Gauntlet: Testing every combination of data types to see what breaks and what works. Environment Mastery: Activating isolated environments to ensure project stability. The goal isn't just to write codeit's to understand the system so we can build smarter, faster, and more autonomous AI. #DataScience #GenAI #AgenticAI #Python #MachineLearning #ContinuousLearning #TechBootcamp Krish Naik Monal S.

📈 𝗙𝗿𝗼𝗺 𝗗𝗮𝘁𝗮 𝘁𝗼 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻: 𝗕𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝗠𝘆 𝗙𝗶𝗿𝘀𝘁 𝗟𝗶𝗻𝗲𝗮𝗿 𝗥𝗲𝗴𝗿𝗲𝘀𝘀𝗶𝗼𝗻 𝗠𝗼𝗱𝗲𝗹 𝗶𝗻 𝗣𝘆𝗧𝗼𝗿𝗰𝗵 Instead of seeing this as “just another model,” I approached this lecture like assembling a 𝗽𝗶𝗽𝗲𝗹𝗶𝗻𝗲 — where every step transforms raw data into a meaningful prediction using PyTorch. Here’s how the entire flow came together: ### 🧱 Phase 1 — Laying the Foundation Before any model: • 𝗗𝗮𝘁𝗮 𝗚𝗮𝘁𝗵𝗲𝗿𝗶𝗻𝗴 → Collect relevant data • 𝗗𝗮𝘁𝗮 𝗣𝗿𝗲𝗽𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴 → Clean, format, remove noise • 𝗙𝗲𝗮𝘁𝘂𝗿𝗲 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 → Create meaningful inputs (𝗲.𝗴., 𝗱𝗲𝗿𝗶𝘃𝗶𝗻𝗴 𝗮𝗴𝗲 𝗳𝗿𝗼𝗺 𝗗𝗢𝗕) 👉 Insight: 𝗕𝗲𝘁𝘁𝗲𝗿 𝗶𝗻𝗽𝘂𝘁 = 𝗯𝗲𝘁𝘁𝗲𝗿 𝗼𝘂𝘁𝗽𝘂𝘁 (no model can fix poor data) ### ⚙️ Phase 2 — Defining the Model At its core, Linear Regression is just: 👉 𝘆 = 𝘄𝘅 + 𝗯 Where: • `w` → weight (importance of input) • `b` → bias (adjustment factor) In PyTorch, these become 𝗹𝗲𝗮𝗿𝗻𝗮𝗯𝗹𝗲 𝗽𝗮𝗿𝗮𝗺𝗲𝘁𝗲𝗿𝘀. ### 🔄 Phase 3 — The Learning Loop Now the real process begins: #### 1️⃣ Forward Pass Input goes through the model → prediction generated #### 2️⃣ Loss Calculation Compare prediction with actual value 👉 Measure error (how wrong the model is) #### 3️⃣ Backpropagation Calculate gradients → understand how to reduce error #### 4️⃣ Optimization Step Update weights & bias → improve prediction ### 🔁 The Iteration Mindset ``` Predict → Measure Error → Adjust → Repeat ``` This loop continues until the model 𝗹𝗲𝗮𝗿𝗻𝘀 𝘁𝗵𝗲 𝗽𝗮𝘁𝘁𝗲𝗿𝗻 𝗶𝗻 𝗱𝗮𝘁𝗮. ### 🎯 What Makes This Powerful? Even though linear regression is simple: • It introduces the 𝗳𝘂𝗹𝗹 𝗠𝗟 𝗽𝗶𝗽𝗲𝗹𝗶𝗻𝗲 • It builds intuition for 𝗺𝗼𝗱𝗲𝗹 𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 • It sets the base for advanced models like CNNs, RNNs ### 💡 Key Realization This lecture wasn’t just about linear regression. It was about understanding: 👉 How data flows through a system 👉 How models learn from mistakes 👉 How iterative improvement actually works ### 🚀 Final Thought Every complex deep learning model starts with something this simple. Mastering this feels like unlocking the 𝗳𝗶𝗿𝘀𝘁 𝗿𝗲𝗮𝗹 𝗯𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝗯𝗹𝗼𝗰𝗸 𝗼𝗳 𝗔𝗜 𝘀𝘆𝘀𝘁𝗲𝗺𝘀. On to more advanced architectures next 🔥 #PyTorch #DeepLearning #MachineLearning #ArtificialIntelligence #LinearRegression #Python #LearningJourney

Pipelines in ML really change how you build models!! So I rebuilt my Customer Churn Prediction project — this time using a proper ML pipeline. 🔧 What I improved: • Built an end-to-end Pipeline using ColumnTransformer • Switched from train-test split to 5-Fold Cross Validation • Removed unnecessary feature selection (Chi-Square) • Handled class imbalance using F1-score & class_weight • Tuned models like Random Forest & XGBoost 📊 Key Results (F1 Score): • Logistic Regression → ~0.62 • Decision Tree → ~0.60 • Random Forest → ⭐ ~0.63 • XGBoost → ~0.56 💡 Key Learnings: • My earlier results were slightly optimistic due to a single train-test split • Cross-validation gave me more honest and stable performance • Random Forest performed best → indicating non-linear patterns • Logistic Regression performed almost as well → dataset isn’t highly complex • XGBoost underperformed → showing advanced models need proper tuning Check out both codes here:- https://lnkd.in/gV5Cb5iC This project helped me move from “just building models” to actually understanding how ML systems should be structured and evaluated in practice. Would love to hear your feedback or suggestions! #MachineLearning #DataScience #Python #ScikitLearn #XGBoost #Analytics #LearningJourney