Feature Engineering for Machine Learning Performance

SVM is one of those algorithms that sounds intimidating until you see what it's actually doing. It just draws the best possible boundary between classes. That's it. Part 7 of my ML from scratch series Support Vector Machines on the sklearn digits dataset. Quick explanation first. SVM tries to find the line, or plane, or curve that separates your classes with the widest possible margin. The wider the gap between classes, the more confident the model is when something new comes in. For digits the setup is: Input: 8x8 pixel images of handwritten digits, flattened into 64 features each Output: which digit it is, 0 through 9 Also covered the three parameters that actually matter when tuning an SVM: C controls how strict the boundary is. Low C lets the model ignore some points for a smoother boundary. High C tries to classify every single point correctly, even at the cost of overfitting. Gamma controls how much influence a single training point has. Low gamma means each point affects a wide area. High gamma means each point only matters very locally. Kernel decides the shape of the boundary. Linear for straight cuts, RBF for curved ones, and a few others for specific cases. Changing any of these moves the accuracy in ways that aren't always obvious. That's the whole point of going through it on real data instead of just reading about it. Also cleaned up the repo and numbered all the folders so the series is easier to follow in order. https://lnkd.in/dC5Pzygv #MachineLearning #Python #DataScience #SVM #100DaysOfCode

A simple problem maybe more complex than expected. Still not getting it right after more than 20 iterations, I mean with realistic and random knob/tab shape. Tried a few LLMs. Question: given a photo folder create Python code to generate a jigsaw puzzle like patchwork from these pictures. Here is my latest test, feeding outputs between tests. First iterations were about basic knobs/tabs shapes creation. Getting realistic knob/tabs shapes took more work. Mismatch between boundary shapes and pictures orientation not respected. The latest generated code now has three passes. You may to try on your side. #ArtificialIntelligence #MachineLearning #LargeLanguageModels #LLM #PythonDevelopment #SoftwareEngineering #ComputerVision #ImageProcessing #GenerativeAI #AIEngineering #PromptEngineering #AlgorithmDesign #DataScience #DeepLearning #TechInnovation #ProblemSolving #SoftwareDevelopment #CodingLife #Debugging #EngineeringChallenges #RAndD #AppliedAI #AIProjects #OpenSourceAI

Most teams treat randomness like magic. Then they’re surprised when models behave like lottery tickets in production. Controlling randomness is not academic — it’s reliability engineering. Tiny checklist that saves you weeks: - Pin your RNG across layers: Python, NumPy, PyTorch/TensorFlow, CUDA. - Bake seeds into configs (not code). Change seed => full experiment trace. - Snapshot the environment: deps, CUDA driver, OS. Reproduce locally and in CI. - Log the seed with every run and tie it to artifacts (model, dataset version). - Test determinism: run the same seed 5–10x in CI. Fail fast on divergence. - Use deterministic ops only where latency and throughput allow; document the trade-offs. Tools & repos that actually help: - Hydra — manage experiment configs (include seeds consistently) - DVC — dataset + pipeline versioning so seeds map to dataset snapshots - MLflow — track runs and attach the seed as a searchable parameter - pytorch-lightning/pytorch-lightning — has seed_everything utilities to standardize seeding Quick config snippet idea: - config.yaml: seed: 12345 - bootstrap script: set all RNGs from config.seed, save that seed to run metadata Operational tip: Don’t just set one seed. Use a seed hierarchy: global -> component -> data-loader. It makes partial replay easier. At FlazeTech we once traced a flaky production endpoint to a missing seed in a custom C++ sampler. Fixing that single line cut customer errors by 70%. Determinism costs time. But unpredictability costs customers. What small seeding rule will you add to your next experiment or CI pipeline? #MLops #MachineLearning #Reproducibility #AIEngineering #DevTools #PyTorch #Hydra #DVC

RAG Day 4: Vector Databases and Indexing Excited to share my latest project from Day 4 of my RAG Learning series: Building a Hybrid Search Engine! 🚀 This hands-on mini-project compares semantic-only, keyword-only (BM25), and hybrid retrieval methods using vector databases like FAISS-inspired indices. It incorporates metadata filtering, reciprocal rank fusion, and efficient indexing techniques to handle document search at scale. Key takeaways: Vector databases are crucial for storing and querying embeddings efficiently, balancing speed, accuracy, and memory. Perfect for prototyping RAG systems! #RAG #VectorDatabases #MachineLearning #Python #AI #SearchEngine Source code: https://lnkd.in/gZwinm3i

🚀 Day 45 of My Learning Journey – NumPy Shape & Reshape Today, I explored how to work with array dimensions using NumPy, focusing on shape and reshape. 🔹 Key Learnings: ✔️ shape Helps to identify the dimensions of an array Example: (3, 2) → 3 rows and 2 columns ✔️ Modifying shape We can directly change the structure of an array Useful when reorganizing data ✔️ reshape() Creates a new array with a different shape Does NOT modify the original array Very helpful in data preprocessing 🔹 Hands-on Task Completed: Converted a list of 9 elements into a 3×3 matrix using NumPy. 💡 Takeaway: Understanding how to manipulate array dimensions is essential for data analysis, machine learning, and efficient problem-solving. 📌 Every small concept builds a stronger foundation! #Day45 #Python #NumPy #LearningJourney #DataScience #Coding #StudentLife

🚀 Machine Learning Exercise: Improving Model Performance For this exercise, I evaluated a classification model using a Random Forest approach, focusing on precision, recall, and F1 score rather than just accuracy. While accuracy gives an overall measure of correctness, it doesn’t always reflect the types of errors within the dataset. Before modeling, tools like pivot tables can be useful for exploring patterns in the data. I then reviewed feature importance and selected the most influential variables to build a refined model using a reduced feature set (cols3). 📊 Results: Accuracy: 86.22% Precision: 85.09% Recall: 78.29% F1 Score: 81.55% This project reinforced the importance of feature selection and evaluating multiple performance metrics when building a model. #MachineLearning #DataAnalytics #Python #DataScience #FeatureEngineering #PredictiveModeling #LearningJourney

✅ Day 97 of 100 Days LeetCode Challenge Problem: 🔹 #1281 – Subtract the Product and Sum of Digits of an Integer 🔗 https://lnkd.in/gxTAZc6U Learning Journey: 🔹 Today’s problem involved extracting digits of a number and performing two operations simultaneously. 🔹 I initialized two variables: one for product (pr) and one for sum (sm). 🔹 Using a while loop, I extracted each digit using n % 10. 🔹 Updated the product by multiplying the digit and updated the sum by adding it. 🔹 Reduced the number using integer division (n //= 10) after each step. 🔹 Finally returned the difference between product and sum. Concepts Used: 🔹 Digit Extraction 🔹 While Loop 🔹 Arithmetic Operations 🔹 Number Manipulation Key Insight: 🔹 Both product and sum can be computed in a single traversal of digits. 🔹 Efficient use of modulus and division avoids converting the number to a string. Complexity: 🔹 Time: O(d) 🔹 Space: O(1) #LeetCode #Algorithms #DataStructures #CodingInterview #100DaysOfCode #Python #ProblemSolving #LearningInPublic #TechCareers

Recently, I worked on a small machine learning project on Fitness Class Attendance Prediction. The goal was to predict whether a member would attend a class or not, using a complete workflow from raw data to final model evaluation. The project included: cleaning inconsistent data formats handling missing values encoding categorical variables preparing preprocessing pipelines training and comparing multiple models I tested: KNN, Decision Tree, SVM, and Naive Bayes What I found interesting was that the “best” model depended on how performance was judged: Naive Bayes gave the best F1-score on the main split SVM gave the highest accuracy Decision Tree looked like the most stable option when the test size changed A good reminder that model selection should not depend on one metric only. Github Repo: https://lnkd.in/d8_ADgY5 Projects like this keep showing me how important it is to combine clean data, correct preprocessing, and thoughtful evaluation to reach a solid conclusion. #MachineLearning #DataAnalytics #Python #ScikitLearn #ClassificationModels #DataScienceProjects

Built a complete PCA + ML pipeline on a student performance dataset (395 rows, 33 features). After cleaning, standardizing numeric variables, and encoding categorical fields, I explored relationships with correlation and study-habit vs grade visualizations. Then I implemented PCA end-to-end (covariance matrix → eigenvalues/eigenvectors, scree plots, biplots, and transformation dashboards) to understand variance and reduce dimensionality. Finally, I trained an SVM classifier on the top 5 principal components to predict Pass vs Fail, comparing kernels—best result: Linear SVM, 94.94% test accuracy. #Python #PCA #MachineLearning #SVM #DataScience #scikitlearn #AICadmey

Ridge Regression is like adding a speed limiter to your model: * No limit → it goes fast, but risks crashing (overfitting) * Too strict → it barely moves (underfitting) * Just right → smooth, stable, reliable The hyperparameter Alpha is the secret sauce. A small tweak in this parameter can completely change how your model behaves. In this post, I break it down with: ✔ Simple intuition (no heavy math) ✔ A simple Python example ✔ Visual comparison of different alpha values 👉 Read it here: https://lnkd.in/eqyYMMBC #DataScience #MachineLearning #AI #Python #Analytics