Python Data Analytics Operators and Conditional Statements

🔷A simple train test split is not always enough. I learned this the hard way when my model looked great on paper and struggled on real data. 📌Here is what nobody tells you about splitting data properly. The basic split gives you two sets. Training and testing. That works for simple projects. But what if you need to tune your model? You test different settings, pick the best one, and evaluate on the test set. The problem is that you have now indirectly used the test set to make decisions. It is no longer a fair judge. This is where a three way split becomes important. 🔹X_train, X_temp, y_train, y_temp = train_test_split( X, y, test_size=0.3, random_state=42 ) 🔹X_val, X_test, y_val, y_test = train_test_split( X_temp, y_temp, test_size=0.5, random_state=42 ) Now you have three sets. Training set. The model learns here. 70 percent of your data. Validation set. You tune and compare models here. 15 percent. Test set. You evaluate the final model here. Once. Never again. 15 percent. The test set is sacred. You look at it exactly one time at the very end. One more thing that most people miss. Always stratify your split when your target column is imbalanced. 🔹train_test_split(X, y, stratify=y, test_size=0.2) stratify=y makes sure both sets have the same proportion of each class. Without it you might end up with a training set that barely sees the minority class and a model that has no idea it exists. The split is not a formality. It is a decision that shapes every result that follows. Get it right before you touch anything else. ❓What split ratio do you use for your projects and why? #DataScience #MachineLearning #Python

🚀#Day11 of #Learning Today I continued working with Pandas and explored more data cleaning and transformation techniques. 🔹 reset_index() – Learned how to reset and reorganize index structures. 🔹 rename() – Practiced renaming columns for better readability. 🔹 unique() & nunique() – Explored identifying and counting unique values in data. 🔹 isnull(), notnull(), dropna() – Worked with detecting and handling missing values. 🔹 groupby() – Learned how to group data for analysis. 🔹 apply() – Used custom functions across data for more flexible transformations. Today’s learning helped me strengthen skills in cleaning, grouping, and transforming data efficiently Github Repo : https://lnkd.in/g34K2vy3 #Python #Pandas #MachineLearning #LearningJourney

Day 6/10 🚀 This is where your data starts to take shape. Collections — the backbone of every Python program. Without the right one? Slower code, messy logic. With the right one? Faster lookups, cleaner design. 📋 What I covered today: 01 → Lists — slicing & comprehensions 02 → Tuples — immutability & unpacking 03 → Dictionaries — CRUD & O(1) lookup 04 → Sets — unique values & operations 05 → Frozenset 06 → Advanced — defaultdict, Counter, namedtuple 07 → Iterators — iter() & next() 08 → Mini Project — Inventory Management System Built a simple system using dictionaries to manage stock & pricing — a real-world pattern used in inventory and data pipelines. Day 1 ✅ Day 2 ✅ Day 3 ✅ Day 4 ✅ Day 5 ✅ Day 6 ✅ 4 more to go. Drop a 🐍 if you’ve ever used a list when a set would’ve been better 😄 #Python #Collections #DataEngineering #LearningInPublic #CleanCode #10DaysOfPython #DataStructures

4 of my #100DaysOfCode Moving from simple variables into actual Data Structures using Python Lists. As I grow in data analytics, I know organizing and manipulating data is the core of the job, so getting comfortable with lists is a critical foundation. Here is what I tackled in day 4. Randomisation: Using the Mersenne Twister (import random) and randint() to generate unpredictable outcomes. Lists: Creating, altering, and managing data structures using brackets []. List Methods: How to use .append(), .extend(), .insert(), and .pop(). Indexing: Accessing specific data points (and successfully conquering negative indexing!). To put it all together, we built a fully functional Rock, Paper, Scissors game that plays against the user.

Day 9/120 – Today I learned something most beginners ignore… but pros don’t 😳🔥 Yesterday → Lists Today → CONTROL over data 👇 👉 Tuples & Sets in Python Here’s the problem 🤯 Lists can be changed anytime… But what if your data SHOULD NOT change? ❌ Example: Coordinates 📍 Dates 📅 Configurations ⚙️ That’s where TUPLES come in 👇 data = (10, 20, 30) ✔ Cannot be modified ✔ Safe & reliable Now comes something even more powerful 👇 👉 SETS nums = {1, 2, 2, 3, 3} Output? 😳 {1, 2, 3} ✔ No duplicates ✔ Clean data This is HUGE in Data Analytics 📊 Now I can: ✔ Protect data (Tuples) ✔ Clean data (Sets) This is getting serious now 🔥 Comment “DATA” if you're learning with me 💪 #Day9 #Python #DataAnalytics #LearningInPublic #CodingJourney #Consistency

Combining data from multiple sources is one of the most common tasks in data analysis and data engineering and in pandas, pd.concat() is the primary tool for getting it done. But there is more to it than just passing two DataFrames and getting one back. Understanding when to use axis=0 vs axis=1, how join handles mismatched columns, why concatenating inside a loop is a performance trap, and when to use concat vs merge. These are the details that separate clean, efficient data pipelines from slow, buggy ones. Get comfortable with pd.concat() and combining data from multiple sources becomes one of the fastest steps in your workflow. Read the full post here: https://lnkd.in/es7KJ7Y9 #Python #Pandas #DataScience #DataEngineering #Analytics #ETL

🐍 Why 𝐢𝐧𝐩𝐥𝐚𝐜𝐞=𝐓𝐫𝐮𝐞 in Pandas Isn’t Always a Good Idea 🚨It looks convenient… but can lead to unexpected issues. 👉𝐖𝐡𝐚𝐭 𝐝𝐨𝐞𝐬 𝐢𝐧𝐩𝐥𝐚𝐜𝐞=𝐓𝐫𝐮𝐞 𝐝𝐨? It directly modifies the original DataFrame. df.dropna(inplace=True) 👉𝐒𝐨𝐮𝐧𝐝𝐬 𝐞𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐭? 𝐘𝐞𝐬. 𝐁𝐮𝐭 𝐡𝐞𝐫𝐞’𝐬 𝐭𝐡𝐞 𝐜𝐚𝐭𝐜𝐡👇 ⚠️𝐖𝐡𝐲 𝐢𝐭 𝐜𝐚𝐧 𝐛𝐞 𝐫𝐢𝐬𝐤𝐲: • Original data gets overwritten • Difficult to debug mistakes • No easy way to revert changes 🎯𝐁𝐞𝐭𝐭𝐞𝐫 𝐀𝐩𝐩𝐫𝐨𝐚𝐜𝐡: Instead of modifying data in place, create a new DataFrame: df = df.dropna() 💡𝐖𝐡𝐲 𝐭𝐡𝐢𝐬 𝐢𝐬 𝐛𝐞𝐭𝐭𝐞𝐫: • Keeps original data safe • Easier to track changes • Improves code readability 𝐂𝐨𝐧𝐯𝐞𝐧𝐢𝐞𝐧𝐜𝐞 𝐢𝐬 𝐠𝐨𝐨𝐝, 𝐛𝐮𝐭 𝐝𝐚𝐭𝐚 𝐬𝐚𝐟𝐞𝐭𝐲 𝐢𝐬 𝐛𝐞𝐭𝐭𝐞𝐫. 🔥 #Python #Pandas #DataAnalytics #DataAnalyst #Learning

Recently, I was working on a small Pandas analysis project involving merging user and order datasets. What looked like a straightforward merge turned into an interesting learning moment. The code ran correctly, the output looked structured, and everything seemed fine initially until I noticed one metric wasn’t aligning with what I expected. That led me to explore how dataset relationships can impact analysis after merges, especially when working with transactional data. I wrote a short blog sharing the example, what I observed, and the approach I used to fix it. #Python #Pandas #DataAnalysis #DataScience #SQL Read here:

Ever noticed how much time goes into just handling files and data every day? I was stuck in a loop — opening multiple Excel files, cleaning data, fixing formats, updating sheets, and repeating the same steps daily. Easily 1.5–2 hours gone. Then one simple thought hit me — what if this entire flow could run on its own? So I built a automation using: 1. Python 2. Pandas (for data handling) 3. Openpyxl (for working with Excel files) Built-in tools like datetime, pathlib, and logging for structure and tracking Now, what used to take hours runs in just a few minutes. More than saving time, it made me realize — a lot of “routine work” is just an automation waiting to happen. Still learning, but definitely seeing work differently now. #Python #Automation #DataAnalytics #Learning

I have used *args and **kwargs for years by copy-pasting patterns I found on Stack Overflow.   Today I actually understand them.   The simple version: *args = accept any number of positional arguments as a tuple **kwargs = accept any number of keyword arguments as a dictionary   Why does this matter in data work?   Imagine a validation function. You want it to accept any number of rules — not just 2, not just 5. Any number.   Without *args: def validate(data, rule1, rule2, rule3): # what if I have 10 rules?    pass   With *args: def validate(data, *rules):    for rule in rules:        if not rule(data):            print(f'Failed: {rule.__name__}')   Now I can call: validate(df, check_nulls, check_schema, check_dates, check_amounts)   Any number of rules. Clean interface. One function definition.   **kwargs is for when the rules need configuration: validate(data, null_threshold=0.05, date_column='txn_date')   The insight from Corey: *args and **kwargs are not advanced Python. They are the way Python lets functions be flexible. Once you see that, they become obvious.   What patterns clicked for you only after someone explained WHY, not just HOW?   ---- #Python #LearningInPublic #DataEngineering #CodingTips #PythonFunctions