Z-score vs IQR Method for Outlier Detection

Some instructions are used more than once. When working with data, certain operations tend to repeat. Cleaning a value. Checking a condition. Transforming a piece of information. Applying the same rule across different parts of a dataset. Writing the same set of instructions every time would quickly make code longer and harder to follow. This is where functions come in. A function is simply a way of grouping a set of instructions under a name so that the same logic can be used again whenever it is needed. Instead of rewriting the steps, the program calls the function and runs those instructions again. For example: def check_pass(score): if score >= 50: return "Pass" return "Fail" Once defined, the same logic can be applied wherever it is needed. check_pass(72) check_pass(43) The instructions stay the same. Only the input changes. Functions don’t introduce new logic. They organize existing logic so it can be reused clearly and consistently. And in larger programs, that organization becomes just as important as the logic itself. Day 28 / 30. #30DaysOfDataScience #Python #Functions #ProgrammingLogic #LearningInPublic

🐍 Day 72 — Mean (Average) Day 72 of #python365ai ➗ The mean is the average value of a dataset. Example: import numpy as np data = [10, 20, 30] print(np.mean(data)) 📌 Why this matters: The mean helps describe the central tendency of data. 📘 Practice task: Calculate the mean of five numbers. #python365ai #Mean #Statistics #Python

📊 Why reset_index() matters after groupby() in Pandas When you use groupby() in Pandas, something important happens behind the scenes. The column you group by becomes the index of the result. This is helpful for analysis, but it can create problems when you want to: • Export the data • Merge it with another dataset • Create visualizations • Work with it like a normal table That’s why analysts often use reset_index() after groupby(). It converts the grouped index back into a regular column, making the dataset easier to work with again. 🧠 Key insight: groupby() changes the structure of your data. reset_index() restores it to a tabular format. It’s a small detail — but one that saves a lot of confusion when working with Pandas. #Pandas #DataAnalytics #Python

Data Correlation Tip: Using the Pearson Coefficient Identifying relationships between variables is a key step in data analysis. The Pearson correlation coefficient measures the strength and direction of a linear relationship between two variables — for example, advertising spend and sales revenue. Here’s how to interpret it: +1 → Strong positive relationship 0 → No linear relationship –1 → Strong negative relationship It’s simple to calculate: In Excel using =CORREL(range1, range2) In Python with pandas using .corr() While correlation does not imply causation, it provides a valuable foundation for deeper analysis and informed, data-driven decisions. #DataAnalytics #Statistics #Correlation #KenyaTech #Python #Excel

📢💡 𝐃𝐚𝐲 𝟗 – 𝐫𝐞𝐩𝐚𝐫𝐭𝐢𝐭𝐢𝐨𝐧() 𝐯𝐬 𝐜𝐨𝐚𝐥𝐞𝐬𝐜𝐞() 🤔 𝐒𝐜𝐞𝐧𝐚𝐫𝐢𝐨 ✔️ Demonstrate partition control impact on small dataset (interview favorite). 📍 𝐈𝐧𝐩𝐮𝐭 𝐝𝐚𝐭𝐚 : 𝐩𝐲𝐭𝐡𝐨𝐧 data = list(range(1, 21)) # 20 numbers 📤 𝐄𝐱𝐩𝐞𝐜𝐭𝐞𝐝 𝐨𝐮𝐭𝐩𝐮𝐭 👉 Original partitions: 4 👉 After repartition(2): 2 👉 After coalesce(8): 8 (no shuffle) 🧠 𝐄𝐱𝐩𝐥𝐚𝐧𝐚𝐭𝐢𝐨𝐧 ✔️ 𝐫𝐞𝐩𝐚𝐫𝐭𝐢𝐭𝐢𝐨𝐧(n) = shuffle + resize (even distribution) ✔️ 𝐜𝐨𝐚𝐥𝐞𝐬𝐜𝐞(n) = merge partitions (reduce only, no shuffle) ✔️ 𝐈𝐧𝐭𝐞𝐫𝐯𝐢𝐞𝐰: Use coalesce for reduce, repartition for increase. #python #Spark #Pyspark #Dataengineering #Bigdata #learnmore #pythonmcq #programmingwithpython #mcq #spark #Practicewithme

Python Data Visualization Quick Guide V1.0 📊 What’s inside: • Distribution plots (Histogram, KDE, Box, Violin) • Categorical analysis (Bar, Count, Pie) • Relationship plots (Scatter, Regression, Bubble) • Time series visualizations (Line, Area) • Multivariate exploration (Heatmaps, Pairplots) • Hierarchical charts (Sunburst, Treemap) • Geographic maps with Plotly • Faceting and subplot layouts • A Visualization Selection Guide to help choose the right chart quickly 🔗 Notebook link: https://lnkd.in/daHNQpdq I’d love to hear your feedback and suggestions for improving it further. #Python #DataScience #DataVisualization #EDA #MachineLearning #Plotly #Seaborn #Matplotlib

📊 Day 19 — 60 Days Data Analytics Challenge Today I learned about Crosstab in Pandas, which helps summarize data by showing the relationship between two categorical variables. 🔍 What I practiced today: • Creating cross-tabulations using pd.crosstab() • Understanding category-wise data distribution • Using margins=True to include total values • Improving table readability with row and column labels This feature is very helpful during Exploratory Data Analysis (EDA) because it allows us to quickly compare categories and identify patterns in the dataset. #DataAnalytics #Python #Pandas #60DaysChallenge #LearningJourney

Learning from Data Cleaning: Handling Mixed Date Formats in a DataFrame While working with a dataset recently, I noticed that the date column contained multiple formats. Because of this, converting the column to datetime was causing errors and incorrect parsing. To handle this, I used pandas to_datetime() with: format="mixed" – which allows pandas to parse multiple date formats within the same column errors="coerce" – which converts invalid or unrecognizable dates into NaT instead of breaking the code After applying this approach, most of the date values were parsed correctly, making the dataset much cleaner and ready for analysis. Key takeaway: Real-world datasets rarely come perfectly formatted. Using parameters like format="mixed" and errors="coerce" can significantly improve data quality and preprocessing efficiency. #DataAnalytics #Python #Pandas #DataCleaning #DataScience #DataPreparation

Excel… but supercharged. ⚡ That’s the simplest way I can describe what working with NumPy, Pandas, and Matplotlib in Python feels like. Organising data, running calculations, filtering information, and creating visual insights all follow familiar logic, but moving from spreadsheets to code removes the usual limits. Everything becomes faster, more flexible, and able to handle far larger datasets. The transition from applications to programming is where data truly comes alive. What seems complex at first starts to feel intuitive once you understand the structure behind it. The deeper I go, the more everything connects. Building the foundation one layer at a time. 🚀 Let’s keep learning… #Python #MachineLearning #DataAnalysis #NumPy #Pandas #Matplotlib #LearningInPublic #ContinuousLearning