Missing Value Interpretation

As an analyst we usually see nan/null values in dataset very frequently. There is a misconception that Nan/NA values are zeros. But actually it is not zero Nan/Na values occurs due to:

• Unavailability of data

• Missing of data

• Intentionally removing data.

Null value handling is one of the major issue which most of the budding analyst’s are facing.As in the case of outliers there is no particular method to fill the Nan/NA values. The method mainly depends on the specific column which the value is residing. Some of the methods are

Central Tendency: we use mean, median mode to fill the remaining rows in a column where the it is not depending on the other values like marks, rating etc.in between all there three we have to use our logical thinking and domain knowledge.

Forward fill/Backward fill: in this type of filling the values are filled from the previous or next data in the row. One of the fine example of this is date, if the data is ordered on basis of date then the missing values can be filled with ffill(forward fill) because there is much more possibility that the next date in the data will be the previous one.

Creating a new variable: if the column is categorical and almost 70% of the data is missing then we can convert that null value into a new variable, so that the model will learn accordingly.

Regression imputation: The predicted value obtained by regressing the missing variable on other variables. So instead of just taking the mean, you’re taking the predicted value, based on other variables. This preserves relationships among variables involved in the imputation model, but not variability around predicted values.

There are so many other methods with which we can handle the missing values.But deleting the feature is one of the last option in machine learning because there will always be something to learn from each feature.

(if any mistake or suggestions please do mention with pleasure)