About Linear Regression

Every Data Scientist starts with this one. So, here it is.

Linear Regression is one of the most widely used Artificial Intelligence algorithms in real-life Machine Learning problems — thanks to its simplicity, interpretability and speed! We shall now understand what’s behind the working of this algorithm in the next few minutes!

What is Linear Regression? 

• It’s a method to predict a target variable by fitting the best linear relationship between the dependent and independent variables.

• It comes Under supervised Learning (Both x and y)

• Linear regression algorithm which is used to predict “Continuous target variable”

• By Fitting Straight Line.

It helps determine the following:

•  If an independent variable does an excellent job of predicting the dependent variable.

• Which independent variable plays a significant role in predicting the dependent variable.

Assumption of Linear Regression:

• The Independent variables should be “linearly related” to the “dependent variables”.
• This can be examined with the help of several visualisation techniques like Scatter plots or maybe you can use Heatmap or pair plot 
• Every feature in the data is “Normally Distributed”.
• This again can be checked with the help of different visualization Techniques, such as Q-Q plot, histogram and much more.
• There should be little or no multi-collinearity in the data. ( Multi-collinearity→ There should not be a high correlation between independent variables.)
• The best way to check the presence of multi-collinearity is to perform VIF(Variance Inflation Factor).
• The mean of the residual is zero.
• A residual is the difference between the observed y-value and the predicted y-value. However, Having residuals closer to zero means the model is doing great.
• Residuals obtained should be normally distributed.
• This can be verified using the Q-Q Plot on the residuals.

Types OF Linear Regression:

Simple Linear Regression: Simple Linear Regression helps to find the linear relationship between two continuous variables, One independent and one dependent feature.

The formula can be represented as y=mx+b or

Multilinear Regression: We Often use Multiple Linear Regression to do any kind of predictive analysis as the data we get has more than 1 independent feature to it.

The formula can be represented as Y=mX1+mX2+mX3…+b, OR

How Linear Regression Works:

• The whole idea of linear Regression is to find the best-fit line, which has a very low error(cost function).

This line is also called the Least Square Regression Line(LSRL).

• Linear Regression learns data by fitting drawing straight Line

Y=mx+c      where m=slope, c=intercept

• Linear Regression will find the slope and intercept

How do you say model Best or not: 

If the error is less then it's the Best model otherwise it's a bad model.

How To Calculate Average Error or Total Error:

Mean Squared Error(MSE)

Mean Absolute Error(MAE)

Root Mean square Error(RMSE)

“Important: MSE, MAE, RMSE are the error function which tells about a total error made by the model”

“ Model is best only if Total error is less”

How to evaluate the Model?/How to measure the performance of the model? 

R-squared /R2-score/R^2 - score: 

Disadvantages of R^2 : 

Increases as the number of independent variables increases which has very less relationship with the target variable to overcome the above issue we use Adjusted R^2

Adjusted R^2: 

Will measure the performance of the model by ignoring columns that have very less relationship with the target.

NOTE: “adjusted R squared <r squared then we say it Good Model.”

Linear Regression with Gradient Descent:

• An optimization algorithm that is commonly used to train machine learning models and neural networks. 

• Gradient Descent is defined as one of the most commonly used iterative optimization algorithms of machine learning to train the machine learning and deep learning models. It helps in finding the local minimum of a function.

The best way to define the local minimum or local maximum of a function using gradient descent is as follows:

• If we move towards a negative gradient or away from the gradient of the function at the current point, it will give the local minimum of that function.
• Whenever we move towards a positive gradient or the gradient of the function at the current point, we will get the local maximum of that function.

This entire procedure is known as Gradient Ascent, which is also known as steepest descent. The main objective of using a gradient descent algorithm is to minimize the cost function using iteration. To achieve this goal, it performs two steps iteratively:

• Calculates the first-order derivative of the function to compute the gradient or slope of that function.
• Move away from the direction of the gradient, which means the slope increased from the current point by alpha times, where Alpha is defined as the Learning Rate. It is a tuning parameter in the optimization process that helps to decide the length of the steps.

How does Gradient Descent work?

Gradient descent starts with a random slope and works iteratively to reach global minima.

I hope this article helped you understand the Algorithm and Most of the concepts related to it.

Coming up next Week, We will Understand the Logistic Regression.

HAPPY LEARNING!!!!!

Like my article? Do give me a clap and share it, as that will boost my confidence. Also, I post new articles every Sunday so stay connected for future articles on the basics of data science and machine learning series.

Also, do connect with me on 

For Model Building , Do connect me on GitHub

_Thank_You_