Linear Regression in Machine Learning

Regression is a supervised Machine learning approach which is used to predict continuous values. For example predicting House Prices (in $) based on features like area (square ft) of house.

In Regression we tries to find a line or curve that best fits our training data. The equation of that line and curve is known as hypothesis (model) in which we input the parameters and we get an output.

We would talk about Linear Regression in this article in which we have one dependent variable (y) and one independent variable (x). In Multi-linear regression we have one dependent variable (y) and more than one independent variables (x1), (x2), ...

Preparing our training Data

For now our X_train would be Size (sq ft) and y_train would be Price($1000s).

Let suppose our model eqaution would be : y_pred = W*x + b , where W and b are the parameters which we need to find.

Initialize W and b randomly, let say W = 1, b = -2

Loss would be calculated as = (y_pred - y)^2 for each example, on expanding it we get

Loss = (W*x + b - y)^2

For updating the parameters W and b after each iteration we need to calculate gradients of loss function with respect to parameters, here dLoss/dW and dLoss/db are gradients

dLoss/dW = 2*(W*x + b - y)*x

dLoss/db = 2*(W*x + b - y)*1

we need to add these up for each example and use the final one to update W and b

after completing an iteration over whole dataset update W and b

W = W - learning_rate * dLoss/dW

b = b - learning_rate * dLoss/db

repeat this process until the value of W and b stop changing anymore

use these new W and b to calculate y_pred on X_train and then plot actual line v/s predicted line and check if both overlaps over each other or not.

Here is the github link for the code : Code link github

This is how finally it looks like.