Is Logistic Regression classification linear?
We all know Neural Network is a non-linear model. And how about the key component: logistic regression? You may hear logistic is linear model, but the sigmoid function itself obviously looks not like linear. Let's take 3 mins to explore it.
Logistic Regression function: output = Sigmoid(w*x), note: w is the weight vector, x is the input vector, * is inner product, w*x is the sum of pairwise product of every elements in w and x, sigmoid is a non-linear function output=1/(1+e^-input).
Logistic Regression Classifier Pipeline: Let's say we want to predict weather (if it will rain tomorrow based on today humidity, today precipitation, today air pressure, today stock price ). Input x is [humidity=0.9, precipitation=0.9, air pressure=0.3, stock price=1.0], wights is [1, 0.9, 0.2, 0.001, after the logistic regression pipeline, you can see you got output is tomorrow will rain.
Sigmoid Function Plot:
The usage for Logistic Regression: the above example shows how logistic regression being used as classifier, but also can be applied for numerical regression when you get rid of the step function in the end. (If you do that, you just get the numerical value 0.895). Since the range of sigmoid function output is (1,0), some people will take the numerical value as probability value. if the model is well trained, you can claim the probability of raining tomorrow is 0.895.
Linearity: from the above sigmoid plot, you know the function is non-linear. However, logistic regression is a member of generalized linear model(GLM)! So, don't get confused when hearing someone says logistic regression is linear next time.
Quoted from Balaji Pitchai Kannu, " Generalized linear models (GLM) are broad class of models that include linear regression, logistic regression, log linear regression, Poisson regression, ANOVA, ANCOVA, etc. In order to call a particular method to be GLM, that method should have following three components.
- Random Component: It refers a response variable (y), which need to satisfy some PDF assumption. For example: Linear regression of y (dependent variable) follows normal distribution. Logistic regression response variable follows binomial distribution.
- Systematic Component: It is nothing but explanatory variables in the model. Systematic components helps to explain the random component.
- Link Function: It is link between systematic and random component. Link function tells how the expected value of response variable relates to explanatory variable. Link function of linear regression is E[y] and link function of logistic regression is logit(π).
"
Source: https://www.quora.com/Why-is-logistic-regression-considered-a-linear-model, Acknowledge: the picture is modified from Quora user Sebastian Raschka. Rain image is from https://i.ytimg.com/vi/JzZh8kJJwe4/maxresdefault.jpg