Logistic Regression in Machine Learning: Easy Guide with Key Examples

Hey guys, today in this post we are going to talk about logistic regression in machine learning so read this post till the end to clear concept about it. Logistic regression is one of the most famous and the fundamental algorithm which is used in machine learning. Logistic regression is the algorithm for classification as it defined, we will go in deep so you will get what exastly it is. It is generally using for the predicting thr binary outcomes and for making it necessary for an applications such as the medicle diagnosis, spamming detection and many more.

What is Logistic Regression in ML?

Logistic regression is the machine learning algorithm for classification and it is a statistical model which uses a logistic function to build a binary depends variable. If we talk it in simple way then it can predicts the probability of given input which belongs to specific group of we can say the category. An output of the LR is between 0 or 1, which is nothing but the probability of the input which is belonging to the positive group or the class.

Mathematically, We can define it as:

Where,

P(Y=1∣X) is probability, [Dependent variable Y is 1 & Input is X]
β0​ = intercept
β1​ is coefficient for an input X

What is Sigmoid Function?

Sigmoid function is one of the famous logistic function which is the main part of the logistic regression. It can plots the random or any number to a value betwen 0 to 1. The sigmoid function is given by,

Sigmoid (x) = 1/1+e^-x

Where, The x is the linear combination of inputs (i.e., x=β0+β1X​).

The sigmoid function assures that an output of logistic regression is between 0 and the 1 which can be interpret as it probability.

How Logistic Regression Works?

Now, We would like to talk about the logistic regression working method like how it is working so it is basically works by the searching the best model which describes the connection between the binary dependent variable and the many more independent variables. So the process can be like as follows:

• Model Initializes: It is starting with the random initial values for coefficients beta 0 and beta 1.
• Prediction: Using the values of the coefficients for the making the predication using the sigmoid function.
• Cost Functin: Calculating the loss function, which is the error between the probablity and the actual outcomes and the most using cost function is for the logistic regression is the binary corss entropy loss.
• Optimization: We can use the various optimisation techniques like the gradient descent, convex to updates the coefficients in a such way that can reduce the cost function.
• Iteration, epochs: To repeating the process of the predication, calculation of the cost and optimisation untill the model can the best set of coefficients.

Also Read: Support Vector Machine (SVM)

Cost Function in Logistic Regression

Now, If we proced further or we would like to talk about the cost function in logistic regression then it is derived from the likelihood function. The target is to max the likelihood of data which is been observed the model params. The cost function is also known as the log loss, for the logistic regression is can defined as the:

Where,

M – Number of training examples.
Yi​ – Actual label of the i-th example.
hθ(xi)​ – Predicted probability for the i-th example.

The objective or the target is to reduce the cost function which would lead to the best fitted model.

Regularization in Logistic Regression

Regularization is the technique to preventing the overfitting. Overfitting is occurs when the model are learning the noise in the data of training rather than the actual patterns. It’s adds as the penalty to cost function which can be troubling the model from becoming too much complex.

There are two types generally:

1. L1 Regularization
2. L2 Regularization

L1 Regularization:

L2 Regularization:

Applications of Logistic Regression in Machine Learning

Logistic regression is using in so many areas so if we would like to talk about some of them then some common applications are the:

• Medicle Diagnosis
• Spamming Detection
• Marketing & Advertising
• Tumor Classification
• Disaster Management

Advantages & Disadvantages of Logistic Regression

There are so many advatages as well as disadvantages of logistic regression but we are going to talk here some of them:

Advantages:

• Simple: It is simple and easy to implement and iterpret.
• Efficient: It can computationally less expensive.
• Probability: It can gives the probablity interoretations.

Disadvantages:

• Linear Assumptions: It is assuming the a linear connection between the input variable and the log odds output.
• Binary Classification: It is been limited for the binary classification tasks.
• Sensitive to an Outliers: It can affects the performance of the model.

Conclusion

Logistic regression is one of the important algorithm in the machine learning. It is very simple and easy to interprets and it’s so efficient for the binary classification tasks. The cost function and the regularisation techniques is also the very important for the applying the logistic regression in the real life issues.

Whether you are predicts the spam content or diagnosing the diseases then the logistic regression is the first algorithm that you should consider, Thanks!

Frequently Asked Questions