Introduction to Logistic Regression - Sigmoid Function, Code Explanation

  • Avinash Mishra
  • Aug 21, 2019
  • Machine Learning
Introduction to Logistic Regression - Sigmoid Function, Code Explanation title banner

Logistic Regression is basically a predictive model analysis technique where the output (target) variables are discrete values for a given set of features or input (X). It is a very powerful yet simple supervised classification algorithm in machine learning.


Around 60% of the world’s classification problems can be solved by using the logistic regression algorithm. In this blog, we go all the basic concepts of logistic regression with code explanation.


What is Logistic Regression?


Logistic regression is one of the most common algorithms used for binary classification. It predicts the probability of occurrence of a binary outcome using a logit function. It is a special case of linear regression as it predicts the probabilities of outcome using log function.


We use the activation function (sigmoid) to convert the outcome into categorical value. There are many examples where we can use logistic regression for example, it can be used for fraud detection, spam detection, cancer detection, etc.



Difference between Linear Regression vs Logistic Regression


Linear Regression is used when our dependent variable is continuous in nature for example weight, height, numbers, etc. and in contrast, Logistic Regression is used when the dependent variable is binary or limited for example: yes and no, true and false, 1 or 2 or 3 or 4, etc. Linear regression uses the ordinary least square method to minimize the error and arrives at the best possible solution, and the Logistic regression achieves the best outcomes by using the maximum likelihood method. In the 19th century, people use linear regression on biology to predict health disease but it is very risky for example if a patient has cancer and its probability of malignant is 0.4 then in linear regression it will show that cancer is benign (because probability comes <0.5). That’s where Logistic Regression comes which only provides us with binary results.



What is the Sigmoid Function?


It is a mathematical function having a characteristic that can take any real value and map it to between 0 to 1 shaped like the letter “S”. The sigmoid function also called a logistic function.


Y = 1 / 1+e -z

Outlining Sigmoid Function curve mapping between the values 0 to 1.


Sigmoid function


So, if the value of z goes to positive infinity then the predicted value of y will become 1 and if it goes to negative infinity then the predicted value of y will become 0. And if the outcome of the sigmoid function is more than 0.5 then we classify that label as class 1 or positive class and if it is less than 0.5 than we can classify it to negative class or label as class 0.


Code in Python


You can find the dataset here Dataset. First of all, before proceeding we first import all the libraries that we need to use in our algorithm


Logistic Regression Python Code


After initializing all the libraries that we need in our algorithm know we have to import our dataset with the help of the pandas library and split our dataset into training and testing set with the help of the train_test_split library.



As we divide our dataset on the basis of train and test split know we have to scale our feature dataset with the help of StandardScaler library and apply logistic regression on the training set and check the accuracy sore with the help of accuracy_score library.


Logistic Regression Sklearn


We have successfully applied logistic regression on the training set and see that our accuracy scores come 89%. So, if we take on basis of algorithm it is not so much worse for prediction. As we get the accuracy score of our model now we can see a pictorial representation of our dataset first we have to visualize the result on the basis of the training dataset.




We plot a picture on the basis of age and estimated salary in which we bifurcate our result in a 0 and 1 value basis. In the same process, we apply for the test set and visualize our result how accurate our prediction is.





The main concept regarding this blog is to explain logistic regression and simple explanation via python code. I think the above blog is very helpful for you to clear your doubts regarding logistic regression more blogs are on the way to stay tuned with us! Keep exploring Analytics Steps.


Avinash Mishra

Avinash claims words should be filled in the void of Knowledge and thus, he started writing on various topics that covers, all what is under Sun. His interests range from political pessimism to Technological scepticism.

Trending blogs

  • Introduction to Time Series Analysis: Time-Series Forecasting Machine learning Methods & Models

  • How is Artificial Intelligence (AI) Making TikTok Tick?

  • 7 Types of Activation Functions in Neural Network

  • 7 types of regression techniques you should know in Machine Learning

  • 6 Major Branches of Artificial Intelligence (AI)

  • Introduction to Logistic Regression - Sigmoid Function, Code Explanation

  • What is K-means Clustering in Machine Learning?

  • Top 10 Big Data Technologies in 2020

  • Introduction to Linear Discriminant Analysis in Supervised Learning

  • Convolutional Neural Network (CNN): Graphical Visualization with Code Explanation

Write a BLOG