Linear Regression using PyTorch

Before jumping into the implementation details of Logistic Regression using PyTorch, it is essential to understand what Logistic Regression is, what PyTorch is, how they work together in implementing regression, what type of output they give, and what type of values they help predict. 

What is Logistic Regression? 

It is a supervised classification algorithm that is used to differentiate between different events or values. For example- filtering spam emails, classifying a transaction as legit or fraudulent, and much more. The variable in question is classified as 0 or 1, True or False, Yes or No depending on the input. 

It is a regression model that helps in building a model that predicts the probability of a data item belonging to a certain category. Logistic Regression uses a ‘sigmoid’ function, which has been defined below: 

g(z) = 1/ (1+  −  ) 

The sigmoid function/logistic function looks like below: 

Note: The outcome of a Logistic Regression lies between the values 0 and 1, it can’t be greater than 1,and can’t be less than 0. 

The logistic regression becomes a classification problem when a decision threshold comes into play. 

What is PyTorch? 

PyTorch is an open source machine learning library, which was developed (is currently being updated as well as maintained) by social media giant Facebook. It is based on the Torch library (Torch is open-source, ML based library, scripting language as well as a scientific computing framework), which is currently not being actively developed. 

Hence, PyTorch came into existence. It is widely used in building deep-learning models, and natural language processing tasks (NLP) since it comes with features including Python support, easy-to-use API, 

and support to build on-the-go computational graphs. It contains multiple machine learning libraries that could be used with Python to build interesting applications and solve real-life problems. It comes with CUDA support, which helps in delivering higher speed by enabling it to make use of GPU and its computing resources. The CUDA characteristic can be ignored as well, based on our requirement. 

Now, let us dive into implementing Logistic Regression using PyTorch. 

Implementing Logistic Regression using PyTorch to identify MNIST dataset 

The MNIST dataset is first downloaded and placed in the /data folder. It is then loaded into the environment and the hyperparameters are initialized. Once this is done, the Logistic Regression model is defined and instantiated. Next the model is trained on the MNIST dataset and tested on 5 epochs. 

import torch 
import torch.nn as nn 
import torchvision.datasets as dsets 
import torchvision.transforms as transforms 
from torch.autograd import Variable 
# Downloading the MNIST dataset 
train_dataset = dsets.MNIST(root ='./data', 
train = True, 
transform = transforms.ToTensor(), 
download = True) 
test_dataset = dsets.MNIST(root ='./data', 
train = False, 
transform = transforms.ToTensor()) 
# Loading the dataset 
train_loader = torch.utils.data.DataLoader(dataset = train_dataset, batch_size = batch_size, 
shuffle = True) 
test_loader = torch.utils.data.DataLoader(dataset = test_dataset, batch_size = batch_size, 
shuffle = False) 

Initializing the hyperparameters input_size = 784 num_classes = 10 num_epochs = 5 
batch_size = 100 learning_rate = 0.001 

Model definition 

class LogisticRegression(nn.Module): 
def __init__(self, input_size, num_classes): 
super(LogisticRegression, self).__init__() 
self.linear = nn.Linear(input_size, num_classes) 
def forward(self, x): 
out = self.linear(x) 
return out 
model = LogisticRegression(input_size, num_classes) 

• Loss and Optimizer 
• Softmax computed internally 
• Parameters which need to be updated are set criterion = nn.CrossEntropyLoss() 

optimizer = torch.optim.SGD(model.parameters(), lr = learning_rate) 

• Model is being trained 

for epoch in range(num_epochs): 
for i, (images, labels) in enumerate(train_loader): 
images = Variable(images.view(-1, 28 * 28)) 
labels = Variable(labels) 
Forward + Backward + Optimize optimizer.zero_grad() 
outputs = model(images) 
loss = criterion(outputs, labels) loss.backward() optimizer.step() 
if (i + 1) % 100 == 0: 
print('Epoch: [% d/% d], Step: [% d/% d], Loss: %.4f' 
% (epoch + 1, num_epochs, i + 1, 
len(train_dataset) // batch_size, loss.data)) 
Model is being tested correct = 0 
total = 0 
for images, labels in test_loader: 
images = Variable(images.view(-1, 28 * 28)) outputs = model(images) 
_, predicted = torch.max(outputs.data, 1) total += labels.size(0) 
correct += (predicted == labels).sum() 
print('Accuracy of the model on test images: % d %%' % ( 100 * correct / total)) 

Output: 

Epoch: [ 1/ 5], Step: [ 100/ 600], Loss: 2.1282 
Epoch: [ 1/ 5], Step: [ 200/ 600], Loss: 2.0498 
Epoch: [ 1/ 5], Step: [ 300/ 600], Loss: 1.9539 
Epoch: [ 1/ 5], Step: [ 400/ 600], Loss: 1.8876 
Epoch: [ 1/ 5], Step: [ 500/ 600], Loss: 1.8286 
Epoch: [ 1/ 5], Step: [ 600/ 600], Loss: 1.8078 
Epoch: [ 2/ 5], Step: [ 100/ 600], Loss: 1.6117 
Epoch: [ 2/ 5], Step: [ 200/ 600], Loss: 1.6151 
Epoch: [ 2/ 5], Step: [ 300/ 600], Loss: 1.5423 
Epoch: [ 2/ 5], Step: [ 400/ 600], Loss: 1.5010 
Epoch: [ 2/ 5], Step: [ 500/ 600], Loss: 1.4743 
Epoch: [ 2/ 5], Step: [ 600/ 600], Loss: 1.3641 
Epoch: [ 3/ 5], Step: [ 100/ 600], Loss: 1.4000 
Epoch: [ 3/ 5], Step: [ 200/ 600], Loss: 1.4146 
Epoch: [ 3/ 5], Step: [ 300/ 600], Loss: 1.4325 
Epoch: [ 3/ 5], Step: [ 400/ 600], Loss: 1.2283 
Epoch: [ 3/ 5], Step: [ 500/ 600], Loss: 1.2623 
Epoch: [ 3/ 5], Step: [ 600/ 600], Loss: 1.2492 
Epoch: [ 4/ 5], Step: [ 100/ 600], Loss: 1.2188 
Epoch: [ 4/ 5], Step: [ 200/ 600], Loss: 1.3165 
Epoch: [ 4/ 5], Step: [ 300/ 600], Loss: 1.1442 
Epoch: [ 4/ 5], Step: [ 400/ 600], Loss: 1.1946 
Epoch: [ 4/ 5], Step: [ 500/ 600], Loss: 1.1096 
Epoch: [ 4/ 5], Step: [ 600/ 600], Loss: 1.0626 
Epoch: [ 5/ 5], Step: [ 100/ 600], Loss: 1.0550 
Epoch: [ 5/ 5], Step: [ 200/ 600], Loss: 1.1386 
Epoch: [ 5/ 5], Step: [ 300/ 600], Loss: 1.0494 
Epoch: [ 5/ 5], Step: [ 400/ 600], Loss: 0.9888 
Epoch: [ 5/ 5], Step: [ 500/ 600], Loss: 0.9876 
Epoch: [ 5/ 5], Step: [ 600/ 600], Loss: 1.0121 
Accuracy of the model on test images: 82 % 

Conclusion 

In this post, we understood how MNIST handwritten digits dataset can be identified with the help of Logistic Regression using PyTorch.