Scikit learn is a module in Python that is used for data analysis and data mining purposes.
pip install scikit-learn
Following are the steps in implementing learning algorithm using scikit-learn:
A dataset can be collected and loaded or a pre-defined dataset can be loaded. It has two features named features and responses.
Features are nothing but attributes or variables present in the dataset. They are represented using a ‘feature matrix’.
Response is also known as a target variable or a label. It is the output which depends on the feature variables. A single response column known as ‘response vector’ is present.
Data can be loaded in different ways and some of them have been demonstrated below:
There are built-in modules, such as ‘csv’, that contains a reader function, which can be used to read the data present in a csv file. The CSV file can be opened in read mode, and the reader function can be used. Below is an example demonstrating the same:
import numpy as np import csv path = path to csv file with open(path,'r') as infile: reader = csv.reader(infile,delimiter = ',') headers = next(reader) data = list(reader) data = np.array(data).astype(float)
The headers or the column names can be printed using the following line of code:
The dimensions of the dataset can be determined using the shape attribute as shown in the following line of code:
The nature of data can be determined by examining the first few rows of the dataset using the below line of code:
Using numpy package
The numpy package has a function named ‘loadtxt’ that can be used to read CSV data. Below is an example demonstrating the same using StringIO.
from numpy import loadtxt from io import StringIO c = StringIO("0 1 2 \n3 4 5") data = loadtxt(c) print(data.shape)
Using pandas package
There are a few things to keep in mind while dealing with CSV files using Pandas package.
Let us look at an example to understand how the CSV file is read as a dataframe.
import numpy as np import pandas as pd #Obtain the dataset df = pd.read_csv("path to csv file", sep=",") df[:5]
id target 0 1 2 ... 295 296 297 298 299 0 0 1.0 -0.098 2.165 0.681 ... -2.097 1.051 -0.414 1.038 -1.065 1 1 0.0 1.081 -0.973 -0.383 ... -1.624 -0.458 -1.099 -0.936 0.973 2 2 1.0 -0.523 -0.089 -0.348 ... -1.165 -1.544 0.004 0.800 -1.211 3 3 1.0 0.067 -0.021 0.392 ... 0.467 -0.562 -0.254 -0.533 0.238 4 4 1.0 2.347 -0.831 0.511 ... 1.378 1.246 1.478 0.428 0.253
Loading a pre-defined dataset
It can be done using the below code.
from sklearn.datasets import load_iris iris = load_iris() #feature matrix and target is stored in 2 variables X = iris.data y = iris.target feature_names = iris.feature_names target_names = iris.target_names #feature names and targets are printed print("Feature names:", feature_names) print("Target names:", target_names) #numpy arrays x and y print("\nType of X is:", type(X)) #first 5 input rows are printed to understand the type of data present in the dataset print("\nFirst 5 rows of X:\n", X[:5])
The next important step in implementing a learning algorithm is to split the dataset into training, testing and validation dataset.
Data is split into different sets so that a part of the dataset can be trained upon, a part can be validated and a part can be used for testing purposes.
This is the input dataset which is fed to the learning algorithm. Once the dataset is pre-processed and cleaned, it is fed to the algorithm. Sometimes, predefined datasets are readily available on multiple websites which can be downloaded and used. Some predefined data sets need to be cleaned and verified but some of them are usually cleaned beforehand. The machine learning model learns from this data and tries to fit a model on this data.
This is similar to the test set, but it is used on the model frequently so as to knowhow well the model performs on never-before seen data. Based on the results obtained by passing the validation set to the learning algorithm, decision can be made as to how the algorithm can be made to learn better- the hyper parameters can be tweaked so that the model gives better results on this validation set in the next run, the features can be combined or new features can be created which better describe the data, thereby yielding better results.
Test data: This is the data on which the model’s performance/its ability to generalize is judged. In theend, the model’s performance can be determined based on how well it reacts to never-before-seen data. This is the data, which is used to test how well the model would generalize on new data. This is a way of knowing whether the model actually understood and learnt the patterns or it just overfit or underfit the data.
It is important to understand that good quality data (less to no noise, less to no redundancy, less to no discrepancies) in large amounts yields great results when the right learning algorithm is applied on the input data.
The dataset needs to be split into training and test datasets, so that once the training is completed on the training dataset, the performance of the learning model is tested on the test dataset. Usually, 80 percent of the data is used for training and 20 percent of the data is assigned for testing purposes. This can be achieved using the scikit-learn library, that has a function named train_test_split. The ‘test_size’ parameter helps in dividing the dataset into training and test datasets.
sklearn.model_selection import train_test_split X_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size=0.2)
This is one of the most important steps in data pre-processing. It refers to standardizing the range of independent variables or features present within the dataset. When all the variables are transformed to
the same scale, it is easier to work with machine learning equations. This can be achieved using the ‘StandardScaler’ class that is present in the scikit-learn library. The training dataset has to first be fit on the learning model and then transformed. On the other hand, the test dataset needs to just be transformed.
from sklearn.preprocessing import StandardScaler sc_X = StandardScaler() X_train = sc_X.fit_transform(X_train) X_test = sc_X.transform(X_test) Model training Let us look at how a model can be trained sing KNN algorithm. from sklearn.datasets import load_iris iris = load_iris() #loading the iris dataset The feature matrix and respons evectors are stored X = iris.data y = iris.target #x and y are split into training and testing datasets from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=1) #model is trained on the training data from sklearn.neighbors import KNeighborsClassifier knn = KNeighborsClassifier(n_neighbors=3) knn.fit(X_train, y_train) #predictions are made on the test data y_pred = knn.predict(X_test) #actual response and predicted response is compared from sklearn import metrics print("kNN model accuracy:", metrics.accuracy_score(y_test, y_pred)) #predictions for sample data sample = [[3, 5, 4, 2], [2, 3, 5, 4]] preds = knn.predict(sample) pred_species = [iris.target_names[p] for p in preds] print("Predictions:", pred_species) #the model is saved from sklearn.externals import joblib joblib.dump(knn, 'iris_knn.pkl')
kNN model accuracy: 0.9833333333333333 Predictions: ['versicolor', 'virginica'] Out: ['iris_knn.pkl']
Advantages of using scikit-learn
In this post, we saw how scikit-learn can be used to implement machine learning algorithms with ease.