top

Search

Machine Learning Tutorial

The input data to a learning algorithm usually has a row x column structure, and is usually a CSV file. CSV refers to comma separated values which is a simple file format that helps in storing tabular data structure. This CSV format can be easily loaded into a Pandas dataframe with the help of the read_csv function. The CSV file can be loading using other libraries as well, and we will look at a few approaches in this post. Let us now load CSV files in different methods: Using Python standard library There are built-in modules, such as ‘csv’, that contains a reader function, which can be used to read the data present in a csv file. The CSV file can be opened in read mode, and the reader function can be used. Below is an example demonstrating the same: import numpy as np  import csv  path = path to csv file  with open(path,'r') as infile:  reader = csv.reader(infile,delimiter = ',')  headers = next(reader)  data = list(reader)  data = np.array(data).astype(float) The headers or the column names can be printed using the following line of code: print(headers) The dimensions of the dataset can be determined using the shape attribute as shown in the following line of code: print(data.shape)  Output: 250, 302 The nature of data can be determined by examining the first few rows of the dataset using the below line of code: data[:2] Using numpy package The numpy package has a function named ‘loadtxt’ that can be used to read CSV data. Below is an example demonstrating the same using StringIO. from numpy import loadtxt  from io import StringIO  c = StringIO("0 1 2 \n3 4 5")  data = loadtxt(c)  print(data.shape) Output: (2, 3) Using pandas package There are a few things to keep in mind while dealing with CSV files using Pandas package. The file header is basically the name of the column which describes that type of data the column holds. If the file already has a header, the function automatically assigns the same names to every column, otherwise every column needs to be manually named. In any case, we need to explicitly mention in the read_csv function whether or not the CSV file contain header. Comments in a CSV file are written using the # symbol. Let us look at an example to understand how the CSV file is read as a dataframe. import numpy as np  import pandas as pd  #Obtain the dataset  df = pd.read_csv("path to csv file", sep=",")  df[:5] Output: target012 ...295296297298299 0  1.0 -0.098 2.165 0.681 ...  -2.097 1.051 -0.414 1.038 -1.065 1  0.0 1.081 -0.973 -0.383 ...  -1.624 -0.458 -1.099 -0.936 0.973 2  1.0 -0.523 -0.089 -0.348 ...  -1.165 -1.544 0.004 0.800 -1.211 3  1.0 0.067 -0.021 0.392 ...  0.467 -0.562 -0.254 -0.533 0.238 4  1.0 2.347 -0.831 0.511 ...  1.378 1.246 1.478 0.428 0.253 [5 rows x 302 columns] Conclusion In this post, we saw how input data can be loaded for machine learning projects. 
logo

Machine Learning Tutorial

Data Loading for ML Projects

The input data to a learning algorithm usually has a row x column structure, and is usually a CSV file. CSV refers to comma separated values which is a simple file format that helps in storing tabular data structure. This CSV format can be easily loaded into a Pandas dataframe with the help of the read_csv function. The CSV file can be loading using other libraries as well, and we will look at a few approaches in this post. 

Let us now load CSV files in different methods: 

Using Python standard library 

There are built-in modules, such as ‘csv’, that contains a reader function, which can be used to read the data present in a csv file. The CSV file can be opened in read mode, and the reader function can be used. Below is an example demonstrating the same: 

import numpy as np 
import csv 
path = path to csv file 
with open(path,'r') as infile: 
reader = csv.reader(infile,delimiter = ',') 
headers = next(reader) 
data = list(reader) 
data = np.array(data).astype(float) 

The headers or the column names can be printed using the following line of code: 

print(headers) 

The dimensions of the dataset can be determined using the shape attribute as shown in the following line of code: 

print(data.shape) 
Output: 
250, 302 

The nature of data can be determined by examining the first few rows of the dataset using the below line of code: 

data[:2] 

Using numpy package 

The numpy package has a function named ‘loadtxt’ that can be used to read CSV data. Below is an example demonstrating the same using StringIO. 

from numpy import loadtxt 
from io import StringIO 
c = StringIO("0 1 2 \n3 4 5") 
data = loadtxt(c) 
print(data.shape) 

Output: 

(2, 3) 

Using pandas package 

There are a few things to keep in mind while dealing with CSV files using Pandas package. 

  • The file header is basically the name of the column which describes that type of data the column holds. If the file already has a header, the function automatically assigns the same names to every column, otherwise every column needs to be manually named. 
  • In any case, we need to explicitly mention in the read_csv function whether or not the CSV file contain header. 
  • Comments in a CSV file are written using the # symbol. 

Let us look at an example to understand how the CSV file is read as a dataframe. 

import numpy as np 
import pandas as pd 
#Obtain the dataset 
df = pd.read_csv("path to csv file", sep=",") 
df[:5] 

Output: 

target012 ...295296297298299 
  1. 0  1.0 -0.098 2.165 0.681 ...  -2.097 1.051 -0.414 1.038 -1.065 
  2. 1  0.0 1.081 -0.973 -0.383 ...  -1.624 -0.458 -1.099 -0.936 0.973 
  3. 2  1.0 -0.523 -0.089 -0.348 ...  -1.165 -1.544 0.004 0.800 -1.211 
  4. 3  1.0 0.067 -0.021 0.392 ...  0.467 -0.562 -0.254 -0.533 0.238 
  5. 4  1.0 2.347 -0.831 0.511 ...  1.378 1.246 1.478 0.428 0.253 
[5 rows x 302 columns] 

Conclusion 

In this post, we saw how input data can be loaded for machine learning projects. 

Leave a Reply

Your email address will not be published. Required fields are marked *