Table of Contents
A dataset has the data neatly arranged in rows and columns. The pandas module in Python allows us to load DataFrames from external files and work on them. The dataset can be in different types of files.
In this tutorial, we will read text file in pandas module in Python.
Using the read_csv() function to read text files in Pandas
The read_csv() function is traditionally used to load data from CSV files as DataFrames in Python. However, a CSV is a delimited text file with values separated using commas. Hence, we can use this function to read text files also.
We can specify various parameters with this function.
- The
headerparameter can ensure whether you want the first row to be the header row or not. By default, it will consider first row to be header of dataframe. - We can specify the index with the
index_colparameter. - The names of the columns can be mentioned in the
namesparameter. - The value of the
delimiterparameter is taken as the separator, which is a comma by default.
Apart from these, there are many more parameters available. These can be found on the documentation for the function on the pandas website.
See the following example.
File Content:
|
1 2 3 4 5 6 7 |
a b 0 5 1 3 5 6 7 8 |
Code:
|
1 2 3 4 5 6 |
import pandas as pd df = pd.read_csv('text.txt') print(df) print(df.shape) |
Output:
|
1 2 3 4 5 6 7 8 |
a b 0 0 5 1 1 3 2 5 6 3 7 8 (4, 1) |
Notice that in the above example, the final shape of the DataFrame is (4,1). This means that the function read the whole file into a single column. This happened because the default delimiter in the case of read_csv() function is a comma. If we wish to rectify this, we need to specify the delimiter as a space.
See the code below.
File Content:
|
1 2 3 4 5 6 7 |
a b 0 5 1 3 5 6 7 8 |
Code:
|
1 2 3 4 5 6 |
import pandas as pd df = pd.read_csv('text.txt', delimiter = ' ') print(df) print(df.shape) |
Output:
|
1 2 3 4 5 6 7 8 |
a b 0 0 5 1 1 3 2 5 6 3 7 8 (4, 2) |
Similarly, we can also have a file where the delimiter is a comma.
See the code below.
File Content:
|
1 2 3 4 5 6 7 |
a,b 0,5 1,3 5,6 7,8 |
Code:
|
1 2 3 4 5 6 |
import pandas as pd df = pd.read_csv('textWithCommas.txt', delimiter = ',') print(df) print(df.shape) |
Output:
|
1 2 3 4 5 6 7 8 |
a b 0 0 5 1 1 3 2 5 6 3 7 8 (4, 2) |
Further reading:
Using the read_table() function to read text files in Pandas
We can read tables from different files using the read_table() function in pandas. This function reads a general delimited file to a DataFrame object.
This function is essentially the read_csv() function with the delimiter = '\t', instead of a comma by default.
For example,
File Content:
|
1 2 3 4 5 6 7 |
a b 0 5 1 3 5 6 7 8 |
Code:
|
1 2 3 4 5 6 |
import pandas as pd df = pd.read_table('text.txt', delimiter = ' ') print(df) print(df.shape) |
Output:
|
1 2 3 4 5 6 7 8 |
a b 0 0 5 1 1 3 2 5 6 3 7 8 (4, 2) |
Note that here also we specified the delimiter as a space. Otherwise, we would have encountered the same problem. This function also supports different parameters like header, index_col, names, and more, but was deprecated in recent versions of pandas.
Using the read_fwf() function to read text files in Pandas
The fwf in the read_fwf() function stands for fixed-width lines. We can use this function to load DataFrames from files. This function also supports text files.
Fixed-width formatted files are not delimited by commas, tabs.
For example,
File Content:
|
1 2 3 4 5 6 7 |
a b 0 5 1 3 5 6 7 8 |
Code:
|
1 2 3 4 5 6 |
import pandas as pd df = pd.read_fwf('text.txt') print(df) print(df.shape) |
Output:
|
1 2 3 4 5 6 7 8 |
a b 0 0 5 1 1 3 2 5 6 3 7 8 (4, 2) |
Since the columns in the file were separated with a fixed-width, this function read the contents effectively into separate columns.
That’s all about read text file in Pandas.
