Read CSV Line by Line in Python

CSV files or comma-separated files are used to store tabular data in files. In this article, we will discuss how we can read a CSV file line by line in Python.

Read CSV Line by Line Using the readline() Method in Python

A CSV file is created using commas and newline characters. Each row in a CSV file is separated using the newline character “\n” and each entry in a row is separated by a comma. Following is a sample CSV file. 

To read the CSV file line by line, we will use the readline() method. The readline() method, when invoked on a file object, returns the next line in the file that has not been read. To read the CSV file line by line we will first open the CSV file using the open() method. After that, we can read each line in the file one by one as follows.

Output:

In the output above, you might observe that, after each line of the CSV file, an extra newline has been printed.

This is due to the reason that CSV files contain the newline character after each row. Also, the print() function prints an extra newline character after printing each line. Due to this, two newline characters are printed. To avoid this, you can modify the behavior of the print() function using the “end” parameter.

The end parameter takes a character as an input argument and prints it after printing the input value of the print() function instead of the newline character. To avoid an extra newline character, we will pass an empty string to the “end” parameter. In this way, the extra newline character will not be printed. You can observe this in the following example.

Output:

Alternatively, you can use the rstrip() method to avoid the extra newline character when you read a CSV file line by line using the readline() method in python. In this approach, we will remove the extra newline character from each line in the CSV file.

The rstrip() method, when invoked on a string, removes the space character like newlines and tabs from the right side of the string and returns a modified string. We can print the modified string in place of the original line from the CSV file to avoid the extra newline as follows.

Output:

If you don’t want to print the header of the CSV file, you can choose not to print the value after executing the readline() method for the first time. In the subsequent executions, the rows containing the values will be printed as you can observe in the following example.

Output:

Read CSV Line by Line Using the readlines() Method in Python

Instead of using the readline() method, we can use the readlines() method to read a CSV file line by line. The readlines() method, when invoked on a file object, returns a list of all the lines in the file. 

To read the CSV file line by line, we will first open the file using the open() method. After that, we will invoke the readlines() method on the file object, which will return a list of all the lines in the CSV file. Then, we will iterate through the list and print all the lines one by one as follows.

Output:

Again, you can observe that an extra newline character has been printed after each line of the CSV file. You can avoid that using the “end” parameter of the print() function as follows.

Output:

Alternatively, you can use the rstrip() method to avoid the extra newline character as shown below.

Output:

To print the lines of the CSV file without the header, you can skip the first element of the list returned by the readlines() method as follows.

Output:

Further reading:

Read text file line by line in Python

Read more →

Python read file into string

Read more →

Read CSV Line by Line Using CSV Module in Python

The CSV module provides us with various methods to read CSV files. To read a CSV file line by line in python, we can use the reader() method and the DictReader() method defined in the CSV module. Let us discuss them one by one.

Read CSV Line by Line Using the reader() Method

With the help of the reader() method, we can read a CSV file line by line in python. The reader() method, when invoked on a file object containing a CSV file, returns an iterator. On each iteration, the iterator returns a new line of the CSV file as a list where the elements of the list are the values in the respective row. 

After opening the file and invoking the reader() method on the file, we can iterate through the iterator to read the CSV file line by line as follows.

Output:

To print the lines of the CSV file without the header, you can skip the first value of the iterator using the next() function. When we pass the iterator to the next() function, it returns the value at which the iterator currently is. After that, the iterator is moved to the next element. 

The header of the CSV file is kept at the first position in the iterator returned by the reader() method. When we execute the next() method once, with the iterator as the input value, the iterator moves to the second element. Thus, when we print a value from the iterator, the header is not printed. You can observe this in the following example.

Output:

Read CSV Line by Line Using the DictReader() Method

The DictReader() method, when invoked on a file object containing a CSV file, returns a DictReader object. The DictReader is an iterator that contains each line of the CSV file as a dictionary. In the dictionary, each key is the column name and the associated value is the corresponding value from the CSV file in each line.

You can read each line of the CSV file line by line as a dictionary using the DictReader() method as follows.

Output:

Conclusion

In this article, we have discussed four ways to read a CSV file line by line in python. You can choose any of the approaches at your convenience. All of the approaches have approximately the same time complexity and it all boils down to the format of the output as a deciding factor for choosing one approach to read any CSV file line by line.