Remove Last Line from File in Bash

1. Overview

In Unix-like environments, especially in scripting and programming with Bash, there might be scenarios where we need to remove the last line from a file. This operation is typical in data processing, log file management, and scripting.

2. Introduction to Problem Statement

Imagine we have a text file named test.txt containing the following 4 lines:
For example:

Our goal is to remove the last line from the file. Expected output is:

The last line ("Line 4: Finally, this is the last line.") should be removed from the file.

3. Using Combination of tail, wc, and truncate Commands

The fastest way to remove last line from file is to use combination of tail, wc, and truncate commands.

Let’s break down the command to understand how it works:
tail -n 1 test.txt : This extracts the last line of the file.
wc -c: This counts the number of characters in the last line.
xargs -I {} truncate test.txt -s -{}: This uses character count to reduce the file from the end with that many characters using truncate command.

Remember, the output of the one command is the input for the next command within the pipeline.

This method might be the fastest of all the methods as it avoids reading the whole file. Instead, it directly truncates the file from the end.

Please note that head and tail commands might not work on some GNUs. For example, These commands won’t work on MAC OS sometimes.

4. Using sed Command

The sed (Stream Editor) is a powerful and versatile text processing tool that performs text transformations on an input stream (a file or input from a pipeline).

To remove last line from the file, we can use following command:

on Mac OS, we can use following command:

In Bash, the $ is a special character representing the last line, while d is a command used for performing delete operations. So, the $d together means to match the last line in the given file and delete it.

But the sed command prints the output to stdin. To write the output to the file directly, we can use -i option.

In case we want to back up the original file, we can use -i.bak option.

5. Using awk Command

The awk command is a powerful text-processing utility. Let’s use awk to remove the last line from the file.

Let’s understand the awk expression ('NR>1 {print prev} {prev=$0}') below.

• The NR>1 specified the action for lines where the line number (NR) exceeds 1.
• The {print prev} printed the value of the prev variable, representing the previous line.
• The {prev=$0} assigned the value of the current line ($0) to the prev variable for the next iteration.

We can use -i option to make changes to file directly.

To take the backup of input file, we can use the -v parameter to set the value of INPLACE_SUFFIX variable to .bak value to create a backup of the input file (test.txt) first and then update it inplace using -i inplace. The backup file will be named test.Extension.bak.

6. Using head Command

The head command is used to display the beginning of the file. Using the head command, we can print all the lines except the last line by passing -1 with -n option.

Therefore, we can use it to remove last line of file in a simple way.

But the head command prints output to stdin. To make changes to the same file, we can create a temp file and move it back to our original one.

Here && operator is used to execute the second command in case the first command is successful. Therefore, if the head command is successful, then only it will execute mv command.

7. Using tail Command

The tail command is used to display the end of the file but can be used here by reversing the file.

The idea is to reverse the file, remove the last line from the file using tail -n +2, and reverse it back to get the file in the original sequence.

The tac command is the reverse of the cat command in Bash. It reads all input lines, and writes them to the standard output in the reversed order.

First, the tac command reverses the file content, tail -n +2 captures all the lines from 2nd line and last tac command reverses the file to get it back in original format.

Similar to the head command, we can use the following code to make changes to the original file via a temp file:

8. Conclusion

In this article, we explored different ways to remove last line from the file in Bash. Combination of tail, wc, and truncate should be fastest of all the options. Sometimes, we don’t have head and tail command available to us, and we can use awk and sed to do it.