Table of Contents
Using awk command with sub() method
Use the awk command with sub() method to remove the first column from the given CSV file (inputFile.csv) in Bash. We will not update the input file but display the output on the Bash console.
|
1 2 3 4 5 6 7 8 |
index,firstname,lastname 1,John,Williamson 2,Nancy,Paul 3,Martin,Guptill 4,Adam,Douglass 5,Sam,Brown |
|
1 2 3 |
awk -F, '{$1="";sub(/,/,"")}1' OFS=, inputFile.csv |
|
1 2 3 4 5 6 7 8 |
firstname,lastname John,Williamson Nancy,Paul Martin,Guptill Adam,Douglass Sam,Brown |
Here we used $1="" to replace first column with empty string and sub() method to replace remaining , with empty string.
Use the awk command to remove the first column from the given CSV file (inputFile.csv) in Bash. Note that we are not required to update the input file but store the modified content in a separate output file (output.csv) using the redirection operator (>).
|
1 2 3 4 5 6 7 8 |
index,firstname,lastname 1,John,Williamson 2,Nancy,Paul 3,Martin,Guptill 4,Adam,Douglass 5,Sam,Brown |
|
1 2 3 |
awk -F, '{$1="";sub(/,/,"")}1' OFS=, inputFile.csv > output.csv |
|
1 2 3 4 5 6 7 8 |
firstname,lastname John,Williamson Nancy,Paul Martin,Guptill Adam,Douglass Sam,Brown |
Use the awk command with the -i parameter set to inplace to remove the first column from the specified CSV file (inputFile.csv) by directly modifying it.
|
1 2 3 4 5 6 7 8 |
index,firstname,lastname 1,John,Williamson 2,Nancy,Paul 3,Martin,Guptill 4,Adam,Douglass 5,Sam,Brown |
|
1 2 3 |
awk -i inplace -F, '{$1="";sub(/,/,"")}1' OFS=, inputFile.csv |
|
1 2 3 4 5 6 7 8 |
firstname,lastname John,Williamson Nancy,Paul Martin,Guptill Adam,Douglass Sam,Brown |
The following example is similar to the above, but we use the cp command to copy the inputFile.csv into the inputFileBackup.csv file to create the backup and then use the awk command to remove the first column from the given CSV file (inputFile.csv).
|
1 2 3 4 5 6 7 8 |
index,firstname,lastname 1,John,Williamson 2,Nancy,Paul 3,Martin,Guptill 4,Adam,Douglass 5,Sam,Brown |
|
1 2 3 4 |
cp inputFile.csv inputFileBackup.csv awk -i inplace -F, '{$1="";sub(/,/,"")}1' OFS=, inputFile.csv |
|
1 2 3 4 5 6 7 8 |
index,firstname,lastname 1,John,Williamson 2,Nancy,Paul 3,Martin,Guptill 4,Adam,Douglass 5,Sam,Brown |
|
1 2 3 4 5 6 7 8 |
firstname,lastname John,Williamson Nancy,Paul Martin,Guptill Adam,Douglass Sam,Brown |
Further reading:
Using awk Command with for loop
Use the awk command to remove the first column from the given CSV file (inputFile.csv) in Bash. We will not update the input file but display the output on the Bash console.
|
1 2 3 4 5 6 7 8 |
index,firstname,lastname 1,John,Williamson 2,Nancy,Paul 3,Martin,Guptill 4,Adam,Douglass 5,Sam,Brown |
|
1 2 3 4 5 6 7 |
awk -F',' '{ for(i=2; i<=NF; i++){ printf("%s%s", $i, (i==NF) ? RS : FS) } }' inputFile.csv |
|
1 2 3 4 5 6 7 8 |
firstname,lastname John,Williamson Nancy,Paul Martin,Guptill Adam,Douglass Sam,Brown |
Use the awk command to remove the first column from the given CSV file (inputFile.csv) in Bash. Note that we are not required to update the input file but store the modified content in a separate output file (output.csv) using the redirection operator (>).
|
1 2 3 4 5 6 7 8 |
index,firstname,lastname 1,John,Williamson 2,Nancy,Paul 3,Martin,Guptill 4,Adam,Douglass 5,Sam,Brown |
|
1 2 3 4 5 6 7 |
awk -F',' '{ for(i=2; i<=NF; i++){ printf("%s%s", $i, (i==NF) ? RS : FS) } }' inputFile.csv > output.csv |
|
1 2 3 4 5 6 7 8 |
firstname,lastname John,Williamson Nancy,Paul Martin,Guptill Adam,Douglass Sam,Brown |
Use the awk command with the -i parameter set to inplace to remove the first column from the specified CSV file (inputFile.csv) by directly modifying it.
|
1 2 3 4 5 6 7 8 |
index,firstname,lastname 1,John,Williamson 2,Nancy,Paul 3,Martin,Guptill 4,Adam,Douglass 5,Sam,Brown |
|
1 2 3 4 5 6 7 |
awk -i inplace -F',' '{ for(i=2; i<=NF; i++){ printf("%s%s", $i, (i==NF) ? RS : FS) } }' inputFile.csv |
|
1 2 3 4 5 6 7 8 |
firstname,lastname John,Williamson Nancy,Paul Martin,Guptill Adam,Douglass Sam,Brown |
The following example is similar to the above, but we use the cp command to copy the inputFile.csv into the inputFileBackup.csv file to create the backup and then use the awk command to remove the first column from the given CSV file (inputFile.csv).
|
1 2 3 4 5 6 7 8 |
index,firstname,lastname 1,John,Williamson 2,Nancy,Paul 3,Martin,Guptill 4,Adam,Douglass 5,Sam,Brown |
|
1 2 3 4 5 6 7 8 |
cp inputFile.csv inputFileBackup.csv awk -i inplace -F',' '{ for(i=2; i<=NF; i++){ printf("%s%s", $i, (i==NF) ? RS : FS) } }' inputFile.csv |
|
1 2 3 4 5 6 7 8 |
index,firstname,lastname 1,John,Williamson 2,Nancy,Paul 3,Martin,Guptill 4,Adam,Douglass 5,Sam,Brown |
|
1 2 3 4 5 6 7 8 |
firstname,lastname John,Williamson Nancy,Paul Martin,Guptill Adam,Douglass Sam,Brown |
The awk is a Linux tool we use to process and manipulate data. Let’s understand how the awk command removed the first column from the input file, which was inputFile.csv in our case.
In the awk command, the -F specified the field separator as a comma (,). This instructed the awk command to treat every comma-separated value in the CSV file as a separate field. The {...} denoted the code block that must be run for every line in the inputFile.csv file.
We used a for loop inside the code block to iterate through the current line’s each field. It began from the second field (i=2) and continued until the last field (i<=NF), where NF represented the total number of fields in the line.
Inside the for loop, we used a printf statement to print the current field's value ($i); it also checks if the current field is the last field using i==NF. If it is the last field, the RS (Record Separator) was printed to move to the next line; otherwise, the FS (Field Separator) was printed to separate the values.
Remember, the awk command preserved the structure of the CSV file, whether we took the backup or wrote the modified content in a separate file.
