awk and sed - anjavdl/PHY517_AST443 GitHub Wiki

awk

awk is useful for performing calculations on tabular data, as well as other things. Consider the following data file:

# Name RA Dec V-mag
Object1 12.3456789 45.012345 4.0
Object2 98.987654 0.345678 15.0
Object3 240.456789 -30.345678 6.9

and assume that this was saved as an ascii file, objects.dat .

Suppose we want to know which objects are brighter than 10th magnitude; the command

awk '{if($4<10) print $0}' objects.dat

will print every row where the number in the fourth column is smaller than 10. To save the output to a new file, simply pipe it into that file:

awk '{if($4<10) print $0}' objects.dat > bright.dat

Note that the header lines (starting with a #) does not meet the criterion, and thus is not printed. To preserve it, you could change the line to one of these

awk '{if(NR==1 || $4<10) print $0}' objects.dat
awk '{if($1=="#" || $4<10) print $0}' objects.dat

More information about awk can be found here:
https://www.gnu.org/software/gawk/manual/gawk.html
http://www.grymoire.com/Unix/Awk.html

sed

sed is a stream-editor, i.e. it works solely on standard output (not on files, though that's easy to work around using cat). It uses regular expressions for search-and-replace. For example, to change Object in the example file to O, you would type

cat objects.dat | sed 's/Object/O/g'

The sed manual can be found here: https://www.gnu.org/software/sed/manual/sed.html