perl awk one liner - ghdrako/doc_snipets GitHub Wiki

cat | egrep 'pattern'
cat | awk '/pattern/ { print }'
cat | sed -n '/pattern/ p'
cat | perl -ne 'print if /pattern/'
cat | perl -ne '/pattern/ and print'  # the same as above

The command line flags -ne accomplish the following:

  • -e is used to specify the expression to evaluate.
  • -n wraps the expression inside a while loop that places each input line into $_ and evaluate the expression.

Alternatively, there is also a -p flag which replaces -n, and it allows Perl to simulate sed:

  • -p wraps the expression inside a while loop, placing each input line into $, evaluate the expression which manipulates $, and prints $_, the result.
cat | sed 's/pattern/replacement/flags'
cat | perl -pe 's/pattern/replacement/flags'

Again, this works because regular expression substitution in perl, normally written as '$value =~ s/pattern/replacement/flags' or '$value =~ s{pattern}{replacement}flags', operates on $_ by default.

Here are a few flags that make Perl more awk like, with field separators.

  • -l makes each print statement output a record separator that is the same as input record separator (newline by default).
  • -Fpattern is used to specify input field separator, much like awk's -F option.
  • -a turns on the autosplit mode, so input fields are placed into @F array.

A good mnemonic is perl -Fpattern -lane 'expression'. Example:

cat /etc/passwd | awk -F: '{ print $1 }'
cat /etc/passwd | perl -F: -lane 'print @F[0]'

Note that Perl fields are @F[0], @F[1], ...; awk fields are $1, $2, ... instead. However, awk $0 (the whole input line) corresponds to $_ in Perl.

If we want to combine regular expression matching and field separation, we might have something like:

find . | awk -F/ '/hw[0-9]+/ { print $1 }'
find . | perl -F/ -lane 'print @F[0] if /hw[0-9]+/'

Many awk variables have their Perl equivalents as well. However, in order to use them, the -MEnglish flag must be passed to Perl like this:

cat | awk '{ print NR, $0 }'
cat | perl -MEnglish -ne 'print $NR, " ", $_'

Most notably, the commas in the Perl print statement does not normally print out an output field separator. To get a behavior more like awk, do this:

cat | awk 'BEGIN { OFS = ": " } { print NR, $0 }'
cat | perl -MEnglish -ne 'BEGIN { $OFS = ": " } print $NR, $_'

Search and replace (in place)

perl -i -pe's/SEARCH/REPLACE/' filename

Edit of *.conf files changing all foo to bar (and backup original)

perl -p -i.orig -e 's/\bfoo\b/bar/g' *.conf

Prints the first 20 lines from *.conf files

perl -pe 'exit if $. > 20' *.conf

Search lines 10 to 20

perl -ne 'print if 10 .. 20' filename

Delete first 10 lines (and backup original)

perl -i.orig -ne 'print unless 1 .. 10' filename

Delete all but lines between foo and bar (and backup original)

perl -i.orig -ne 'print unless /^foo$/ .. /^bar$/' filename

Reduce multiple blank lines to a single line

perl -p -i -00pe0 filename

Convert tabs to spaces (1t = 2sp)

perl -p -i -e 's/\t/  /g' filename

Read input from a file and report number of lines and characters

perl -lne '$i++; $in += length($_); END { print "$i lines, $in characters"; }' filename