Learning Linux Commands: awk - JohnHau/mis GitHub Wiki

In the case of this article, the Learning Linux Commands: awk title might be a little misleading. And that is because awk is more than a command, it’s a programming language in its own right. You can write awk scripts for complex operations or you can use awk from the command line. The name stands for Aho, Weinberger and Kernighan (yes, Brian Kernighan), the authors of the language, which was started in 1977, hence it shares the same Unix spirit as the other classic *nix utilities.

If you’re getting used to C programming or know it already, you will see some familiar concepts in awk, especially since the ‘k’ in awk stands for the same person as the ‘k’ in K&R, the C programming bible. You will need some command-line knowledge in Linux and possibly some scripting basics, but the last part is optional, as we will try to offer something for everybody. Many thanks to Arnold Robbins for all his work involved in awk.

In this tutorial you will learn:

What does awk do? How does it work? awk basic concepts Learn to use awk through command line examples

What is it that awk does?

awk is a utility/language designed for data extraction. If the word “extraction” rings a bell, it should because awk was once Larry Wall’s inspiration when he created Perl. awk is often used with sed to perform useful and practical text manipulation chores, and it depends on the task if you should use awk or Perl, but also on personal preference. Just as sed, awk reads one line at a time, performs some action depending on the condition you give it and outputs the result.

One of the most simple and popular uses of awk is selecting a column from a text file or other command’s output. One thing I used to do with awk was, if I installed Debian on my second workstation, to get a list of the installed software from my primary box then feed it to aptitude. For that, I did something like this:

$ dpkg -l | awk ' {print $2} ' > installed Most package managers today offer this facility, for example rpm’s -qa options, but the output is more than I want. I see that the second column of dpkg -l‘s output contains the name of the packages installed, so this is why I used $2 with awk: to get me only the 2nd column.

Basic concepts As you have noticed, the action to be performed by awk is enclosed in braces, and the whole command is quoted. But the syntax is awk ' condition { action }'. In our example, we had no condition, but if we wanted to, say, check only for vim-related packages installed (yes, there is grep, but this is an example, plus why use two utilities when you can only use one?), we would have done this:

$ dpkg -l | awk ' /'vim'/ {print $2} ' This command would print all packages installed that have “vim” in their names. One thing about awk is that it’s fast. If you replace “vim” with “lib”, on my system that yields 1300 packages. There will be situations where the data you’ll have to work with will be much bigger, and that’s one part where awk shines.

Anyway, let’s start with the examples, and we will explain some concepts as we go. But before that, it would be good to know that there are several awk dialects and implementations, and the examples presented here deal with GNU awk, as an implementation and dialect. And because of various quoting issues, we assume you’re using bash, ksh or sh, we don’t support (t)csh.

awk command examples See some of the examples below to gain an understanding of awk and how you can apply it in situations on your own system. Feel free to follow along and use some of these commands in your terminal to see the output you get back.

Conclusion As you can see, with awk you can do lots of text processing and other nifty stuff. We didn’t get into more advanced topics, like awk‘s predefined functions, but we showed you enough (we hope) to start remembering it as a powerful tool.