bash process substitution - ghdrako/doc_snipets GitHub Wiki

Treat process output as a file so we dont't need use temp files

diff <(ls) <(ls -a)

Process Substitution operator, which comes in two forms: <(cmd) and >(cmd). It's a convenient way to get the benefits of temporary files or named pipes without having to create them yourself. Whenever you think you need a temporary file to do something, process substitution might be a better way to handle it.

The >() form is relatively rare, and we won't cover it here, because it'll just confuse things. Once you understand how <() works, >() just does the same thing in reverse (writing instead of reading).

What <() does, is run the command inside the parentheses, and gives you a temporary filename that you can use to read the command's output. The advantage over pipes is that you can use the filename as an argument to a command that expects to see a filename.

For example, let's say you have two files that you'd like to diff, but they aren't sorted yet. You could generate two temporary files, to hold the sorted versions of your original files, and then run diff on those. Or, you could use process substitutions:

$ diff <(sort file1) <(sort file2)

With the <() operator, the command's output is sent through a named pipe (or something similar) that's created by bash. The operator itself in your command is replaced by the filename of that file. After your whole command finishes, the file is cleaned up.

The same thing can be done with any commands whose output you'd like to pass to diff. Imagine you want to see the difference between the output of two commands. Ordinarily, you'd have to put the two outputs in two files and diff those:

$ head -n 1 .dictionary > file1
$ tail -n 1 .dictionary > file2
$ diff -y file1 file2
Aachen                                                        | zymurgy
$ rm file1 file2

Using the process substitution operator, we can do all that with a one-liner and no need for manual cleanup:

$ diff -y <(head -n 1 .dictionary) <(tail -n 1 .dictionary)
Aachen                                                        | zymurgy

The <(..) part is replaced by the temporary FIFO created by bash, so diff actually sees something like this:

$ diff -y /dev/fd/63 /dev/fd/62

Here we see how bash runs diff when we use process substitution. It runs our head and tail commands, sending their respective outputs through the "files" /dev/fd/63 and /dev/fd/62. Then it runs the diff command, passing those filenames where originally we had put the process substitution operators.

The actual implementation differs from system to system. In fact, you can see what the above would actually look like to diff on your box by putting an echo in front of our command:

box1$ echo diff -y <(head -n 1 .dictionary) <(tail -n 1 .dictionary)
diff -y /dev/fd/63 /dev/fd/62

box2$ echo <(cat /dev/null)
/var/tmp//sh-np-605454726

For examples using the >() form, see ProcessSubstitution and Bash FAQ 106.

Good Practice: Process Substitution gives you a concise way to create temporary FIFOs automatically. They're less flexible than creating your own named pipes by hand, but they're perfect for common short commands like diff that need filenames for their input sources.