All Shell Scripting Tips

7 May 2016

Brian Kernighan, one of the co-creators of Unix, on the Unix Pipeline

The Pipeline is one of the defining features of the Unix shell, and obviously, the same goes for Linux, MacOSX, and any other Unix-based or inspired systems.

In a nutshell, it allows you to tie the output of one program to the input of another. That may not seem such a radical idea, but the immediate benefit that it brings is that you don't have to save the results of one program before you can start processing it with another. The longer-term, and even more important benefit, is that it encourages programs to be small, simple, and to do one thing well. There is no need for every program to include a word-counter, if they can all be piped into wc. Similarly, no program needs to offer its own built-in pattern matching facilities, as it can be piped into grep. These simple concepts are what has made the Unix approach still massively important in today's world, more than 40 years after its invention.

This is another example of the Unix concept that "everything is a file". The output from one command is put into the pipeline, and the receiving command treats it just as it treats input from a file. If you run "ls | grep foo", the result is identical to if you ran "ls > /tmp/my-temp-file" followed by "grep foo /tmp/my-temp-file" (and, of course, remembered to "rm /tmp/my-temp-file" afterwards).

Anyway, over to Brian Kernighan to explain it all:

Learn another neat trick about pipelines with the pipestatus tip.