When a command is executed, the operating system needs to work out how to execute it. It may be a built-in command (such as cd, or it may need to search the filesystem (using the $PATH variable to tell it where to look) for files which may provide the appropriate commands.

Depending on the operating system, it may or may not consider the file extension (.exe, .com, etc) but if the filename matches, it will then go and look at the first few bytes of the file contents. These are known as the "magic" bytes.

• In Microsoft's DOS, .exe files all begin with "MZ". If DOS saw those two characters, it would run it as an exe program.
• For an ELF binary (a common Unix/Linux format), the file will start with the Hexadecimal characters 7f 45 4c 46 - that's a 0x7f followed by the hex representation of the text "ELF". The kernel will know to execute this binary file directly.
• For shell scripts and other interpreted languages, the magic characters are "#!". If these are followed by the path to an executable, that will be used to translate the rest of the script.

So if your script starts with "#!/bin/bash", the content of your script will be passed to the /bin/bash program to execute. Because the shell treats lines starting with "#" as a comment, it doesn't matter that ""#!/bin/bash" is not valid bash syntax; Bash simply ignores that line.

Similarly, if your code starts with "#!/usr/bin/perl", then the Perl binary will process the script. Perl also ignores lines starting with "#", as does Python (#!/usr/bin/python), and most other interpreted languages. This is a handy way for the kernel to be told which program to use to execute the script, in a way in which the script does not need to deal with that detail.

Adding switches

You can also add switches to the interpreter. For example, "sh -x" will display each line before it processes it, or "sh -e" will exit if any command fails. So if your script starts with the line "#!/bin/bash -xe", then it will display each line, execute it, and abort as soon as any command in the script reports an error.

#!/usr/bin/env

It is not uncommon for people to advocate using #!/usr/bin/env foo as a way for a script to say "call /usr/bin/env and get it to work out where to find the foo interpreter."

That deals with the fact that your preferred Python installation may be in /home/fred/python/3.2.1, and you have set that in your $PATH, and such a Python script will call your preferred Python installation.

The first obvious flaw is that if you are trying to deal with the possibility of not knowing exactly where the interpreter (bash, perl, python, etc) might be, then you surely also can't be sure that the env binary is in /usr/bin.

The second problem is that the coder now has absolutely no control over what interpreter will execute the script - if the user has a Perl interpreter in $HOME/bin/sh, and PATH=$HOME/bin/:$PATH, then Perl will try to execute your shell script, if you prefaced it with "#/usr/bin/env sh".

This may be an entertaining topic for some; I am not convinced. Feel free to try to convince me (contact details below) but I may not respond.

Further Reading

https://en.wikipedia.org/wiki/Shebang_(Unix) has a reasonably decent explanation of the #! convention.

Get this tutorial as a PDF for only $5