All Shell Scripting Tips

30 Oct 2015

Checking the exit status of ANY command in a pipeline

It's a pretty common thing in a shell script to want to check the exit status of the previous command. You can do this with the $? variable, as is widely known:

#!/bin/bash
grep some.machine.example.com /etc/hosts
if [ "$?" -ne "0" ]; then
  # The grep command failed to find "some.machine.example.com" in /etc/hosts file
  echo "I don't know the IP address of some.machine.example.com"
  exit 2
fi

What gets difficult is when you execute a pipeline: (see pipelines for more information on the Unix Pipeline)

#!/bin/bash
grep some.machine.example.com /etc/hosts 2>&1 | tee /tmp/hosts-results.txt
if [ "$?" -ne "0" ]; then
  # Ah - what we get here is the status of the "tee" command, 
  # not the status of the "grep" command :-(

What you get is the result of the tee command, which writes the results to the display as well as to the /tmp/hosts-results.txt file.

To find out what grep returned, $? is of no use.

Instead, use the ${PIPESTATUS[]} array variable. ${PIPESTATUS[0]} tells us what grep returned, while ${PIPESTATUS[1]} tells us what tee returned.

So, to see what grep found, we can write our script like this:

#!/bin/bash
grep some.machine.example.com /etc/hosts 2>&1 | tee /tmp/hosts-results.txt
if [ "${PIPESTATUS[0]}" -ne "0" ]; then
  # The grep command failed to find "some.machine.example.com" in /etc/hosts file
  echo "I don't know the IP address of some.machine.example.com"
  exit 2
fi

Here's The Rub

The downside is, that any command you use to access ${PIPESTATUS[]}, will automatically replace the current state of the array with the return code of the command you have just run:

So as soon as we use echo to tell us about the return code of grep, the ${PIPESTATUS[]} array now tells us about the return code of the echo statement itself, which is pretty likely to be zero, as not much can cause echo to fail!

The Fix

Because ${PIPESTATUS[]} is a special variable, it changes all the time. However, we can copy this array into another array, which is just a regular array, which will not be changed at all just by running some more commands. Copying an array requires a slightly different syntax to simply copying contents of a variable into another:

RC=( "${PIPESTATUS[@]}" )

Where RC stands for Return Code. We can then investigate the status of RC at our leisure. For testing purposes, the program true always returns zero, and false always returns 1:

#!/bin/bash
echo "tftf"
true | false | true | false
RC=( "${PIPESTATUS[@]}" )
echo "RC[0] = ${RC[0]}"		# true = 0
echo "RC[1] = ${RC[1]}"		# false = 1
echo "RC[2] = ${RC[2]}"		# true = 0
echo "RC[3] = ${RC[3]}"		# false = 1

echo "ftft"
false | true | false | true
RC=( "${PIPESTATUS[@]}" )
echo "RC[0] = ${RC[0]}"		# false = 1
echo "RC[1] = ${RC[1]}"		# true = 0
echo "RC[2] = ${RC[2]}"		# false = 1
echo "RC[3] = ${RC[3]}"		# true = 0

echo "fftt"
false | false | true | true
RC=( "${PIPESTATUS[@]}" )
echo "RC[0] = ${RC[0]}"		# false = 1
echo "RC[1] = ${RC[1]}"		# false = 1
echo "RC[2] = ${RC[2]}"		# true = 0
echo "RC[3] = ${RC[3]}"		# true = 0

Download the tftf.sh script

You can use this for more nuanced checking of return codes, such as this combination of curl and grep.

#!/bin/bash
WEB_SERVER=steve-parker.org
curl -# -f -u ${USERNAME}:${PASSWORD} http://${WEB_SERVER}/ | grep "SomeMessage"
RC=( "${PIPESTATUS[@]}" )
if [ "${RC[0]}" -eq "22" ]; then
  # curl returned 22, indicating some error above 400,
  # such as: 404 Not Found, 401 Unauthorized, etc.
  echo "Invalid credentials"
  exit 1
fi

# PIPESTATUS has gone, but we can still inspect RC
if [ "${RC[1]}" -eq "0" ]; then
  echo "Grep succeeded"
  echo "Web server reported SomeMessage"
else
  echo "Web server didn't report SomeMessage"
fi

Download the curlgrep.sh script