Comments on: Pipe command output, but keep the error code

Question: Pipe command output, but keep the error code

ChrisR — Sat, 24 Nov 2007 14:55:15 -0800

How do I get the correct return code from a unix command line application after I've piped it through another command that succeeded?

In detail, here's the situation writ small:

# mycmd_that_fails | postprocesor_that_succeeds

# echo $?

0

And, what I'd like to see is:

# mycmd_that_fails | postprocesor_that_succeeds

# echo $?

1

The obvious solution was to capture the output of mycmd_that_fails into a file and then pipe it through the postprocesor_that_succeeds, but since my command is a build script it's nice to see the progress as it runs. I'd rather have the postprocessor operate on the command as it runs than after the fact.

Does anyone know how to accomplish this?

By: freebird

freebird — Sat, 24 Nov 2007 15:10:56 -0800

You could maybe slip a little perl (or whatever) script between them to save off the return code for later use?

mycmd | perl_thingy | postprocessor

Where the perl_thingy could stick $? in a file, then postprocessor could read that file. More or less complicated, depending on what you're actually doing and your personal tastes, would be to have second perl script that would accept the postprocessor's output, read the stashed result from the first, and generate whatever kind of summary you like.

By: Steven C. Den Beste

Steven C. Den Beste — Sat, 24 Nov 2007 15:31:57 -0800

IIRC, this:

mycmd_that_fails 2&> /path/erroroutputfile | postprocesor_that_succeeds

Without that addition, all the outputs are merged and fed to the pipe. What that does is to separate out the error flow from the normal flow, and redirect the error flow to the file.

But this only allows you to see the error after the fact.

By: Steven C. Den Beste

Steven C. Den Beste — Sat, 24 Nov 2007 15:32:56 -0800

I think I botched that syntax. "two ampersand into" isn't right, but I'm not sure what is.

By: Steven C. Den Beste

Steven C. Den Beste — Sat, 24 Nov 2007 15:34:04 -0800

...what I shoulda said was...

Also, that redirect can go to the console device instead of to a file if you use the proper "/dev/something" path.

By: flabdablet

flabdablet — Sat, 24 Nov 2007 15:34:32 -0800

You pretty much have to save the first command's result in a file, since each command in the pipeline is going to run inside its own environment. So just have my_cmd_that_fails echo $? into a file at some appropriate point, and pick it up later.

By: grouse

grouse — Sat, 24 Nov 2007 15:37:31 -0800

Use ${PIPESTATUS[*]} in bash. I have something like this in my .bashrc, which gives me the exit status of every command in the last pipe I just ran, if anything is not 0:

export PROMPT_COMMAND="
       _RES=\${PIPESTATUS[*]};
       _RES_STR='';
       for res in \$_RES; do
         if [[ ( \$res > 0 ) ]]; then
           _RES_STR=\" [\$_RES]\";
         fi;
       done"

export PS1="\n\u@\h \w\$_RES_STR\n\\$ "

Try it, you'll like it.

The actual code puts things in color if you're using a terminal that supports that. But it's somewhat more complicated and therefore harder to understand. I can provide it if you want.

By: puddpunk

puddpunk — Sat, 24 Nov 2007 15:40:00 -0800

Have you tried this:

# my_cmd_that_fails; RET_VAL=$? | postprocessor_that_succeds; echo $RET_VAL

You can then access $RETURN within that command. If you need to access it later use the export command:

# my_cmd_that_fails; export RET_VAL=$? | postprocessor_that_succeeds
# echo $RET_VAL

By: flabdablet

flabdablet — Sat, 24 Nov 2007 15:40:08 -0800

SCDB is apparently a little confused about the behaviour of standard output and standard error. Only standard output file descriptor 1) is redirected down a pipeline by default. Standard error (file descriptor 2) is not merged with standard output. That's kind of the point of standard error.

If you want them merged, you have to include the redirect 2>&1 on the command that generates them.

In any case, this question is about the error return code ($?) rather than the error output stream.

By: puddpunk

puddpunk — Sat, 24 Nov 2007 15:40:41 -0800

Typo, I mean "You can then access $RET_VAL within that command..."

By: flabdablet

flabdablet — Sat, 24 Nov 2007 15:44:34 -0800

puddpunk, I don't think that will work, since each command in the pipeline gets its own environment; RET_VAL won't be the same variable on both sides of the pipe.

The PIPESTATUS array built into bash is a partial workaround for this, but in the general case, it's easier to save $? into a file at some appropriate point, and pick it up later.

By: jepler

jepler — Sat, 24 Nov 2007 15:46:12 -0800

PIPESTATUS is the way to go, so long as you can use bash and not any other shell. Here's a handy script that can filter the output of a program while preserving its exit value:

#!/bin/bash

PAT=$1

shift

"$@" 2>&1 | LANG=C egrep -ve "$PAT"

exit ${PIPESTATUS[0]}

Example usage:

filterout "Uninitialized" gcc dodgy.c

By: jepler

jepler — Sat, 24 Nov 2007 15:51:12 -0800

It looks like there's a solution for ksh, but it's ugly. more variations for various shells

By: mbrubeck

mbrubeck — Sat, 24 Nov 2007 15:56:43 -0800

The mispipe command from the moreutils package will do this. Just 'aptitude install moreutils' on Debian or Ubuntu or similar.

By: foodgeek

foodgeek — Sat, 24 Nov 2007 15:57:50 -0800

In bash and others you can do:

# mycmd && postprocessor

then you can just echo $? to see the output from the failed command since the post processor will only be run if the first command runs cleanly. Obviously this is useless if you want the post processor to run even if your command fails...

By: mbrubeck

mbrubeck — Sat, 24 Nov 2007 16:30:11 -0800

foodgeek, that doesn't pipe the output from mycmd into postprocessor.

By: rhizome

rhizome — Sat, 24 Nov 2007 17:28:35 -0800

grouse and jepler have the way. "man bash" and search for PIPESTATUS ("/PIPESTATUS" while in the manpage, with slash and without quotes).

By: foodgeek

foodgeek — Sat, 24 Nov 2007 17:52:30 -0800

mbrubeck, true 'nuf - but aside from it not working, it's a perfect solution! :)

By: philomathoholic

philomathoholic — Sat, 24 Nov 2007 20:31:01 -0800

Here's a way to do it by making your own (named) pipe.



$ mkfifo pipe  #make a pipe, named "pipe"

$ postprocesor_that_succeeds < pipe &  #the call to read from pipe will block until a program writes to pipe

$ mycmd_that_fails > pipe

I hope the above is understandable, I included comments (which start at the # and extend to the end of the line, they are safe to put in the command line).

To make sure this worked, I did this:



$ mkfifo pipe

$ cat < pipe &

$ bash  #start a new shell, just for the exit value

$ exit 2 > pipe  #exit out of new shell

$ echo $?  #back in first shell

2

$

By: philomathoholic

philomathoholic — Sat, 24 Nov 2007 20:37:33 -0800

Really though, just install moreutils and use mispipe (like mrbrubeck suggests). It's the non-hacked-together way to do it.

By: ChrisR

ChrisR — Sun, 25 Nov 2007 08:44:14 -0800

philomathoholic:

That sounds promising, but I'll have to experiment to ensure that it works under older Solaris _and_ on cygwin. That's a nice, tidy soultion, so I hope it works.

puddpunk: I tried yours as well, and it works well enough, so that might be the short-term solution.

grouse: You've piqued my curiosity; The only question: What are the indices in the PIPESTATUS array? The order of commands in the last pipe?

By: grouse

grouse — Sun, 25 Nov 2007 09:02:56 -0800

What are the indices in the PIPESTATUS array? The order of commands in the last pipe?

Yes.

Are you using bash? If so I think it is a no-brainer to use the built-in $PIPESTATUS array for what it was designed for, rather than hacky solutions or requiring the installation of another package.

By: ChrisR

ChrisR — Sun, 25 Nov 2007 10:29:15 -0800

I'll have to check to see if I am able to do so on all of our build boxes; we've got some older architectures that are... bash-unfriendly.

I hope to be able to; I implemented it on the Linux architectures and it works like a charm.