No more shell games!

May 31, 2006 5:02 PM

*nix shell "best practices" anyone?

Over the years I've been writing more and more shell scripts (bash, ksh, perl, expect) and they've been getting more and more complexe. I haven't been able to find a comprehensive list of best practices for shell programming although I have learned a lot by looking at other people's scripts (thank you sanity checks!) I'm looking for links or your own best practices.

Here's an example:

Use ${FOO} instead of $FOO so that the variable is more likely to be interpreted when concatenated in a string and use >> "${FOO}" instead of >> ${FOO} in case ${FOO} has spaces in it.

posted by furtive to computers & internet (9 comments total)

Have you read Kernighan and Pike's The UNIX Programming Environment?
posted by russilwvong at 5:11 PM on May 31, 2006

No. But it looks elite. Thanks.
posted by furtive at 5:20 PM on May 31, 2006

set -o noclobber if you're in bash. We all fuck up from time to time.

Unix Power Tools by O'Reilly taught me a great deal.
posted by NucleophilicAttack at 6:57 PM on May 31, 2006

Oh right: alias rm="rm -i."

Touch a file called -i in /.

That kind of thing.
posted by NucleophilicAttack at 6:57 PM on May 31, 2006

If you're interested in writing portable scripts, then there's no better reference than Chapter 10 of the autoconf manual.

Some of my pet peeves:

Always test your scripts with path and filenames that contain spaces. You might not use them, but with operating systems like OS X becoming popular, many people are combining "friendly" filenames with unix shell concepts.

This includes quoting arguments (e.g. cd "$var") and always using the "-print0 | xargs -0" paradigm when processing the results of 'find'. In fact, you should almost always use this in favor of that awful '-exec foo {}' junk. It can run multiple files per command, for example, which means your script will be faster on systems were fork/exec is slow.

Though it is not as portable to ancient shells, $(foo) is much easier to code than `foo` because it allows for nested quoting with far fewer headaches. Example: echo "$(echo "one $(echo "two $(echo "three")")")"

Use functions instead of aliases. They are much more powerful and a whole heck of a lot less confusing. In fact, function use is a very good concept to use as much as possible, again unless you are trying to write very portable scripts (though it's only extremely ancient shells that don't support functions.)

Learn the power of "set -e" for doing something that has multiple steps. Also learn the power of "set -x" for debugging. Along those lines, also consider bashdb for those hard to tackle bugs.
posted by Rhomboid at 7:38 PM on May 31, 2006

And of course the normal stuff: bash FAQ, Advanced bash scripting guide. Do be aware that not everyone's /bin/sh is bash, though, so if you use bash specific features set your shebang to /bin/bash not /bin/sh.
posted by Rhomboid at 7:46 PM on May 31, 2006

1. I usually start scripts out in perl, unless I know that they will primarily involve calling other binaries
2. If a perl script gets close to 1000 lines, I seriously consider reimplementing in a more maintainable language
3. CVS, svn, or some other source code control system will save many headaches.
posted by b1tr0t at 8:46 PM on May 31, 2006

Good way to test is

# if [ x"${FOO}" = x"" ]; then

.. should handle FOO with spaces, or empty.

Things I learnt recently that I wish someone had shown me a few years back, the "set" thing in sh.

Like if you get:
"bit1 bit2 other items etc" in $FOO, say, output from ls:

# set -- $FOO
# echo "$3 and $5"

Would print "other and etc". Used to be always awk/cut mess.

Ctrl-R in bash is beautiful, works like Ctrl-R in emacs.
posted by lundman at 10:21 PM on May 31, 2006 [1 favorite has favorites]

Using the shell's inbuilt string manipulation facilities instead of building little pipelines involving grep and/or sed can often make scripts run much more quickly. Check the bash reference manual under "parameter expansion" to find out what kinds of stuff can be done.
posted by flabdablet at 10:52 PM on May 31, 2006