Comments on: No more shell games!

Question: No more shell games!

furtive — Wed, 31 May 2006 17:02:01 -0800

*nix shell "best practices" anyone?

Over the years I've been writing more and more shell scripts (bash, ksh, perl, expect) and they've been getting more and more complexe. I haven't been able to find a comprehensive list of best practices for shell programming although I have learned a lot by looking at other people's scripts (thank you sanity checks!) I'm looking for links or your own best practices.

Here's an example:

Use ${FOO} instead of $FOO so that the variable is more likely to be interpreted when concatenated in a string and use >> "${FOO}" instead of >> ${FOO} in case ${FOO} has spaces in it.

By: russilwvong

russilwvong — Wed, 31 May 2006 17:11:36 -0800

Have you read Kernighan and Pike's The UNIX Programming Environment?

By: furtive

furtive — Wed, 31 May 2006 17:20:55 -0800

No. But it looks elite. Thanks.

By: NucleophilicAttack

NucleophilicAttack — Wed, 31 May 2006 18:57:03 -0800

set -o noclobber if you're in bash. We all fuck up from time to time.

Unix Power Tools by O'Reilly taught me a great deal.

By: NucleophilicAttack

NucleophilicAttack — Wed, 31 May 2006 18:57:42 -0800

Oh right: alias rm="rm -i."

Touch a file called -i in /.

That kind of thing.

By: Rhomboid

Rhomboid — Wed, 31 May 2006 19:38:56 -0800

If you're interested in writing portable scripts, then there's no better reference than Chapter 10 of the autoconf manual.

Some of my pet peeves:

Always test your scripts with path and filenames that contain spaces. You might not use them, but with operating systems like OS X becoming popular, many people are combining "friendly" filenames with unix shell concepts.

This includes quoting arguments (e.g. cd "$var") and always using the "-print0 | xargs -0" paradigm when processing the results of 'find'. In fact, you should almost always use this in favor of that awful '-exec foo {}' junk. It can run multiple files per command, for example, which means your script will be faster on systems were fork/exec is slow.

Though it is not as portable to ancient shells, $(foo) is much easier to code than `foo` because it allows for nested quoting with far fewer headaches. Example: echo "$(echo "one $(echo "two $(echo "three")")")"

Use functions instead of aliases. They are much more powerful and a whole heck of a lot less confusing. In fact, function use is a very good concept to use as much as possible, again unless you are trying to write very portable scripts (though it's only extremely ancient shells that don't support functions.)

Learn the power of "set -e" for doing something that has multiple steps. Also learn the power of "set -x" for debugging. Along those lines, also consider bashdb for those hard to tackle bugs.

By: Rhomboid

Rhomboid — Wed, 31 May 2006 19:46:31 -0800

And of course the normal stuff: bash FAQ, Advanced bash scripting guide. Do be aware that not everyone's /bin/sh is bash, though, so if you use bash specific features set your shebang to /bin/bash not /bin/sh.

By: b1tr0t

b1tr0t — Wed, 31 May 2006 20:46:52 -0800

1. I usually start scripts out in perl, unless I know that they will primarily involve calling other binaries
2. If a perl script gets close to 1000 lines, I seriously consider reimplementing in a more maintainable language
3. CVS, svn, or some other source code control system will save many headaches.

By: lundman

lundman — Wed, 31 May 2006 22:21:06 -0800

Good way to test is

# if [ x"${FOO}" = x"" ]; then

.. should handle FOO with spaces, or empty.

Things I learnt recently that I wish someone had shown me a few years back, the "set" thing in sh.

Like if you get:
"bit1 bit2 other items etc" in $FOO, say, output from ls:

# set -- $FOO
# echo "$3 and $5"

Would print "other and etc". Used to be always awk/cut mess.

Ctrl-R in bash is beautiful, works like Ctrl-R in emacs.

By: flabdablet

flabdablet — Wed, 31 May 2006 22:52:26 -0800

Using the shell's inbuilt string manipulation facilities instead of building little pipelines involving grep and/or sed can often make scripts run much more quickly. Check the bash reference manual under "parameter expansion" to find out what kinds of stuff can be done.