Batch-correct files?

June 29, 2004 12:34 PM

sed: How do I use it to do a general search/replace on a bunch of files? The docs seem to believe all I have to do is feed it s/regexp/replacetext/g and a list of files and I'm off to the races, but rather than getting the files changed... [exciting conclusion inside]

... rather than changing the files, it just spits all of the changed lines to stdout.

So... if file "one" contains:

the quick brown fox jumped over the lazy dog

and file "two" contains:

the grove of trees was interesting

Then: sed s/the/an/g *

gives me:

an quick brown fox jumped over an lazy dog
an ash grove

Which would be great if I wanted to concatenate all the results into a single file, but I don't. I want the changed results to sit in their source files. Without resulting to something like find . -exec sed s/the/an/gw{} {} \; -freak -I -dont -understand -unix.

Of course, I'd also like world peace and a pony.

posted by namespan to Computers & Internet (7 answers total)

perl -pi -e 's|foo|bar|g' *
posted by greasepig at 12:50 PM on June 29, 2004

Thanks greasepig! This alleviates the pressure of my practical needs ... however, I'm still curious about how sed is supposed to do what it does...
posted by namespan at 1:17 PM on June 29, 2004

Well, sed is doing what it's supposed to do, which is process a stream. It doesn't really think about files as such.

A simple way to do what you want with sed would be this:

mkdir ../processed
find . -exec sh -c "cat {} | sed -e 's/frank/howard/g' > ../processed/{}" \;


That will make a copy of all of the files to be processed in a directory called "processed." Having sed overwrite the files as its processing them isn't an option, so it has to write to somewhere different. You could make that a more complicated one-liner and have it rename the file first, have sed do its thing to a new file with the original name and delete the newly renamed version. Also, be sure not to have the copies of the files go into a directory that find will find, or it will recurse forever. (Or until your hard drive is full.)
posted by mragreeable at 1:30 PM on June 29, 2004

With gnu sed (which is what your average Linux machine has), you can just use the -i option:

sed -i 's/foo/bar/g' *
posted by sfenders at 3:36 PM on June 29, 2004

sfenders, that's awesome.

mragreeable.... out of curiousity, how come the cat/sed command is encased in the sh -c part? I understand the rest of the find command except for that part.
posted by namespan at 5:56 PM on June 29, 2004

Yeah, sfenders gets the award, I think.

As for the other question, find will run a single program when using -exec. If it sees something like "|" or ">" it won't interpret them for you, it will just pass them as parameters to the command it's running. So I use sh -c, and have bash (or whatever) interpret the shell symbols for me. When using that technique it's usually a good idea to put the {} symbol in single-quotes if there's a chance that the filenames will have spaces.

At least that's been my finding. If someone knows of a way to use pipes and stuff with find without using a similar technique, I'd love to hear about it.
posted by mragreeable at 8:16 PM on June 29, 2004

Using find, and pretending sed -i doesn't exist, I'd do it like this:

for i in `find . -type f` ; do cat $i | sed 's,xxx,yy,g' > .$i.tmp ; mv .$i.tmp $i; done
posted by sfenders at 9:57 PM on June 29, 2004