Batch-correct files?

June 29, 2004 12:34 PM

sed: How do I use it to do a general search/replace on a bunch of files? The docs seem to believe all I have to do is feed it s/regexp/replacetext/g and a list of files and I'm off to the races, but rather than getting the files changed... [exciting conclusion inside]

... rather than changing the files, it just spits all of the changed lines to stdout.

So... if file "one" contains:

the quick brown fox jumped over the lazy dog

and file "two" contains:

the grove of trees was interesting

Then: sed s/the/an/g *

gives me:

an quick brown fox jumped over an lazy dog
an ash grove

Which would be great if I wanted to concatenate all the results into a single file, but I don't. I want the changed results to sit in their source files. Without resulting to something like find . -exec sed s/the/an/gw{} {} \; -freak -I -dont -understand -unix.

Of course, I'd also like world peace and a pony.

posted by namespan to Computers & Internet (7 answers total)

perl -pi -e 's|foo|bar|g' *
posted by greasepig at 12:50 PM on June 29, 2004

Response by poster: Thanks greasepig! This alleviates the pressure of my practical needs ... however, I'm still curious about how sed is supposed to do what it does...
posted by namespan at 1:17 PM on June 29, 2004

Well, sed is doing what it's supposed to do, which is process a stream. It doesn't really think about files as such.

A simple way to do what you want with sed would be this:

mkdir ../processed
find . -exec sh -c "cat {} | sed -e 's/frank/howard/g' > ../processed/{}" \;


That will make a copy of all of the files to be processed in a directory called "processed." Having sed overwrite the files as its processing them isn't an option, so it has to write to somewhere different. You could make that a more complicated one-liner and have it rename the file first, have sed do its thing to a new file with the original name and delete the newly renamed version. Also, be sure not to have the copies of the files go into a directory that find will find, or it will recurse forever. (Or until your hard drive is full.)
posted by mragreeable at 1:30 PM on June 29, 2004

With gnu sed (which is what your average Linux machine has), you can just use the -i option:

sed -i 's/foo/bar/g' *
posted by sfenders at 3:36 PM on June 29, 2004

Response by poster: sfenders, that's awesome.

mragreeable.... out of curiousity, how come the cat/sed command is encased in the sh -c part? I understand the rest of the find command except for that part.
posted by namespan at 5:56 PM on June 29, 2004

Yeah, sfenders gets the award, I think.

As for the other question, find will run a single program when using -exec. If it sees something like "|" or ">" it won't interpret them for you, it will just pass them as parameters to the command it's running. So I use sh -c, and have bash (or whatever) interpret the shell symbols for me. When using that technique it's usually a good idea to put the {} symbol in single-quotes if there's a chance that the filenames will have spaces.

At least that's been my finding. If someone knows of a way to use pipes and stuff with find without using a similar technique, I'd love to hear about it.
posted by mragreeable at 8:16 PM on June 29, 2004

Using find, and pretending sed -i doesn't exist, I'd do it like this:

for i in `find . -type f` ; do cat $i | sed 's,xxx,yy,g' > .$i.tmp ; mv .$i.tmp $i; done
posted by sfenders at 9:57 PM on June 29, 2004