Comments on: Batch-correct files?

Question: Batch-correct files?

namespan — Tue, 29 Jun 2004 12:34:05 -0800

sed: How do I use it to do a general search/replace on a bunch of files? The docs seem to believe all I have to do is feed it s/regexp/replacetext/g and a list of files and I'm off to the races, but rather than getting the files changed... [exciting conclusion inside]

... rather than changing the files, it just spits all of the changed lines to stdout.

So... if file "one" contains:

the quick brown fox jumped over the lazy dog

and file "two" contains:

the grove of trees was interesting

Then: sed s/the/an/g *

gives me:

an quick brown fox jumped over an lazy dog
an ash grove

Which would be great if I wanted to concatenate all the results into a single file, but I don't. I want the changed results to sit in their source files. Without resulting to something like find . -exec sed s/the/an/gw{} {} \; -freak -I -dont -understand -unix.

Of course, I'd also like world peace and a pony.

By: greasepig

greasepig — Tue, 29 Jun 2004 12:50:19 -0800

perl -pi -e 's|foo|bar|g' *

By: namespan

namespan — Tue, 29 Jun 2004 13:17:59 -0800

Thanks greasepig! This alleviates the pressure of my practical needs ... however, I'm still curious about how sed is supposed to do what it does...

By: mragreeable

mragreeable — Tue, 29 Jun 2004 13:30:49 -0800

Well, sed is doing what it's supposed to do, which is process a stream. It doesn't really think about files as such.

A simple way to do what you want with sed would be this:

mkdir ../processed

find . -exec sh -c "cat {} | sed -e 's/frank/howard/g' > ../processed/{}" \;

That will make a copy of all of the files to be processed in a directory called "processed." Having sed overwrite the files as its processing them isn't an option, so it has to write to somewhere different. You could make that a more complicated one-liner and have it rename the file first, have sed do its thing to a new file with the original name and delete the newly renamed version. Also, be sure not to have the copies of the files go into a directory that find will find, or it will recurse forever. (Or until your hard drive is full.)

By: sfenders

sfenders — Tue, 29 Jun 2004 15:36:51 -0800

With gnu sed (which is what your average Linux machine has), you can just use the -i option:

sed -i 's/foo/bar/g' *

By: namespan

namespan — Tue, 29 Jun 2004 17:56:08 -0800

sfenders, that's awesome.

mragreeable.... out of curiousity, how come the cat/sed command is encased in the sh -c part? I understand the rest of the find command except for that part.

By: mragreeable

mragreeable — Tue, 29 Jun 2004 20:16:54 -0800

Yeah, sfenders gets the award, I think.

As for the other question, find will run a single program when using -exec. If it sees something like "|" or ">" it won't interpret them for you, it will just pass them as parameters to the command it's running. So I use sh -c, and have bash (or whatever) interpret the shell symbols for me. When using that technique it's usually a good idea to put the {} symbol in single-quotes if there's a chance that the filenames will have spaces.

At least that's been my finding. If someone knows of a way to use pipes and stuff with find without using a similar technique, I'd love to hear about it.

By: sfenders

sfenders — Tue, 29 Jun 2004 21:57:18 -0800

Using find, and pretending sed -i doesn't exist, I'd do it like this:

for i in `find . -type f` ; do cat $i | sed 's,xxx,yy,g' > .$i.tmp ; mv .$i.tmp $i; done