Editing data in a unix/linux pipeline by hand

April 12, 2014 6:55 PM

I am looking for a command line tool that will let me edit data in a shell pipeline by hand before sending it out again on STDOUT.

Essentially, I want to be able to do:


tail -n 1000 some_log_file | pipe_data_editor | sed '[some sed script]'


and be greeted with the output of tail in my favorite editor, where I can run some quick macros on it, manually remove some lines, etc, and have it go out again on STDOUT through sed. I'm imagining that it would simply save STDIN to a file in /tmp, pop open $VISUAL, and then spit the contents back on STDOUT when I saved and exited.

This seems like such an obvious utility that it has to exist, but it's really hard to google. I could write a shell script to do it, but it seems like there is probably a 25-year old unix tool that already does this and has ironed out all of the corner cases already.

posted by LiteOpera to Computers & Internet (3 answers total)

This probably can't exist because pipes will block. That is, the writer process will block at some point waiting for the reader to read more data, or the reader can block waiting for the writer to write. Sticking an editor in the middle sort of assumes that the writer will never block.

That being said....

cat /etc/passwd | vi -

does something interesting.

But I don't know how to make vi (actually vim) write to stdout.

Maybe,

:w! /proc/self/fd/1
posted by smcameron at 7:09 PM on April 12, 2014

Best answer: Good question! vipe from moreutils does this. Found a few more ideas on this Stackoverflow thread.
posted by evariste at 7:14 PM on April 12, 2014 [4 favorites]

Response by poster: Thank you evariste! There's some other neat ones there as well, especially "sponge" and "pee"!
posted by LiteOpera at 7:29 PM on April 12, 2014