Unix text-editing command question

May 9, 2004 1:53 PM

Unix text-editing command question:

I have a file that is automatically generated every day at midnight. For simplicity, I'll call it netflix.html, cause that's its name. The autogenerated file contains, among other things, long URLs, and in some of the URLs is '&lnkctr=RecListStar'. This phrase causes the URLs to break, and I would like to run a cron command at 12:05 or so to go through netflix.html and strip out all occurences of that bit of text, leaving everything else untouched.

I know this must be on Google, and in my man pages, but I just am getting lost and confused when I look. Administrators! Help me!

to clarify, I know how to do the cron part of it.

It's the command itself I cannot figure out.

posted by John Kenneth Fisher to Computers & Internet (16 answers total)

$ cat foohttp://foo.bar?foo=bar&lnkctr=RecListStar&a=b$ sed 's/&lnkctr=RecListStar//g' foohttp://foo.bar?foo=bar&a=b

is that what you want?
posted by andrew cooke at 2:27 PM on May 9, 2004

Not sure I totally understand, but if it's breaking on the "&" character (ie: the shell is interpreting it as an operator -- not a character), couldn't you swap it out for the & html entity?
posted by RavinDave at 3:03 PM on May 9, 2004

... um ... enclosing the URL in " marks.

(I'm assuming it's breaking when you're launching lynx or something similar.)
posted by RavinDave at 3:06 PM on May 9, 2004

Response by poster: Ravin: oh, no, it isn't anything to do with that. It's kinda hard to explain. It's a matter of it technically being right, but me preferring what you get without that part.

I'll go check andrew's suggestion...
posted by John Kenneth Fisher at 3:14 PM on May 9, 2004

Response by poster: andrew cooke:

Thanks, that's doing what I want, yes, but its outputting it to the screen that way, instead of writing it back to netflix.html

I tried doing sed 's/&lnkctr=RecListStar//g' netflix.html >> netflix.html , but it didn't work...
posted by John Kenneth Fisher at 3:29 PM on May 9, 2004

rm -f foo~ ; mv foo foo~ ; sed 's/&lnkctr=RecListStar//g' foo~ > foo

posted by andrew cooke at 3:37 PM on May 9, 2004

or add -i to sed (apparently - just found it in the man page)
posted by andrew cooke at 3:38 PM on May 9, 2004

Best answer: yep, works:

cat foohttp://foo.bar?foo=bar&lnkctr=RecListStar&a=b$ sed -i 's/&lnkctr=RecListStar//g' foo$ cat foohttp://foo.bar?foo=bar&a=b

posted by andrew cooke at 3:40 PM on May 9, 2004

Response by poster: Perfect! Thanks a bunch, truly. Been struggling with that off and on for a few days.
posted by John Kenneth Fisher at 3:46 PM on May 9, 2004

you need this
posted by andrew cooke at 4:06 PM on May 9, 2004

obligatory perl:
perl -wpi -e 's/&lnkctr=RecListStar//g' netflix.html
posted by duckstab at 4:15 PM on May 9, 2004

I was going to say, the Perl would work better.
posted by SpecialK at 8:05 PM on May 9, 2004

How would Perl work better in this situation?
posted by Galvatron at 9:23 PM on May 9, 2004

How would Perl work better in this situation?

Perl advocates like to remind us that, while hard-to-read and confusing, at least it's nicer than awk, and that's got to be worth _something_.
posted by Space Coyote at 11:00 PM on May 9, 2004

Yeah, maybe. Perl is clearly better suited to a wide variety of tasks, but I'm lost as to how the Perl one-liner in this thread is superior to a functionally equivalent sed one-liner.
posted by Galvatron at 12:22 AM on May 10, 2004

"the right tool for the job" is not part of the perl mindset.
posted by andrew cooke at 6:17 AM on May 10, 2004