URL Find and Replace
November 26, 2007 12:21 PM   Subscribe

I'm working on a quick Automator workflow and need some help replacing the local hostname in a series of URLs.

Here's the deal: the Automator workflow retrieves a list of return delimited URLs with random alphanumeric local hostnames (e.g., wrim881q.example.net, ghu2z.example.net, etc.). I need to pass these into a shell script and change them all to the same local hostname (e.g., foo.example.net) and then download all the URLs with curl.

posted by nathan_teske to Computers & Internet (3 answers total)
As long as you're running a shell script, you might as well run them through sed inside the script.

cmiller@zippy:~ $ sh t.sh http://ask.metafilter.com/77126/URL-Find-and-Replace
cmiller@zippy:~ $ cat t.sh
local_url=`echo "$inet_url" |sed -e "s=^\\([^:]*\\)://[^/]*\\(/.*\\)$=\1://$local_hostname\2="`

echo $local_url
posted by cmiller at 1:57 PM on November 26, 2007

cmiller I think he just wants to replace the hostname part of the FQDN. And OS X's "sed" is not GNU sed so to get extended regexes it's -E. nathan, just put this in the shell script action:

sed -E 's/^([^:]+:\/\/)[^\.]+/\1foo/'

And is there a reason you're calling curl rather than just using the "Download URLs" action? (It's under "Internet".)
posted by nicwolff at 2:49 PM on November 26, 2007

Thanks nicwolff - that worked perfectly. I need to run through curl as the server I'm hitting horks if you have more than 3-4 concurrent downloads.
posted by nathan_teske at 7:49 AM on November 27, 2007

« Older Image of Vault Boy Spoofing Ayn Rand?   |   help me finish this god forsaken paper Newer »
This thread is closed to new comments.