Is there an easy way to search through a range of web pages to find which ones are active?
November 2, 2004 11:05 AM Subscribe
Is there an easy way to search through a range of web pages to find which ones are active? [more inside]
I'm trying to find a job posting that is no longer openly posted at a web site (the closing date was several months ago). The url for the job posting is in the format:
http://anyplace.com/aa-1234.cfm
Is there a quick and simple way for me to check pages /aa-1000 through /aa-9999, without having to type each one?
(I did do a Google search, and while the posting I was looking for didn't come up, it did hit several pages that no longer appear on the general "Jobs Available" page.)
I'm trying to find a job posting that is no longer openly posted at a web site (the closing date was several months ago). The url for the job posting is in the format:
http://anyplace.com/aa-1234.cfm
Is there a quick and simple way for me to check pages /aa-1000 through /aa-9999, without having to type each one?
(I did do a Google search, and while the posting I was looking for didn't come up, it did hit several pages that no longer appear on the general "Jobs Available" page.)
Nah, one line.
curl -O 'http://anyplace.com/aa-[1000-9999].cfm'
Of course that or any other brute-force solution would hammer the site with nine thousand separate page requests -- not a very nice thing to do. And you'd still have to search through the results to find whatever it is you're looking for.
Can you contact the site administrators and ask for whatever you're looking for, instead?
posted by ook at 11:57 AM on November 2, 2004
curl -O 'http://anyplace.com/aa-[1000-9999].cfm'
Of course that or any other brute-force solution would hammer the site with nine thousand separate page requests -- not a very nice thing to do. And you'd still have to search through the results to find whatever it is you're looking for.
Can you contact the site administrators and ask for whatever you're looking for, instead?
posted by ook at 11:57 AM on November 2, 2004
Okay, on the non-scripting end, have you tried a Google cache search, or possibly the Internet Archive? Either might have stored the site when your page was openly posted.
posted by DrJohnEvans at 12:03 PM on November 2, 2004
posted by DrJohnEvans at 12:03 PM on November 2, 2004
Response by poster: Thanks for the quick advice. I've actually been able to find what I was looking for a different way, but this is a general question that I've often wondered about.
FYI:
- I'm running WinXP
- I did try Google and the Archive, but neither had what I was looking for.
posted by neurodoc at 12:30 PM on November 2, 2004
FYI:
- I'm running WinXP
- I did try Google and the Archive, but neither had what I was looking for.
posted by neurodoc at 12:30 PM on November 2, 2004
If you're using Firefox, this extension can inc / decrement numeric parts of a URL, and even run a slideshow of the pages.
posted by punilux at 1:04 AM on November 3, 2004
posted by punilux at 1:04 AM on November 3, 2004
« Older Where to find exit polls online? | Why do women use paper toilet seat covers in a... Newer »
This thread is closed to new comments.
posted by DrJohnEvans at 11:16 AM on November 2, 2004