Search is a search is a search...

November 11, 2015 12:46 PM

Back in the olden days I had a whole slew of search engines I would use to find stuff on the interwebs, some engines were better than others, some were slanted in areas, science, images, etc. These days I just use Google and it seems to do me pretty well.

But, a good friend of mine is constantly looking for new ways to look at archived information on the internet, for her job it is a great service to use sites like the wayback machine, to try and find deleted material on the Internet (she is an investigator and surprisingly there is no list of semi secret search engines at her disposal.)

Recently a colleague of hers stumbled onto archive.is and was pretty impressed with the results she was getting. That made her ask me what if any search engines that I used to use were still available. I checked my oldest set of bookmarks (2010), many of them were dead links like "The Mining Company" others had been reborn into search engines I know nothing about about such as: Search.com, Factbites and Zoo. Some still existed like Dogpile, HotBot and Lycos.

When I describe to her how I spent my days trolling the internet before AOL came out, (for me mostly usenet and some chat rooms and the like) she is always interested if any of that data is still available in archived format. What she really wants to find is someplace to search through old IRC chats room and Usenet archives if such data exists.

I figure the pundits here would have plenty to say about it all, so lets have it!

posted by silsurf to Computers & Internet (8 answers total)

for usenet there's google groups - i just had a look, and comp.lang.list goes back to november 86, for example (i can no longer remember the details, but vaguely remember they bought some big archive back in the day).
posted by andrewcooke at 1:05 PM on November 11, 2015

Hmmm... I used to work at KnowYourMeme, depending on just what you're looking for, I bet some of the researchers there now could point you in a good direction. Similarly, I bet people who were involved with the Web Ecology Project while that was a thing might have some direction. I'm not a part of that crowd any longer (and was never in a research position) so I'm not really what or who else might know.

The only place that comes to mind for an IRC archive is bash.org, though that's more of a funny quotes repository than anything else.
posted by oh.ghoulin at 1:05 PM on November 11, 2015

IRC chat is typically not logged, so I doubt she'll have much luck there. But Usenet archives can be found through Google Groups.

Your question is not very clear to me though. Is this about old search engines that search the Web, or about Usenet (which is not on the Web), or about actual old content on the Web? Can you please clarify?
posted by Too-Ticky at 1:05 PM on November 11, 2015

I am curious of people have found unique search engines in the is day of google-dom. In the past there were all these mom and pop search engines and some were terrific at mining for unique and obscure data, so my question is how many engines like that still exist, archive.is being a good example

Thanks

Henry
posted by silsurf at 1:22 PM on November 11, 2015

[dumb typo - comp.lang.lisp, obviously]
posted by andrewcooke at 1:34 PM on November 11, 2015

archive.is isn't a search engine, though- it's basically archive.org on-demand. Its db only has what people have saved into it, which isn't much.

I think the reason that so many of the specialized search engines have gone by the wayside is that Google really covers everything if you know how to mine it. The only differentiators other search engines bring to the table are privacy (DuckDuckGo) or the ability to understand "facts" (Wolfram Alpha).
posted by mkultra at 1:35 PM on November 11, 2015

Correct me if I'm reading your question wrong, but maybe you're looking for deep web search engines.
posted by ernielundquist at 1:45 PM on November 11, 2015

So the reason why archive.is is powerful in the investigatory world is looking for clues in people social media accounts, so to be able to see someones facebook page from a year ago can be VERY helpful when looking for connections and clues on peoples behavior, which is what my friend basically does for a living is search through databases.

She uses many paid services which are higher end, but they are all about government data and news organizations, etc, it seems social media access is harder to find, yet in her line of work it is indispensable in tracking down activity.

Henry
posted by silsurf at 1:47 PM on November 11, 2015