How to search for anything when all else has failed?

October 21, 2008 6:08 PM

How to search for anything (or find the answer to anything) on the internet when all else has failed?

When Google's advanced search (and operators) isn't enough and I have gone through the major search engines, where do I turn? I know there have been discussions of several types of semantic search engines like powerset.com. And then there has been discussion of cuil.com among many others. They dont seem to work well.

But when I need to find something that I give up searching for, I am forced into asking the question to a comment board related community. And even then its sometimes hard to find the comment board specifically for the industry/topic I am interested in. General answer sites like yahoo answers work well, but sometimes I get frustrated at how long it takes to get a valid answer and by that time I am over it.

I would be curious to know what you guys have in your tool set to tackle the hardest questions and find the best results.

posted by schindyguy to computers & internet (20 answers total)

You've just used it. This place is excellent.

sometimes I get frustrated at how long it takes to get a valid answer and by that time I am over it

Problem solved!
posted by flabdablet at 6:16 PM on October 21, 2008

I think we need to know what you're looking for to give you a useful answer. For example, if I have a medical question I go to PubMed. If I'm trying to kill scale on my lemon tree I look at gardening sites (I may go to ask.me for suggestions of the best gardening sites to search first.)

So, what are you trying to find?
posted by serazin at 6:19 PM on October 21, 2008

Google is my primary search engine. I will occasionally try other search engines because their crawlers may have picked up pages that Google missed.

But my single best recommendation - find the message boards that cover the topic you are interested in, and then search them. Normal search engines do not do a good job of indexing forums - the UBBs and PHPBBs of the world.
posted by zippy at 6:19 PM on October 21, 2008

Um, also, this may not solve your need for an instant answer, but librarians are actually really good at helping people research stuff.
posted by serazin at 6:20 PM on October 21, 2008 [4 favorites]

Ask here. If there is a specific resource that has a wealth of information (e.g. PubMed, FlyerTalk, CreditBoards), someone here will usually point you to it at some point.
posted by grouse at 6:23 PM on October 21, 2008

but librarians are actually really good at helping people research stuff.

This sort of echoes the answer I was going to give, which is that one of the real keys to finding information is knowing when the internet is not the right approach, and instead you should be going out and talking to real, live people who know stuff, or who know how to learn stuff.

The library is a decent first stop, although librarians tend to be pretty hit or miss for finding information, in my experience. A good reference librarian is worth his/her weight in gold, but simply having a MLS and sitting at that desk is no guarantee of usefulness. The real advantage of most libraries is that they will be able to give you access to databases that are not open to everyone. With that, you can dig into peer-reviewed articles, or old maps, or whatever it is you are looking at, in a way that Google can't match.

And everyone you meet, all day every day, is an expert in something -- the key is asking the right question to the right person, and knowing what to do with the answer. Most people want to be helpful, but they don't have endless time and they are thin-skinned about having their time wasted or being disrespected. But there are a host of questions that are really tricky to figure out as an outsider that are easily answered by a person who knows how that thing works... but they may not have that information typed up and condensed into a form ready for you to read on the internet. You may need to visit, or call, or figure out someone who can get you in the door.
posted by Forktine at 6:30 PM on October 21, 2008 [1 favorite]

I'm a librarian. I have lists of sites that search the dark net or whatever you want to call it. Here is one. Really sometimes collaborative info systems/sites [like this one] are the best at answering something, it really depends what you're looking for. If there is a good website already -- I think like Howards Forums or CrazyMeds or library listserv archives -- you go to one of those places. Part of the deal is that most people, save librarians, don't have to know where to get ALL sorts of esoteric knowledge, you find the sites that are in your interest areas and then you go to a profesional or a fanatic in another subject area if you need something outside of your area of expertise. Part of the process too is understanding "is this a question the internet can likely answer for me" and sometimes just answering *that* question is a time consuming thing, like you don't know that the Internet doesn't know something until someone tells you.

So the flow chart is like this

GOOGLE FAIL

- did google fail because it doesn't exist online?
-- if YES, go outside and go to the library
-- if NO try
--- darknet resources in specific areas
--- hobbyist speciaist areas
--- searchable listserv archives, google groups, etc
--- info silos like subscription databases and other big archives with narrow doorways

There's even a big open question of whether people really know how to use Google, do site limiters and whatnot. i have greasemonkey script for my browser that lets me redo a Google Search in one of ten search engines just by clicking a link and I'm often surprised what other stuff I turn up on other site searches, so make sure you're going down that road as well.

That's the top of my head ideas. Really there's no such thing as just searching for "some information" you have to have at least a general direction of what sorts of things you might be looking for before people can nudge you in a better direction. Do you have examples of questions you have asked that you couldn't research on your own? I know it can be annoying as hell to have someone say "oh I found that by searching THIS way..." but that might also be useful information depending where you're getting hung up.
posted by jessamyn at 6:35 PM on October 21, 2008 [24 favorites]

askme is pretty much it (I found it billed as "the place to go when Google Answers doesn't cut it," which turned out to be uber-accurate). Or your local librarian. (Gee, that didn't really add much.) Don't forget you can search specific sites with Google with the site:somewebsitename.com tag.

One thing you might want to do is let whoever answered your question that it their answer solved your problem or that it didn't, but you appreciate their help anyway. I hate it when I answer someone's question and I think I did a pretty good job, and they fade into the aether never to be heard from again - it just makes me feel incomplete. I must be shallow, or something.
posted by ostranenie at 6:42 PM on October 21, 2008 [1 favorite]

Echoing Serazin above -- it depends on what it is you're trying to find information about.

I write on a variety of weird topics, and where I go to find info depends on what I'm looking for. Medical information? Pubmed. Theater-themed stuff? The special collection at Lincoln Center's branch of the Public Library. Mafia activity in Eastern Pennsylvania circa 1930? The Scranton Historical Society. There are more specific resources.

Shoot, if I can track down the last living relative of a playwright from 1911 when all I have to go on is the playwright's daughter's name, and get his phone number, it's possible for anyone to find anything.
posted by EmpressCallipygos at 8:02 PM on October 21, 2008

Just to beat a dead horse a little, my local library actually has a web chat widget thingy that lets you talk to the reference desk, if you really really want an internet-based solution. I'm guessing pretty much all libraries answer things over the phone, too.
posted by hattifattener at 8:06 PM on October 21, 2008

There are some meta-search engines out there - These are search engines which search search engines (!).

They submit your search to other sites and lump all the results into one.
try DogPile, Clusty and Mamma. Some are better than others at different things.

Jimmy Ruska's site has some handy stuff, too.

Also, sometimes it's worth just typing WIKI at the front of a Google search - it suggests to Google to try there, too. e.g. google WIKI Metafilter

Someone once told me that there was a very good search engine used by the EU (and/or its governments) - but I don't know what it's called.
posted by Dub at 8:29 PM on October 21, 2008

One part of the answer, that people have touched on a little bit above, but I wanted to expand on, is for-pay databases.

I'm a corporate librarian at a Fortune 500 company. We pay literally millions of dollars a year for access to information that isn't available for free on the internet. Actually, that's a bit of an oversimplification, as there's different bits to different databases that make them worth paying for:

1) Some do provide information that isn't available for free online. If you're looking for newspaper articles, Factiva or Lexis-Nexis is the way to go. Google News doesn't even scratch the surface, as they only index what the newspapers make available on their own websites. The paid newspaper databases cover thousands of newspapers, often going back 30 years or more.

2) Some cover documents which are freely available online, but provide means of searching those documents which are not. Patents are a key example here. Pretty much all developed countries make their patents available online, but searching through the patent offices' own websites is often cumbersome--sometimes because the website has a poor interface, but more importantly because the patents are all in each country's native language, and because patents sometimes use intentionally obfuscatory language. Some of the for-pay patent databases provide their own titles and abstracts for patents, which both de-obfuscates the language as well as providing the information in English.

3) Some are worth paying for because they compile information from many disparate sources. A good analyst report may compile information from hundreds of different sources. All the information in the report may be freely available online, but compiling all the information from the free sources yourself would take days or weeks.

Now, saying that these are "for-pay" databases doesn't necessarily mean you, personally, have to pay for them—at the risk of being repetitive, your local public library likely provides many of them at no cost to you. Some even make the databases available online if you have a library card for the system. If you're affiliated with a research institution (university, large hospital, corporation in an R&D-heavy industry, etc.) you probably have even more through that library.

More generally, what everyone else is saying above: it depends on what information you're looking for. I have scores of different resources I commonly use—some free, some paid—but there's no single one I go to after Google. (Actually, for the sorts of questions I get professionally, I frequently go to one or more of those other sources instead of Google, not after Google.)
posted by DevilsAdvocate at 9:00 PM on October 21, 2008 [3 favorites]

It depends what you're trying to find. But I sometimes find intelligence / newspaper archive services such as Factiva (expensive, I know) succeed where Google fails. They tend to be less neophile than the net and less given to snowballing around certain topics. Also, the information is often, but not always, "truthier." If you don't want to pay for one of these services, individual online archives like the (London) Times and the Guardian are also pretty good although obviously nowhere near as exhaustive.
posted by rhymer at 1:10 AM on October 22, 2008

This is a good article discussing the dark-web "problem" with some handy links to boot.
posted by Jofus at 3:36 AM on October 22, 2008

Know what you are searching for
Know how groups who are interested in / experts on that thing would describe it
Know where they hang out / publish / rant
Look there
posted by zippy at 8:57 AM on October 22, 2008 [1 favorite]

One other quick thing on those subscription-only databases: My local library system offers online access to many of these databases to all library card holders. This is the link (just so you have a sense of what I mean):
http://www.oaklandlibrary.org/databaselist.htm

Perhaps your local library system offers the same service?
posted by serazin at 1:06 PM on October 22, 2008

Wow I love this community :)

I wasnt writing this question because I was having trouble finding a specific piece of information, I was just thinking in general how one could go about finding anything if they tried hard enough (you guys certainly answered my question). Thanks for the answers
posted by schindyguy at 2:26 PM on October 22, 2008

I've found good answers to tough questions on IRC and (believe it or not) chan boards. AskMefi is great too, if you choose your questions wisely.
posted by Zeker at 12:07 PM on October 23, 2008

Oh another I forgot to mention: the stumbleupon toolbar for firefox has a great search by category feature, where you can jump through web sites personally favored by people. This has saved my butt countless times.
posted by Zeker at 12:14 PM on October 23, 2008

Try delicious.com search-box, you'll be amazed with the results.
This is my second option. (after google)
posted by nagalr at 12:50 PM on October 23, 2008