Looking for programmatic web searching for commercial purposes now that Alexa Web Search is gone.

December 3, 2008 4:11 PM

Looking for programmatic web searching for commercial purposes now that Alexa Web Search is gone.

I'm looking for programmatic web search access (e.g. Google API or Alexa Web Search) for commercial purposes.

I'd like to look up product IDs for existence on the web and pass it to a human if there is at least one result. I may be searching for 10,000 to 50,000 items per day (though not every day) and would like not to be locked out due to over-usage.

It looks like Google search API and Yahoo search API are both limited in the amount of searches you can do per day. Anyone have any luck getting approved higher load access from them?

Since this is for commercial purposes I don't mind paying a (hopefully nominal) fee for access.

It looks like Alexa Web Search would have been usable/perfect, but they are closed as of this week it appears.

Any suggestions?

posted by bottlebrushtree to computers & internet (4 comments total)

Google's current search API allows unlimited use, although I assume they would still block you for extremely high-rate requests from a single IP address.
posted by teraflop at 5:05 PM on December 3, 2008

Teraflop, thanks.

The Google API Terms of use (http://code.google.com/apis/ajaxsearch/terms.html) contains some terms that make me think it won't allow me to do what I need such as:

"The API may be used only for services that are accessible to your end users without charge."

No "intermixing Search Results from sources other than Google,"

etc....

so I'd be interested in something that has a more clear TOS or commercial interface. (though I just might end up using Google until I get shut off)
posted by bottlebrushtree at 6:15 PM on December 3, 2008

What about Yahoo BOSS?
posted by Good Brain at 8:38 PM on December 3, 2008

Good Brain thanks for the pointer, unfortunately Yahoo BOSS's Terms of Service (http://info.yahoo.com/legal/us/yahoo/search/bosstos/bosstos-2317.html) say:

"You are permitted to use the Services only for the purpose of incorporating and displaying Web Search Results from such Services as part of a Search Product deployed on your Web site"

I'm looking for a search engine which will allow me to use their results for medium volume internal research.
posted by bottlebrushtree at 8:43 PM on December 3, 2008