Can Spotlight index websites?

March 6, 2009 11:56 PM

Can Spotlight index websites?

I have a mediawiki install on my macbook that I would like Spotlight to index, but more generally, I would like a Spotlight plugin that can index arbitrary URLs, local or WAN (wouldn't it be cool to have wikipedia results show up in spotlight?). Google has not been helpful in finding this. I do have a plugin that downloads delicious bookmarks which suggests my idea should be feasible, but I'm not a programmer and I don't want to start tearing apart the delicious plugin to see how it works. Anyone have a suggestion? If spotlight can't do this, I would be open to alternative approaches.

posted by Chris4d to computers & internet (7 answers total)

I'm pretty sure the answer is no: it's very much keyed to indexing files on disk. I'm sure you could hack it by writing an "indexer" which produces index terms for a file by retrieving a corresponding url, and that might even be an interesting hack, but I don't think spotlight can do what you describe out-of-the-box.
posted by hattifattener at 12:33 AM on March 7, 2009

The GoogleImporter plugin page says "When you begin a Spotlight search, Google Importer will instantly grab the top results from Google and show them right next to the files from your computer. With one click, the result will open in your default browser. You can even choose to only search a particular website."
posted by nicwolff at 12:47 AM on March 7, 2009

googleimporter is for Tiger only.
posted by Gungho at 5:26 AM on March 7, 2009

Webarchive the URIs, this will create documents that are locally saved with the contents of the web page. You can do this just with Safari manually or automate it in about thirty seconds with Applescript/Automator.

I use DEVONthink to manage many things, including several hundred webarchives now.
posted by Brian Puccio at 6:24 AM on March 7, 2009

I should add you don't need DEVONthink to do this, I just use it for organization and have a keyboard shortcut (⌘ + 2) that takes whatever page I'm currently looking at, webarchive's it and sticks it into DEVONthink for me, which I clean up once a week and organize.

You could just save the webarchive into a folder and organize it however you want. Spotlight will search them like any other file.
posted by Brian Puccio at 6:29 AM on March 7, 2009

Spotlight is for filesystems. Try Monocle, instead.
posted by Blazecock Pileon at 8:44 AM on March 7, 2009

Thanks for the suggestions so far. I would prefer to use spotlight because it is already there and I don't need another service running and another place to have to search. Besides, the division between filesystems and webpages is not meaningful in the primary context of my question. If my local mediawiki install were a collection of static HTML pages instead of a database, spotlight would happily index and search them. So I guess, rather than get derailed talking about web searching, can I ask, is there a way to get spotlight to index a SQL database?
posted by Chris4d at 4:44 PM on March 12, 2009