Help me preserve my deliciousness

December 22, 2006 12:11 PM

Help me preserve my deliciousness (i.e. disappearing del.icio.us bookmarks)

Lately I’ve noticed that my delicioused pages are slowly disappearing (mostly how-tos and recipes) so I’d like to find a tool that will crawl and locally cache all my delicious bookmarks. Bonus if I can specify a tag (or tag wildcard. Example: I tag as recipes.french, recipes.dessert, recipes.dessert.ice_cream etc. so I would love to cache recipes.*)

My google fu found me some python scripts (I have no idea how to use em) and someone suggested using spur (but spur doesn’t seem to like my delicious account).

Any better ideas, mefites?

posted by special-k to computers & internet (14 comments total)

PS: I'm running FF on XP
posted by special-k at 12:12 PM on December 22, 2006

If you're only wanting to backup your bookmarks, what I do is call the del.icio.us API and ask for the all? action:

https://api.del.icio.us/v1/posts/all?

It'll ask you for your delicious authentication, and give you an XML file which you can save.

As for getting any of that information out of the XML file in a sane way, that's a far deeper topic. At worst, you can crawl it or search it manually and find what you need.
posted by Skorgu at 12:16 PM on December 22, 2006

sorry, I should have made it clear. My bookmarks are fine but the pages are no longer available. I'd like to save them before they disappear forever.

I'm quite happy syncing with foxylicious.
posted by special-k at 12:23 PM on December 22, 2006 [1 favorite has favorites]

Hmm, that's somewhat more interesting. My first instinct (as it's easiest) is to use wget which is a generally unix-only command. You could install cygwin, but that is probably more trouble than it's worth for you.

In any case, I think that running wget with the -i switch and giving it that XML file should work. Maybe. Otherwise, I have a ruby script that could probably be munged into doing what you want if you want to go that far.
posted by Skorgu at 12:29 PM on December 22, 2006

Aha, note to self, google before posting. There's a wget for windows.

I think getting those XML files from the delicious api and running wget -k -i XML_FILENAME should pull what you want.
posted by Skorgu at 12:33 PM on December 22, 2006

I know its not quite the question you asked, as it's not a local copy but how about using another bookmarking tool that does cache the page as well? There are a few, I think Spurl, Furl, Yahoo Myweb and a few others do so.

I've been using Diigo (link here). It stores a copy of the page, and also posts it to Delicious and other bookmarking services. Also, it can import directly from Delicious, and cache of copy of the page as it does so, so it would be a very quick and easy solution
posted by Boobus Tuber at 12:53 PM on December 22, 2006

some python scripts (I have no idea how to use em)

Assuming the scripts just need a couple of command line parameters, you just need to install Python on Windows (Linux and OSX should already have it) and run the scripts from the command line like:

>python scriptname.py -argument=whatever

That's as much as I've got. I've noticed the same problem with my del.icio.us account, but it's not an easy one to solve programatically as some links 404 and some get redirected. I suppose you really just need to compare the title delicious has with what a URL request comes back with, but that might create some false positives.
posted by yerfatma at 1:12 PM on December 22, 2006

dead.licious is a tool for verifying that all of your bookmarks in your del.icio.us accounts are still valid and gives you the option of removing those dead links.

Think it's mac only though, but it does what you want.
posted by chrissyboy at 1:47 PM on December 22, 2006 [2 favorites has favorites]

ooh sorry, apart from the locally caching thing. duh
posted by chrissyboy at 1:48 PM on December 22, 2006

My wget script from an earlier thread:

#!/bin/sh

/usr/local/bin/wget --recursive -k --span-hosts --level=1 --wait=1 \
--tries=3 --cut-dirs=1 --exclude-domains="del.icio.us" \
--output-file="./logfile.txt" http://del.icio.us/Username

Read and modify as needed. The good news, it's free. The bad news, some urls may not result in file extensions.
posted by KirkJobSluder at 5:23 PM on December 22, 2006

Furl saves pages and URLs, if you're willing to switch services.
posted by puckupdate at 5:29 PM on December 22, 2006

And it lets you import del.icio.us files: http://www.furl.net/import.jsp (might need an account to see that page, though)
posted by puckupdate at 5:36 PM on December 22, 2006

Sign up for blinklist (or similar -- I cannot remember all the ones that cache copies of pages), import your de.icio.us bookmarks, and they should each cache copies of the pages for you. Most of these services recognize that delicious is their major frontrunner, so they offer import features to "move you" over.

on preview, yeah or furl. same idea
posted by misterbrandt at 5:38 PM on December 22, 2006

Duh, use the --html-extension option.

A slightly better script:

#!/bin/bash

/usr/local/bin/wget --recursive -k --span-hosts --level=1 --wait=1 \
--tries=3 --cut-dirs=1 --exclude-domains="del.icio.us" \
--html-extension \
http://del.icio.us/CBrachyrhynchos

for i in `find . -name "*[\?=&]*"`; do
	mv $i ${i//[\?&=]/_}
done

The last little bit replaces characters from html queries with underscores. Tested with bash on OS X tiger.
posted by KirkJobSluder at 6:12 PM on December 22, 2006 [1 favorite has favorites]