Help me preserve my deliciousness
December 22, 2006 12:11 PM
Help me preserve my deliciousness (i.e. disappearing del.icio.us bookmarks)
Lately I’ve noticed that my delicioused pages are slowly disappearing (mostly how-tos and recipes) so I’d like to find a tool that will crawl and locally cache all my delicious bookmarks. Bonus if I can specify a tag (or tag wildcard. Example: I tag as recipes.french, recipes.dessert, recipes.dessert.ice_cream etc. so I would love to cache recipes.*)
My google fu found me some python scripts (I have no idea how to use em) and someone suggested using spur (but spur doesn’t seem to like my delicious account).
Any better ideas, mefites?
Lately I’ve noticed that my delicioused pages are slowly disappearing (mostly how-tos and recipes) so I’d like to find a tool that will crawl and locally cache all my delicious bookmarks. Bonus if I can specify a tag (or tag wildcard. Example: I tag as recipes.french, recipes.dessert, recipes.dessert.ice_cream etc. so I would love to cache recipes.*)
My google fu found me some python scripts (I have no idea how to use em) and someone suggested using spur (but spur doesn’t seem to like my delicious account).
Any better ideas, mefites?
If you're only wanting to backup your bookmarks, what I do is call the del.icio.us API and ask for the all? action:
As for getting any of that information out of the XML file in a sane way, that's a far deeper topic. At worst, you can crawl it or search it manually and find what you need.
posted by Skorgu at 12:16 PM on December 22, 2006
https://api.del.icio.us/v1/posts/all?It'll ask you for your delicious authentication, and give you an XML file which you can save.
As for getting any of that information out of the XML file in a sane way, that's a far deeper topic. At worst, you can crawl it or search it manually and find what you need.
posted by Skorgu at 12:16 PM on December 22, 2006
sorry, I should have made it clear. My bookmarks are fine but the pages are no longer available. I'd like to save them before they disappear forever.
I'm quite happy syncing with foxylicious.
posted by special-k at 12:23 PM on December 22, 2006
I'm quite happy syncing with foxylicious.
posted by special-k at 12:23 PM on December 22, 2006
Hmm, that's somewhat more interesting. My first instinct (as it's easiest) is to use wget which is a generally unix-only command. You could install cygwin, but that is probably more trouble than it's worth for you.
In any case, I think that running wget with the -i switch and giving it that XML file should work. Maybe. Otherwise, I have a ruby script that could probably be munged into doing what you want if you want to go that far.
posted by Skorgu at 12:29 PM on December 22, 2006
In any case, I think that running wget with the -i switch and giving it that XML file should work. Maybe. Otherwise, I have a ruby script that could probably be munged into doing what you want if you want to go that far.
posted by Skorgu at 12:29 PM on December 22, 2006
Aha, note to self, google before posting. There's a wget for windows.
I think getting those XML files from the delicious api and running wget -k -i XML_FILENAME should pull what you want.
posted by Skorgu at 12:33 PM on December 22, 2006
I think getting those XML files from the delicious api and running wget -k -i XML_FILENAME should pull what you want.
posted by Skorgu at 12:33 PM on December 22, 2006
I know its not quite the question you asked, as it's not a local copy but how about using another bookmarking tool that does cache the page as well? There are a few, I think Spurl, Furl, Yahoo Myweb and a few others do so.
I've been using Diigo (link here). It stores a copy of the page, and also posts it to Delicious and other bookmarking services. Also, it can import directly from Delicious, and cache of copy of the page as it does so, so it would be a very quick and easy solution
posted by Boobus Tuber at 12:53 PM on December 22, 2006
I've been using Diigo (link here). It stores a copy of the page, and also posts it to Delicious and other bookmarking services. Also, it can import directly from Delicious, and cache of copy of the page as it does so, so it would be a very quick and easy solution
posted by Boobus Tuber at 12:53 PM on December 22, 2006
some python scripts (I have no idea how to use em)
Assuming the scripts just need a couple of command line parameters, you just need to install Python on Windows (Linux and OSX should already have it) and run the scripts from the command line like:
>python scriptname.py -argument=whatever
That's as much as I've got. I've noticed the same problem with my del.icio.us account, but it's not an easy one to solve programatically as some links 404 and some get redirected. I suppose you really just need to compare the title delicious has with what a URL request comes back with, but that might create some false positives.
posted by yerfatma at 1:12 PM on December 22, 2006
Assuming the scripts just need a couple of command line parameters, you just need to install Python on Windows (Linux and OSX should already have it) and run the scripts from the command line like:
>python scriptname.py -argument=whatever
That's as much as I've got. I've noticed the same problem with my del.icio.us account, but it's not an easy one to solve programatically as some links 404 and some get redirected. I suppose you really just need to compare the title delicious has with what a URL request comes back with, but that might create some false positives.
posted by yerfatma at 1:12 PM on December 22, 2006
dead.licious is a tool for verifying that all of your bookmarks in your del.icio.us accounts are still valid and gives you the option of removing those dead links.
Think it's mac only though, but it does what you want.
posted by chrissyboy at 1:47 PM on December 22, 2006
Think it's mac only though, but it does what you want.
posted by chrissyboy at 1:47 PM on December 22, 2006
ooh sorry, apart from the locally caching thing. duh
posted by chrissyboy at 1:48 PM on December 22, 2006
posted by chrissyboy at 1:48 PM on December 22, 2006
My wget script from an earlier thread:
posted by KirkJobSluder at 5:23 PM on December 22, 2006
#!/bin/sh /usr/local/bin/wget --recursive -k --span-hosts --level=1 --wait=1 \ --tries=3 --cut-dirs=1 --exclude-domains="del.icio.us" \ --output-file="./logfile.txt" http://del.icio.us/UsernameRead and modify as needed. The good news, it's free. The bad news, some urls may not result in file extensions.
posted by KirkJobSluder at 5:23 PM on December 22, 2006
Sign up for blinklist (or similar -- I cannot remember all the ones that cache copies of pages), import your de.icio.us bookmarks, and they should each cache copies of the pages for you. Most of these services recognize that delicious is their major frontrunner, so they offer import features to "move you" over.
on preview, yeah or furl. same idea
posted by misterbrandt at 5:38 PM on December 22, 2006
on preview, yeah or furl. same idea
posted by misterbrandt at 5:38 PM on December 22, 2006
Duh, use the --html-extension option.
A slightly better script:
posted by KirkJobSluder at 6:12 PM on December 22, 2006
A slightly better script:
#!/bin/bash /usr/local/bin/wget --recursive -k --span-hosts --level=1 --wait=1 \ --tries=3 --cut-dirs=1 --exclude-domains="del.icio.us" \ --html-extension \ http://del.icio.us/CBrachyrhynchos for i in `find . -name "*[\?=&]*"`; do mv $i ${i//[\?&=]/_} doneThe last little bit replaces characters from html queries with underscores. Tested with bash on OS X tiger.
posted by KirkJobSluder at 6:12 PM on December 22, 2006
This thread is closed to new comments.
posted by special-k at 12:12 PM on December 22, 2006