Delicious Dupefilter?

August 17, 2008 5:26 PM

When watching a tag on Delicious, whether through subscribing on-site or to the tag's RSS feed, I find reading through the deluge of posts quite cumbersome. Is there a way to filter out duplicate links to the same site that have been posted by different users?

There apparently used to be a service to do just this, but it was taken down at some point.

There's got to be a way! Oh hive mind, can you help me?

posted by defenestration to computers & internet (5 comments total)

Try Yahoo Pipes.

Use the fetch feed widget, then connect it to Operator:Unique, based on item.link, then to the output.

I'm not sure how their unique is implemented, so I don't know if it will "remember" the link forever, but it will at least remove the dups from each update.
posted by chrisamiller at 5:44 PM on August 17, 2008

Definitely a step in the right direction. Yahoo Pipes is pretty awesome, too. I hadn't checked it out before.

If this was able to compare every new entry to every single past RSS entry would be perfect. I can see why this wouldn't be feasible in an web app/service, but I could see it working in a desktop application. Hmmm.
posted by defenestration at 6:18 PM on August 17, 2008

... entry *it* would be ...

... feasible in *a* web app/service ...

I should probably preview my responses. Oops.
posted by defenestration at 6:21 PM on August 17, 2008

I don't know how programatically inclined you are, but if I were desperate for this service, I'd probably just roll my own.

Pull down the feed and parse it with your favorite scripting language, and have it keep a db or flat file with all the links that have come through already. Filter out the ones you've seen, add the ones you haven't, and output the new links.

If your script-fu isn't up to it, maybe you know a friend who could help?
posted by chrisamiller at 6:38 PM on August 17, 2008

Thanks for the pipes. new toy.........
posted by prodevel at 8:02 PM on August 17, 2008