Is there a way to filter or remove duplicate items from Google Reader?

February 25, 2009 2:08 PM

Is there a way to filter or remove duplicate items from Google Reader?

Some of the websites I read, especially The Guardian, have multiple RSS feeds and often publish the same article to all of them, which means that in some cases half a dozen copies appear in stream because they cover roughly the same range of topics.

There is a Greasemonkey script which added a button to the older design of Reader to automatically strip them from whichever label was being looked at, but it hasn't been updated and doesn't work now. Not knowing anything about whatever language it's written in, I haven't a clue how it might be repaired to bring the button back.

I've tried pushing the relevant feeds them through Yahoo Pipes and other RSS mixing websites with the no-repeat function engaged but the process isn't simultaneous and frequently an article would appear in the new mashed feed two days after it was originally published, and with things such as the author and original feed origin stripped out.

Any suggestions?

posted by feelinglistless to Computers & Internet (8 answers total)

Yea, same problem here. The only way I have found around it is to use Yahoo Pipes to filter out duplicates, then creating a new feed from the filtered content. Not ideal, though, as the time frame for Pipes is longer than for the original feed, in most cases.
posted by gemmy at 4:22 PM on February 25, 2009

If you really like the broken script, and the author isn't responding to feedback on the script page, you can post to the userscripts.org forums asking a scripwright for a fix. That approach sometimes works with problematic scripts, particularly for small changes.

But if no volunteers are found, or the forum idea doesn't appeal, I can probably fix the script. If you haven't found a better solution, or someone else doesn't update the script before then, post or drop a MeMail in a day or two or whatever and should be able to hack a working version.
posted by mdevore at 4:30 PM on February 25, 2009

mdevore: will do, thanks. I've emailed the author -- he still seems to be around.
posted by feelinglistless at 1:44 AM on February 26, 2009

There also the Google Reader Filter script for Greasmonkey, which can grey out or hide dupes, but it still doesn't work unless the dupes are present simultaneously. And I mean in the same window - you'd have to be reading the different feeds in the same tag/folder for it to work, you couldn't read them individually.
posted by timepiece at 10:25 AM on March 2, 2009

timepiece: I've tried that one but it's a bit too techy -- also it only filters by title and The Guardian often have a range of different titles for each article which has the same URL.
posted by feelinglistless at 1:06 PM on March 5, 2009

The author of the broken "Remove Dupes from Google Reader" script in the posted question didn't respond to feelinglistless, and so after contacting me, and MeMailing back and forth, we have a fixed script. He suggested I post back here with the question resolution, and for anyone else who might be interested in a fix.

The modified script is available for downloading for anyone else interested in the original script functionality. It has several enhancements and a detailed write-up on the fixes and new features is available for anyone curious or bored enough to read about it.
posted by mdevore at 2:43 PM on March 9, 2009

This is great! Just what I needed! Super super super!
posted by Mo Nickels at 1:57 PM on March 10, 2009

Someone MeMailed me asking how to change the look of the new button in the script code, so I'll post a tip here for others who might want to goof around with it.

The new button is ugly compared to the Google buttons because it's a bog-standard default button, while the Google buttons are really nested divs which are built and massaged to their final form through classes and styles.

Anyway, you can easily insert a new style for the button; simply give it a unique class name and use the GM_addStyle() function. After the existing script line:

dupeNode.setAttribute("id", rdID);

add the two lines:

dupeNode.setAttribute("class", "SomeGoofyClass");

GM_addStyle("button.SomeGoofyClass { height: 20px }");

where SomeGoofyClass is replaced by the name of your new class name (or keep that one), and the style changes the button height to 20 px. Modify and add to the style to your heart's content.
posted by mdevore at 3:37 PM on March 10, 2009