Comments on: How to get full RSS instead of partial RSS

Question: How to get full RSS instead of partial RSS

eebs — Mon, 20 Aug 2007 15:57:15 -0800

Help me get full RSS instead of partial RSS. As many of you might know, the Freakonomics blog has moved to NY Times. They used to give full RSS feeds; now all we get is partial RSS. While this generated quite a buzz on the internets, no solution was provided. I get the feeling that a significant portion of the AskMeFi community has programming and internet development expertise. I do not belong to that portion, however I can follow directions. Looking at the whole Web 2.0 development, I cannot help but think that this problem should be a trivial one for programmers. So, questions:

1) Is there a straight forward way of automatically extracting the contents of a blog and generate a full feed for it ? (Isn't Yahoo! Pipes supposed to do this sort of tasks?)

2) (If not) why not ?

Thanks.

By: AaRdVarK

AaRdVarK — Mon, 20 Aug 2007 16:31:36 -0800

Ponyfish and Feed43 both work well for scraping websites into feeds.

By: scottreynen

scottreynen — Mon, 20 Aug 2007 18:40:29 -0800

Openkapow might help too. I've never used it, but others have recommended it to me, and it looks like it might suit your needs.

By: eebs

eebs — Mon, 20 Aug 2007 23:40:32 -0800

First, thanks for your answers.

Here are my observations:

Ponyfish
I was impressed by how straight forward the rss creation procedure they've put in place. Unfortunately, it grabs only the titles.

Feed43
After reading the help pages, I was able to implement what I was asking for.(Here is the feed.) But, it is not working for me:
1) My aggregator, Google Reader, does not recognize this as a feed. Bloglines does, but then you cannot click on the title and jump to the nytimes site (the link assigned to the title takes you back to Bloglines). Even if it worked properly, I would not switch over to Bloglines just for this reason.
2) In order to remedy this "title does not link to nytimes" problem, I modified my extraction method in Feed43. I was able to get the links directing to nytimes printed under the titles. But these links have dashes (-) and for some reason Feed43 converts them into %20 or something like that, which causes the link to fail.

Openkapow
Looks interesting but it seems like it requires too much learning for this purpose.

Conclusion:
I will learn to live with partial rss for Freakonomics (they have to come up with really catchy titles for their posts now). But I will definitely keep Ponyfish and Feed43 in mind as they might come in handy.

By: MetaMonkey

MetaMonkey — Tue, 21 Aug 2007 05:23:13 -0800

You could also try Dapper, though it seems a bit fiddly.

By: ethorson

ethorson — Tue, 21 Aug 2007 09:03:22 -0800

I use the full text RSS tool from EchoDitto Labs for my partial-to-full needs--just enter in the url of the feed and it'll magically spit out a full-text version.

By: eebs

eebs — Tue, 21 Aug 2007 16:01:16 -0800

@etherson:

Thanks. That worked just the way I wanted. The "Google Reader not recognizing as a feed" problem is still there. But in Bloglines, it works like a charm. Now I am going to have to try this in other aggregators to see if they could sway me to switch over.

By: hedgehog

hedgehog — Wed, 22 Aug 2007 08:34:55 -0800

Hey, this is great. Thanks ethorson (and eebs for the OP). The full text RSS tool works fine for me, for the Freakonomics blog, with Google Reader.