How to get full RSS instead of partial RSS

August 20, 2007 3:57 PM

Help me get full RSS instead of partial RSS. As many of you might know, the Freakonomics blog has moved to NY Times. They used to give full RSS feeds; now all we get is partial RSS. While this generated quite a buzz on the internets, no solution was provided.

I get the feeling that a significant portion of the AskMeFi community has programming and internet development expertise. I do not belong to that portion, however I can follow directions. Looking at the whole Web 2.0 development, I cannot help but think that this problem should be a trivial one for programmers. So, questions:

1) Is there a straight forward way of automatically extracting the contents of a blog and generate a full feed for it ? (Isn't Yahoo! Pipes supposed to do this sort of tasks?)

2) (If not) why not ?

Thanks.

posted by eebs to computers & internet (7 comments total)

Ponyfish and Feed43 both work well for scraping websites into feeds.
posted by AaRdVarK at 4:31 PM on August 20, 2007

Openkapow might help too. I've never used it, but others have recommended it to me, and it looks like it might suit your needs.
posted by scottreynen at 6:40 PM on August 20, 2007

First, thanks for your answers.

Here are my observations:

Ponyfish
I was impressed by how straight forward the rss creation procedure they've put in place. Unfortunately, it grabs only the titles.

Feed43
After reading the help pages, I was able to implement what I was asking for.(Here is the feed.) But, it is not working for me:
1) My aggregator, Google Reader, does not recognize this as a feed. Bloglines does, but then you cannot click on the title and jump to the nytimes site (the link assigned to the title takes you back to Bloglines). Even if it worked properly, I would not switch over to Bloglines just for this reason.
2) In order to remedy this "title does not link to nytimes" problem, I modified my extraction method in Feed43. I was able to get the links directing to nytimes printed under the titles. But these links have dashes (-) and for some reason Feed43 converts them into %20 or something like that, which causes the link to fail.

Openkapow
Looks interesting but it seems like it requires too much learning for this purpose.

Conclusion:
I will learn to live with partial rss for Freakonomics (they have to come up with really catchy titles for their posts now). But I will definitely keep Ponyfish and Feed43 in mind as they might come in handy.
posted by eebs at 11:40 PM on August 20, 2007 [1 favorite]

You could also try Dapper, though it seems a bit fiddly.
posted by MetaMonkey at 5:23 AM on August 21, 2007

I use the full text RSS tool from EchoDitto Labs for my partial-to-full needs--just enter in the url of the feed and it'll magically spit out a full-text version.
posted by ethorson at 9:03 AM on August 21, 2007 [2 favorites]

@etherson:

Thanks. That worked just the way I wanted. The "Google Reader not recognizing as a feed" problem is still there. But in Bloglines, it works like a charm. Now I am going to have to try this in other aggregators to see if they could sway me to switch over.
posted by eebs at 4:01 PM on August 21, 2007

Hey, this is great. Thanks ethorson (and eebs for the OP). The full text RSS tool works fine for me, for the Freakonomics blog, with Google Reader.
posted by hedgehog at 8:34 AM on August 22, 2007