How do I find and feed blog posts into my website?

March 22, 2006 9:06 AM

FeedFilter! How do I: a) get bloggers to tag their posts about X with “X”; b) subscribe to the “X” tag via RSS; c) feed the “X”-tagged posts into a non-profit web site I run about X?

I run a non-profit website that collects memories about, well, everything (MemoryWiki). I want to find a way for bloggers to feed their memories, memoirs and reminiscences into the site automatically. (No stealing! I'll include the posts only if they want to share.) I think tags might be the way to go, but I’m pretty much clueless as to how to go about getting folks to tag and feed. As you can see, I'm confused. Help!

posted by MarshallPoe to Computers & Internet (4 answers total)

First, do you mean all bloggers everywhere, or some group of bloggers that you have a rapport with? If the former, abandon all hope. If the latter, presumably you are communicating to those bloggers (perhaps through your own blog). To suggest a tag to them, you might use the Technorati tagging microformat. There are other ways of doing this, although not all blogging software will necessarily cooperate: the keywords could also be embedded in "category" tags in an RSS2 or Atom feed. Del.icio.us deals with tags in its RSS1 feeds using the taxo: namespace, and they practically invented tagging.

If you go with the Technorati microformat, just say "include this HTML snippet in your posts about X" in your communication to your bloggers. Otherwise, your bloggers will need to be able to edit their RSS/Atom templates, which gets messier--but will probably result in feeds that are easier for you to parse.

If you've got a known group of bloggers, you can subscribe directly to all their feeds and parse them as you see fit. There are numerous feed-parsing tools out there, but this gets a little out of my league.

Finally, you're going to need to take their feeds and somehow post them into your wiki. I'm not sure how you'd go about that. I'm sure a Perl-meister could roll something up.
posted by adamrice at 9:31 AM on March 22, 2006

If bloggers use Technorati tags as adamrice suggests, you can then just use the feed from the Technorati tag search page to aggregate their posts.
posted by mbrubeck at 9:55 AM on March 22, 2006

a) Getting Bloggers to do anything with continuity is impossible.

c) This will probably violate copywrite of someone's material and is illegal.
posted by kc0dxh at 1:28 PM on March 22, 2006

Sheesh, there's nothing prima-facie illegal about what MarshallPoe is proposing. Assuming the bloggers are willing collaborators, they can creative-commons license their works, or grant MarshallPoe some other kind of reuse license. Also: it's copyright, not copywrite.

I agree that it can be hard to get bloggers to do anything consistently, but it's not impossible, and I can imagine this working about 80% right.

Scraping tag feeds out of Technorati would work, but could create problems if there were posts from bloggers unaffiliated with this project using those tags.
posted by adamrice at 2:01 PM on March 22, 2006