How can I create a blog that autoposts specific content when I post it at another site?
April 20, 2012 4:22 AM   Subscribe

How does the Metafilter Deleted Threads blog pull info and then publish it automatically?

I'm not a programmer, so forgive me if this is a easily answerable question.

I want to know how the blog pulls the name of the thread, creates a link to the thread in the post title, and then pulls and publishes the reason for deletion.

I'd like to set up something similar for my links and have no idea how to do so.
posted by reenum to Computers & Internet (8 answers total) 7 users marked this as a favorite
I am pretty sure it's done manually. This has been discussed before, I remember asking the same thing.
posted by jayder at 4:29 AM on April 20, 2012

I'd like to set up something similar for my links and have no idea how to do so.

When I do scraping for MetaFilter or other sites I use BeautifulSoup. There are similar tools for other languages. If I was writing a deleted thread scraper for MetaFilter I would probably set it up to look for gaps in the thread IDs on the front page, then pull the pages for the unlisted IDs, and grab the relevant data from those pages. If a site has an actual API for accessing their content it's obviously a better idea to use that, but fetching and parsing pages is a good quick and dirty method that works pretty much anywhere.
posted by burnmp3s at 4:48 AM on April 20, 2012 [2 favorites]

MeFi threads are numbered sequentially, suggesting you could pull the front page and then hunt for skips in the numbering.
posted by Rat Spatula at 5:28 AM on April 20, 2012

Deleted threads aren't technically removed, but hidden from view. You could alternatively use a greasemonkey script to show the hidden threads. What's likely going on with this blog, is it's searching for post IDs that do not show up in the main feed.
posted by samsara at 5:30 AM on April 20, 2012

Metafilter Deleted Threads is kind of a red herring here if your real question is the one in the title ("How can I create a blog that autoposts specific content when I post it at another site?").

Delicious provides its own communication methods (an API as well as a variety of feeds) so your content management system can communicate with it and pull in links. Are you using WordPress? Install something like Delicious RSS Display and you're done:
If you have a delicious account and you want to display your bookmarks on your blog, Delicious RSS display is for you. Depending on the categories or tags assigned to a specific post, Delicious RSS display fetch relevant bookmarks from your delicious account and displays them on your post. Everything is based on delicious tags so you must have tagged your delicious bookmarks. If some bookmarks are tagged with the same tags / categories than your post, they are displayed on your post page.
Similar plugins may exist for other blogging platforms—in the most basic sense you just need a plugin that can read an external RSS feed since that's pretty common.
posted by bcwinters at 5:35 AM on April 20, 2012

It's a good first project, if you want to learn how to program.

Bascially, you...
1 - Grab the RSS feed of your delicious
2 - Parse the feed
3 - Add any missing items to your database and flag them to be grabbed
4 - Loop:
Missing page? Yes? Grab it and parse title and / or contents of the site and add them to your database and unset the flag on each "to grab" item.
5 - Build a way to display all the entries in your database

Personally, I'd recommend PHP / MySQL for the task since that's pretty common in shared host environments... but really, any web-type language you want to do will work. If you want to learn how to build internet stuff, something like this is a great place to start.
posted by ph00dz at 5:37 AM on April 20, 2012

Replying to the question in the title "How can I create a blog that autoposts specific content when I post it at another site?": Have you seen ifttt?
posted by Memo at 6:14 AM on April 20, 2012 [4 favorites]

I've got a site that does something like this. Basically, I grab an RSS feed, parse it and reformat the content, then post it by sending it as an email to Blogger.
posted by Knappster at 11:55 AM on April 20, 2012

« Older Tuna water   |   How to plan a trip to Paris Newer »
This thread is closed to new comments.