Since I couldn t find anyone who could help us, I decided

January 4, 2013 3:06 PM

I need to have access to all 16,679 of @Horse_ebooks' tweets. Twitter and most other archives will only give me 3,200. As you can imagine this is a dire need.

It would be REALLY nice if the tweets were also in a simple, downloadable or searchable format; I ultimately need to be able to search by keyword.

There are a lot of services that will show the entirety of your OWN tweets but obviously that won't help. Is all of this gold seriously lost to history?

Help me Ask MeFi you're my only hope.

posted by six-or-six-thirty to Computers & Internet (13 answers total)

I looked into this a year or so ago, and Twitter has pretty successfully eliminated the ability to download full histories, both of individual Twitter users or of specific search terms.
posted by NotMyselfRightNow at 3:15 PM on January 4, 2013

Eventually this sort of thing should be available via the Library of Congress, but apparently they're having some difficulty wrangling the immense amount of data involved.
posted by lovecrafty at 3:17 PM on January 4, 2013 [1 favorite]

If they are all still viewable, there is always a way to get the data. If a tool isn't available and you can't do it yourself, I would hire someone to scrape the data for you.
posted by wongcorgi at 3:24 PM on January 4, 2013

Response by poster: Well, the problem isn't getting the ones that are visible; I've collected those.

The problem is the ones that fall beyond twitter's 3,200 display limit.

Apparently that's just inaccessible for now. DISAPPOINTED

Oh also I have less than 0 budget for this, which I suppose I should have possibly mentioned.
posted by six-or-six-thirty at 3:29 PM on January 4, 2013

This article from Bloomberg Businessweek says some people can request an archive of their Tweets. Maybe you/@Horse_ebooks are one of the some?
posted by dtp at 4:27 PM on January 4, 2013

Best answer: Per discussion here it's just not possible.
posted by dhartung at 4:57 PM on January 4, 2013

There's a pastebin archive of 3,200 tweets from 07/31/11 to 03/30/12, but beyond that you may need to WHEAT IN 1840 AND 1940. EACH f REPRESENTS ONE HOUR. 1840 1940 tttttttttttttttttttttt ttt tttttttttttttttttttttt tttttttttttttttttttt How Is
posted by arco at 5:06 PM on January 4, 2013 [6 favorites]

Response by poster: That got me back a lot further than I was arco. my heart, thank

I guess if anyone has any other small archives like this that they could share I'd be thrilled to have them.
posted by six-or-six-thirty at 5:53 PM on January 4, 2013

This is absolutely possible but is going to cost real money (and likely require some technical skills). What you need to do is run a historical search of the data from the full twitter fire hose (all public tweets not marked as spam).

Datasift Historics and Gnip Replay are the two product lines that would give you historical twitter search capability.

These are enterprise tools that carry enterprise prices, so not likely to be the solution you were looking for. But hey, it's possible.
posted by reeddavid at 8:01 PM on January 4, 2013

People who work at Twitter can absolutely pull these, but it's not standard policy for civilians. If your project is something really cool, you might try contacting them and seeing if you can get some goodwill access. Probably would need the account holder's permission as well.
posted by amoeba at 8:45 PM on January 4, 2013 [1 favorite]

Also, this search at Topsy looks like it can pull lots of tweets, over 20K, but from any account mentioning it: http://topsy.com/s/%40horse_ebooks/tweet — You might try the folks at Topsy to see if they've got an archive...
posted by amoeba at 8:48 PM on January 4, 2013 [1 favorite]

Is this an example of why one to many shortform messaging should be a feature of the internet not a company? try https://support.twitter.com/forms and submit a request, most likely they will give you hand with what you are looking for.
posted by specialk420 at 10:34 PM on January 4, 2013

I would try asking @Horse_ebooks for a copy, if downloading is turned on for their account.
(They can see if it's turned on by looking at the bottom of the "Settings" page, right below the "Country" selection and above "Save Changes" button. It's not turned on for my account yet, and I'm not sure how fast they're rolling it out.)
posted by Ookseer at 12:29 AM on January 5, 2013