How can I get my Twitter Archive into a CSV?

January 12, 2024 9:16 PM

I've asked for, and gotten, my Twitter archive but I was hoping to transfer it into a CSV.

I requested my Twitter Archive and they give it in a JSON format, I believe, but I'm hoping that I could (somewhat) easily convert it into a CSV format.

posted by jaybeans to Technology (6 answers total)

How handy are you with Python? Python has a CSV modile and a JSON module in the standard library. I don't know what format Twitter's data is using, but in general you should be able to read it as some combination of dictionaries and Lists -- I'd guess somewhere in the data there is a list of all the tweets, each as a dictionary, so you could just iterate over the list, pull whatever data you want out of the dict, and slap it into CSV rows.
posted by Alterscape at 9:28 PM on January 12

If your Twitter archive looks like mine, you’ll have a file called `data/tweets.js` which is mostly JSON but in a Javascript wrapper. Mine starts with a variable assignment on the first line, like this:

window.YTD.tweets.part0 = [
  {
    "tweet" : {

Each element of the top-level array is a JS dictionary with a `tweet` property. Inside that dictionary is a bunch of nested stuff you’ll need to flatten to make a CSV. This brief guide from The Carpentries shows an example and describes its contents, and this documentation from Twitter explains it in more detail.

To convert it to CSV, you’ll need to decide what you want to do with the nested properties. Since it’s already valid Javascript you might do this by opening the archive’s included HTML file and navigating the tweet array in your browser console.

To give you a sense of what else is in there this is the README file that came with my archive.
posted by migurski at 10:33 PM on January 12

You could try a site like this to convert the file in the browser. No affiliation, can't say if they're good on privacy etc. -- it's just the first hit for "flatten json to csv web".

"Flatten" is a useful keyword here, since you're converting a nested format to a column based format.
posted by john hadron collider at 6:43 AM on January 13

If you happen to pay for ChatGPT Pro, the "Data Analyst" feature will also take any file and write code to convert it to other formats for you based on a back and forth conversation, which can give you more control over the output.
posted by john hadron collider at 6:46 AM on January 13

I am happy to write a python converter program for you if that's helpful. Similar to what John suggested but I'm not chat gpt. At least I don't think I am.
posted by KevCed at 6:52 AM on January 13

Apologies for just reporting results from the obvious Google search but this tool is promising: Convert JS from Twitter archive to CSV. I tested it on my archive of 40,000+ tweets and it seemed to work, I could import the CSV into Google Sheets and a quick browse was plausible.

Note that tweets are very complex media. Whatever reads this CSV file needs to do quoting correctly. You won't get images, videos, etc. And not sure what it does with unusual tweet formats like megatweets or polls. But the basic text seems to work. There's some notes on exactly what metadata is carried over.
posted by Nelson at 7:38 AM on January 15