Finally downloading all that e-mail

December 2, 2023 5:58 PM

I, and honestly several people I know, have a lot of e-mail stored on my web hosting provider's server. I also have some massive Thunderbird archives somewhere. Under Windows, will MailStore or Mailbird or something similar let me download, store, and later search and read, all those messages (together) with peace and harmony reigning supreme? Do you recommend any specific mail archiving program?

I can use POP or IMAP to download - no access to the mail files in the file system on the web hosting platform.

The Thunderbird files are on old machines. I expect to copy them to a new location to properly archive them, so MailStore's insistence on at least Windows 10 isn't a disqualifier.


I do want a local / laptop based solution if possible, but am open to learning about other solutions.

TimHare apparently loves MailStore - how does it compare to Mailbird?


Thanks!

posted by amtho to Computers & Internet (11 answers total)

Won't Thunderbird let you do this? I used Thunderbird to consolidate three IMAP accounts to one, putting, among other things, a large gmail archive onto fastmail's servers where I can search and use it from anywhere. Thunderbird remains the backup.
posted by deadwax at 6:17 PM on December 2, 2023 [1 favorite]

Response by poster: I used to rely on Thunderbird. a) It has decided to compress/delete messages with zero explanation in the past (years ago, but still); b) apparently there are size limits to the folders; and c) it seems likely that a program designed to be an archive is better at being an archive.

For example, while I am super double plus in favor of storing things in easy-to-read plain-text files, it occurs to me that 1) it might be possible to compress that a bit if the program is smart about encoding chars, and 2) if the archive program uses a more databasey storage method, then searching might be a lot faster. Also, apparently, at least one of these programs will look for search terms inside of attachments.
posted by amtho at 6:22 PM on December 2, 2023

Possibly Mailstore Home?
posted by nostrada at 3:53 AM on December 3, 2023 [1 favorite]

if the archive program uses a more databasey storage method, then searching might be a lot faster

Thunderbird indexes mails in its global database to make searching faster.
posted by Klipspringer at 4:15 AM on December 3, 2023

And if you're on Windows, you can turn on the file compression attribute on any folder containing MBOX files. This makes Windows compress them on disk and decompress them transparently on access. Text-heavy formats like MBOX compress very nicely with this scheme.
posted by flabdablet at 7:06 AM on December 3, 2023

Response by poster: Ugh I want to like Thunderbird, I used it for years, but I can't trust it right now. I've seen reports of other people losing messages too. It was connected to Thunderbird's automatic "compress folders?" thing that sounded innocuous -- I _think_. I really want those messages back. It's been 10 years or something (which I know means the software might have changed).

Also: since I'll be downloading thousands of messages, some software that handles that well in case of interruptions, etc. would be nice.
posted by amtho at 12:10 PM on December 3, 2023

Thunderbird handles interruptions fine, as will any other sane mail program.

If you don't trust it you don't trust it though. I'd suggest adding any closed source / for profit archiving program to that list list of things not to trust; you need multi decade long archiving, longer than most startups hang around. Double or quadruple that mistrust if they don't use an open file standard for the mail they keep.
posted by deadwax at 1:27 PM on December 3, 2023

I have no doubt that using Thunderbird has caused people to lose mail messages. The only way I've actually seen it do that, though, involves IMAP accounts where some server-side change has bollixed up the message IDs. I have not seen it drop non-deleted messages during folder compaction; I think the most likely pathway toward it doing that involves undetected filesystem corruption.

Thing is, though: if your strategy for preserving the data you archive is sound (multiple copies, with per-file checksums kept to detect corruption, the oldest copies regularly transcribed to fresh media, and the oldest media discarded only when the checksums test OK on all of the new copies) and the formats you archive in are sane (preferring widely used general-purpose formats, possibly with some task-specific specialization, to bespoke proprietary nonsense) and your strategy for accessing archival material involves making a copy from the archive before doing anything else with it, then it really doesn't matter how buggy or destructive or loaded up with ransomware your search tools are.
posted by flabdablet at 9:10 AM on December 4, 2023

Response by poster: Thanks. Given all that, and given that I was hoping for advice on which archiving software to use (like choosing a programming language, for any choice anything is possible, but some things are easier) - I was hoping to learn "Do you recommend any specific mail archiving program?"
posted by amtho at 11:42 AM on December 4, 2023

I use GMVault to archive my Gmail to plain text eml files, plus metadata. It's a python script that hasn't been maintained since 2016 and now requires all sorts of hackery to comply with Google's security requirements. So, while it's still just-about-working for me, I'd stay away from that one!
posted by Klipspringer at 11:53 AM on December 4, 2023

advice on which archiving software to use

I'm personally a big fan of Borg.
posted by flabdablet at 5:59 PM on December 4, 2023