How to recover lost email in Thunderbird, OSX?
June 14, 2006 12:21 PM Subscribe
Lost email in Thunderbird 1.5.0.4, OSX 10.4 (Tiger), caused by changing the settings in the news reader.
I changed the settings in the Thunderbird news reader to not keep news (usenet) messages older than 10 days. Thunderbird then got rid of all of my emails as well older than 10 days (thats a lot of email, and some of it I'd rather not lose!). It deleted it from inbox, sent mail, and all saved folders -- any email older than 10 days! I suspect it's there somewhere since my free disk space didn't increase noticeably, and I'd guess there were several gigs of email, at least. This happened a few days ago and I only just clued in to what I did. The email isn't in the Thunderbird trash, nor in the system trash. I am a <1 year mac user so forgive me if this is terribly obvious, but, where might my email be? Can it be recovered? How might I recover it?
I changed the settings in the Thunderbird news reader to not keep news (usenet) messages older than 10 days. Thunderbird then got rid of all of my emails as well older than 10 days (thats a lot of email, and some of it I'd rather not lose!). It deleted it from inbox, sent mail, and all saved folders -- any email older than 10 days! I suspect it's there somewhere since my free disk space didn't increase noticeably, and I'd guess there were several gigs of email, at least. This happened a few days ago and I only just clued in to what I did. The email isn't in the Thunderbird trash, nor in the system trash. I am a <1 year mac user so forgive me if this is terribly obvious, but, where might my email be? Can it be recovered? How might I recover it?
Take all of this with a grain of salt, because I haven't actually ever had to recover deleted Thunderbird mails before and it may be wrong; but one of the things I like about Thunderbird is that its basic storage format for mail is text files, not some bizarro binary thing like Outlook Express uses - so recovery from disasters is much more likely to work.
All your mails, exactly as retrieved, complete with all headers, simply get appended to each other in one huge text file. There's one of these for each Thunderbird folder (Inbox, Sent, Trash etc.) For each mail folder, there's also a completely separate text file containing indexing information, which Thunderbird uses to figure out which parts of the actual mail file to show you, which mails you've read etc.
When Thunderbird deletes mails, it just changes the index files - the actual mail files aren't touched until you also do a "compact folders". As far as I know, Thunderbird will never do a "compact folders" behind your back; you either have to do it explicitly, or at least consent to it by clicking an OK button. So your mails should, in theory, be completely recoverable.
I think the right things to do would be:
1. Without Thunderbird running, make a backup of your existing Thunderbird profile folder, so that if this procedure hoses you up completely, at least you can get back to where you are right now and await more competent assistance.
2. Start Thunderbird, turn off the older-than-10-days thing, and quit Thunderbird again.
3. Delete all the index files from your Thunderbird profile.
4. Start Thunderbird again, which should make it rebuild those indexes by scanning the original mail files.
I've only used Thunderbird on Windows, but I'm assuming it works pretty much the same on Mac with some file location differences. If you poke around in Account Settings, you should be able to find the storage location for the various mail account folders. There will be one of those for each account, and they'll all be inside a folder called (something mac-specific)/profiles/(random gibberish).default/
That's your Thunderbird profile folder, and that's the one you should make a complete copy of, somewhere quite completely else.
Inside the profile folder, you should find a subfolder called "mail"; inside that, a subfolder for each of your mail accounts (probably named after the pop3 server associated with the account) and another one for "Local Folders". Inside each of those, there will be a bunch of files whose names match your Thunderbird folder names.
Some of these will have no filename extension (e.g. Inbox), and be very big. Those are your actual mail files. Then there's a set of smaller files with .msf extensions (e.g. Inbox.msf). Those are Thunderbird's index files, which are the ones you want to delete in step 3.
After letting Thunderbird rebuild the indexes in step 4 (which may take a good long while, if you have gigabytes of mails) you will probably have to do a lot of marking-as-read, and maybe a lot of dragging-out-of-trash - but at least you'll have your mails back.
And once your world's been put back together, make another full backup of the Thunderbird profile folder before fiddling about with your newsreader settings again :)
posted by flabdablet at 8:06 PM on June 14, 2006
All your mails, exactly as retrieved, complete with all headers, simply get appended to each other in one huge text file. There's one of these for each Thunderbird folder (Inbox, Sent, Trash etc.) For each mail folder, there's also a completely separate text file containing indexing information, which Thunderbird uses to figure out which parts of the actual mail file to show you, which mails you've read etc.
When Thunderbird deletes mails, it just changes the index files - the actual mail files aren't touched until you also do a "compact folders". As far as I know, Thunderbird will never do a "compact folders" behind your back; you either have to do it explicitly, or at least consent to it by clicking an OK button. So your mails should, in theory, be completely recoverable.
I think the right things to do would be:
1. Without Thunderbird running, make a backup of your existing Thunderbird profile folder, so that if this procedure hoses you up completely, at least you can get back to where you are right now and await more competent assistance.
2. Start Thunderbird, turn off the older-than-10-days thing, and quit Thunderbird again.
3. Delete all the index files from your Thunderbird profile.
4. Start Thunderbird again, which should make it rebuild those indexes by scanning the original mail files.
I've only used Thunderbird on Windows, but I'm assuming it works pretty much the same on Mac with some file location differences. If you poke around in Account Settings, you should be able to find the storage location for the various mail account folders. There will be one of those for each account, and they'll all be inside a folder called (something mac-specific)/profiles/(random gibberish).default/
That's your Thunderbird profile folder, and that's the one you should make a complete copy of, somewhere quite completely else.
Inside the profile folder, you should find a subfolder called "mail"; inside that, a subfolder for each of your mail accounts (probably named after the pop3 server associated with the account) and another one for "Local Folders". Inside each of those, there will be a bunch of files whose names match your Thunderbird folder names.
Some of these will have no filename extension (e.g. Inbox), and be very big. Those are your actual mail files. Then there's a set of smaller files with .msf extensions (e.g. Inbox.msf). Those are Thunderbird's index files, which are the ones you want to delete in step 3.
After letting Thunderbird rebuild the indexes in step 4 (which may take a good long while, if you have gigabytes of mails) you will probably have to do a lot of marking-as-read, and maybe a lot of dragging-out-of-trash - but at least you'll have your mails back.
And once your world's been put back together, make another full backup of the Thunderbird profile folder before fiddling about with your newsreader settings again :)
posted by flabdablet at 8:06 PM on June 14, 2006
Response by poster: fladbalet -- thank you so much. Your answer is almost perfect! I asked this over at Mozilla Forums as well.
It seems it does actually change the text file of each folder's emails slightly, by changing the value of the line
X-Mozilla-Status: 0000 to some other value (usually 0009 but others also). if you change that value back in the textfile, then delete the .msf file, then T-Bird will re-integrate those emails back into their folders -- its like freakin' magic.
As I note over there, my current problem is that some of my folders are huge - sent-mail, my archive, my current inbox are all 300 to 600 megs because of all the attachments I get for work.
TextEdit chokes on these very large files when you do a replace ( I have 1.5 gigs of ram in this powerbook). I am hoping someone on the other board (or here, if they know) can suggest a more robust text editor or a script I can run without opening the files to do the replace with wildcards.
Anyway, thanks very much fladbalet and cgc for your responses. I really appreciate it and I am optimistic I can get all my emails back. *phew*
posted by Rumple at 12:29 AM on June 15, 2006
It seems it does actually change the text file of each folder's emails slightly, by changing the value of the line
X-Mozilla-Status: 0000 to some other value (usually 0009 but others also). if you change that value back in the textfile, then delete the .msf file, then T-Bird will re-integrate those emails back into their folders -- its like freakin' magic.
As I note over there, my current problem is that some of my folders are huge - sent-mail, my archive, my current inbox are all 300 to 600 megs because of all the attachments I get for work.
TextEdit chokes on these very large files when you do a replace ( I have 1.5 gigs of ram in this powerbook). I am hoping someone on the other board (or here, if they know) can suggest a more robust text editor or a script I can run without opening the files to do the replace with wildcards.
Anyway, thanks very much fladbalet and cgc for your responses. I really appreciate it and I am optimistic I can get all my emails back. *phew*
posted by Rumple at 12:29 AM on June 15, 2006
You want robust? They don't come more robust than sed. In particular, because sed makes exactly one pass through the file it's editing from start to finish, it can handle enormous files as easily as small ones. And because you're running a Unix, you already have sed installed.
Try this, in a terminal window:
Translated into English:
Set the working folder to the one containing the mail files;
For each of the files 'Inbox', 'Sent', 'Trash' and 'Unsent' (plus any others whose names you tack onto that line) do as follows:
Rename the existing file to Original Inbox, Original Sent etc;
Make a new version of the file by feeding the original version through sed (the Stream EDitor), and on each line beginning with X-Mozilla-Status:, change any run of digits at the end of the line to 0000.
All punctuation and spacing must be entered exactly as shown, or Bad Things may happen. In particular, double quotes and single quotes do not mean the same thing, and where you see '^X' at the start of the sed script, you should enter a caret (^, shift-6) followed by an X; it doesn't mean Control-X.
posted by flabdablet at 5:16 AM on June 17, 2006
Try this, in a terminal window:
cd '/path/to/folder/with/mail/files'
for file in 'Inbox' 'Sent' 'Trash' 'Unsent'
do
mv "$file" "Original $file"
sed '/^X-Mozilla-Status:/s/[0-9]*$/0000/' <"Original $file" >"$file"
done
Translated into English:
Set the working folder to the one containing the mail files;
For each of the files 'Inbox', 'Sent', 'Trash' and 'Unsent' (plus any others whose names you tack onto that line) do as follows:
Rename the existing file to Original Inbox, Original Sent etc;
Make a new version of the file by feeding the original version through sed (the Stream EDitor), and on each line beginning with X-Mozilla-Status:, change any run of digits at the end of the line to 0000.
All punctuation and spacing must be entered exactly as shown, or Bad Things may happen. In particular, double quotes and single quotes do not mean the same thing, and where you see '^X' at the start of the sed script, you should enter a caret (^, shift-6) followed by an X; it doesn't mean Control-X.
posted by flabdablet at 5:16 AM on June 17, 2006
This thread is closed to new comments.
posted by cgc373 at 1:57 PM on June 14, 2006