How do you get mail in "mbox" fomat into Gmail via pop?

March 4, 2007 11:01 PM

How do you get mail in "mbox" fomat into Gmail via pop?

I am trying to move my mail from an IMAP webmail account into gmail and then back up the whole thing via a pop client. So there are two steps,

Step 1: Getting the mail to gmail.
I've currently got all the webmail downloaded through thunderbird and saved in local folders in mbox format. The first thing I found was "gmail loader". This program sends all the messages in the T-bird mbox file to gmail one by one. While this works pretty well, the date displayed by gmail are all wrong and the messages are in reverse order. Now I'm trying to use gmail's new mail fetcher function to pull the messages off a pop server as this seems to preserve the dates properly.

To this end, I set up a pop server on my computer with a program called "Personal Mail Server Pro". The server stores the messages in ".msg" format and works fine and gmail can extract messages that are on the server if they arrived there as an email. The problem is that when I convert the mbox to ".msg" files w/ Thunderstor and manually put them in the mail directory, these messages don't get served properly. Clients see them as new messages but they don't download and things just hang.

I investigated the ".msg" files created by the pop server and the ones extracted from the mbox file and they appear to be of two different formats. The ones created by the server look like gibberish in notepad while the ones extracted by Thunderstor (that don't work), look like normal legible emails.

So basically, how do I get these messages onto a pop server without forwarding them? Can I convert the ".msg" files to the server format somehow and just drag them into the directory in Windows Explorer? Is there a way to copy the messages onto the pop server through a client like you can do w/ IMAP emails?

Step 2: Backup the emails that were already on gmail locally
I was able to download all the messages that were already on Gmail w/ outlook express (T-bird kept timing out), but the drafts and sent mail ended up in in the local inbox in addition to the normal inbox content.

Is there a way to get them to download in the appropriate folders? And if not, is there a way to label the messages by in gmail so that those labels persist when transferred to a client and I can then reorganize them into inbox, drafts, and sent mail accordingly?

Thanks!

posted by Popcorn to computers & internet (6 answers total)

Mailtraq is a bit of a beast, but it serves mbox mailboxes in POP3. the fully-functional trial version should do what you need within the 30 day period.
posted by holgate at 11:32 PM on March 4, 2007

I think I figured out the second part: stuff that is in the gmail trash doesnt get pulled by the client. If I mark the entire inbox and draft folder as trash and then download the mail, it only gets the sent mail.

The first qustion still stands though.
posted by Popcorn at 11:51 PM on March 4, 2007

Oops, I didn't see your answer when I posted that last one. I'll give that a try. Thanks Holgate!
posted by Popcorn at 11:52 PM on March 4, 2007

Whenever I wire Gmail into a POP3 client, I always set up a message filter to match the sender against the Gmail account itself, mark any such message as read, and move it to the local Sent box. That keeps the local and remote Sent boxes pretty much in sync. I don't generally leave a lot of things lying around in Drafts, and it hadn't occurred to me that Gmail would POP those as well; I'd probably just move mine by hand.

As far as I know, the Gmail labelling system is web-interface only. You can use FreePOPs to turn the web interface into a local POP3 server for download and backup purposes; its Gmail plugin supports per-label downloads.

FreePOPs is actually a very useful tool for making almost anything accessible via POP3. By writing a small plugin (in Lua, a very straightforward scripting language) it could even be made to serve messages from an mbox as POP3 mails, which would let you use it with the Gmail POP3 import thing to do your initial upload.
posted by flabdablet at 5:48 AM on March 5, 2007

Mailtraq is working perfectly, good stuff! Now I have a follow up: How do I get the sent mails that gmail pulls off of the mailtraq pop server to appear as sent mails?

Since gmail appears to keep all the mail in one directory, It must have some way of determining if somthing is sent mail. My first thought was that it made all mails with the gmail address in the "from" considered "sent mail". I was led to believe this because Gmail loader changes the "from" field in the email to the gmail address if you ask it to send stuff to the gmail sent mail area.

However, gmail also knows to put the messages that I ask it to send from a non gmail address in the sentmail folder and the "from" field on those remains as the address from the non gmail domain. So the critical field is not the "From" field.

I did some snooping and compared the files and it appears that gmail makes the distinction as followes:

IF "From" = "Gmail Address", message = sent mail

But also, if gmail is sending mail to look like it is coming from a non-gmail domain, it adds a "sender" field containing the gmail address. And, when differtiating, gmail checks the "sender" field first before looking at the "from" field:

IF "sender" = "Gmail Adderss", message = sentmail

Does this sound reasonable?

If this is correct, Is there an easy way to put this "sender" field in all the local sent messages so that when gmail hits the pop server, they end up in sent mail?
posted by Popcorn at 1:07 AM on March 6, 2007

After further investigation, that stuff about the "sender" field doesn't appear to be true. It seems that the way gmail knows the mail it sends to look like it's from another domain belongs in sentmail is that if the "from" field matches any of the "send as" addresses or the actualy gmail address, then it is sent mail.

So, as with gmail loader, any mail gmail pulls from a POP server that meets the "sentmail" criteria ends up in both the inbox and the sentmail and then, if you "archive" the message it disappears form the inbox, remains in the sentmail, and starts behaving like any other piece of sentmail in gmail. It seems that the reason mail sent from gmail doesn't end up in the inbox is b/c gmail automatically archives sent messages.

I haven't actually done this w/ my real account, but I've been experementing with throwaway accounts and I think I've made sense of the mechanisms at work. If all this stuff is in fact true, then gmail stores all messages, both inbox and sent, in one directory and uses this rule to sort them in to the appropriate catagories. Probably using some flag to do the same with drafts.
posted by Popcorn at 1:58 AM on March 6, 2007