Mailman, have you really got my back?

March 28, 2011 2:29 PM

How can I tell if my e-mail list is underperforming, delivery-wise?

So, a few months ago, I dragged my local spinning and weaving guild into the digital age by setting up a website and an e-mail announcement list using Mailman.

The e-mail list is low-volume, with just 70 users, but each month after the newsletter gets e-mailed via the list, I spend a fair amount of time corresponding with a handful of people (different people each time) who didn't get the newsletter and are concerned that the new system is not as reliable as the old system of cutting and pasting e-mail addresses into the To: field of your e-mail to the guild members.

I am new to administering mailing lists, but feel like a 2-3 percent non-delivery rate is not too bad, given that servers are temperamental on the receiving end as well as the sending. Are my standards too low? Can I improve delivery? Can I set up Mailman to give me some sort of bounce report each time an e-mail goes out, rather than just when the excessive bounce filter is applied? What else should I be looking for to know whether this is a problem, or a "problem?"

posted by deliriouscool to computers & internet (9 answers total)

You're correct that a 2-3 percent bounce rate is really not bad, and well within what should be expected. I'm not familiar with Mailman, but the program I use (mailchimp) gives a report that distinguishes between hard and soft bounces. Soft bounces might still get delivered, so your delivery rate may go up after time.
There's not too much you can do to improve delivery, other than ensuring that you don't have any spammy-sounding subject lines. Relying too much on HTML or having large sized e-mails will also lead to trouble. Other than that, it's mostly out of your hands.
posted by Gilbert at 2:47 PM on March 28, 2011

Mailman's auto bounce processing refers to mail sent by list members to the list--not undeliverable mail properly sent to the list. Still, can you see Bounce Processing on your admin panel? Do you have a list admin panel? Is auto bounce processing turned on? Do you see any options about Notifications (to the list owner)?

Have you noticed any patterns among those who say they aren't receiving messages? Are they all aol.com addresses or @yahoo.com or anything like that? It could be a problem on the user end where a spam filter is catching mail from the mailing list. Even if you set up a bounce report, it wouldn't report messages that ended up in spam folders or were properly delivered but not properly read.
posted by mattbucher at 3:12 PM on March 28, 2011

E-mails are generally 1MB or smaller, and attachments are almost always PDFs. There are few trends among instances of non-delivery; I have an AOL user one month, and I also have problems reported from a user who passes e-mail received via a Roadrunner account to her smartphone.

So there are people not savvy enough to realize that problems can happen on the receiving end, and people savvy enough to manipulate e-mail accounts successfully.

I hate to even ask, but if I moved the list off my own server to GoogleGroups, would that be better/more reliable?

> Still, can you see Bounce Processing on your admin panel? Do you have a list admin panel? Is auto bounce processing turned on? Do you see any options about Notifications (to the list owner)?

Yes to all of this - I get notifications and I've lowered the bounce score to 2.0 from 5.0 in an effort to intercept more problems earlier, but I fear that much of this may be happening on the recipients' end, and if that's so, it seems the only way to handle it is move the list to a third-party site like Google or Yahoo Groups - so I can count on their reliability and pass the blame if something goes wrong. I hate to do that, though.
posted by deliriouscool at 3:32 PM on March 28, 2011

(Just want to recommend MailChimp for email marketing - it would be free for your size list, and their support and documentation are really great. They have detailed 'soft-bounce' and 'hard-bounce' reports, and you can track exactly who has received, opened, and clicked on your emails in real-time.)
posted by ella wren at 4:25 PM on March 28, 2011

You're going to run into trouble using any sort of blast email tool with attachments. Honestly, a list of this size probably is best dealt with in Outlook (or whatever) without bringing a broadcast email service into it.
posted by Ignatius J. Reilly at 6:32 PM on March 28, 2011

Yep. MailChimp. That is all.
posted by PareidoliaticBoy at 7:56 PM on March 28, 2011

You might insert a footer that asks list subscribers to add you to their address books. If their mail is going in to the spam folder, that will fix that.
posted by Gilbert at 10:16 PM on March 28, 2011

I think using the old massive To: list was probably losing individual emails from time to time, it was just less noticeable.

Reliable mail delivery in the modern world is hard. AOL will randomly decide that you're a spammer and blackhole all your email, as will other large email services & getting off the blacklists can be an huge amount of hassle. There's a reason people pay for services like MailChimp.

Moving to GoogleGroups probably won't help much: many sites view Google as suspicious by default since a large amount of spam is sent via Google's mailservers. (Presumably via compromised Google mail accounts.)
posted by pharm at 1:16 AM on March 29, 2011

[IANAEDE (I am not an email delivery expert) but I have some experience with email marketing delivery.]

I agree about the attachment being an issue -- the PDF attachment and size are going to be your biggest roadblocks. They won't help the spam score for the emails.

Can you access/read the bounce emails you get back from the ISPs? This is a big advantage of running the mailing list yourself -- it's time-consuming if you have a huge email list, but 2-3% of your 70 subscribers shouldn't be that bad. What are the bounce reasons? Do see any "missing" bounce messages from people who said they didn't get the newsletter? etc. Actually seeing the bounces should give you a better idea of what might be going wrong and if/why the email was rejected by the recipient's ISP. For example, it could be a spam rejection, or it could be a general failure where the recipient's email server was down temporarily and wasn't responding, or it could be that the user's mailbox was full, etc.

If your goal is to minimize the time spent administering the list and troubleshooting delivery issues, then you'll probably have better luck with some or all of these options:

- Instead of attaching the PDF, put it up somewhere on the guild website. Create a basic email. Include text with highlights of the month's newsletter, and add in the link to the page on the guild website, where the members can download the PDF at their convenience. Generally speaking it's better for the email to have some related text with a download link, rather than just a link.

- Compose the body of the email in plain text, so it works well for phone email too (having a rich-text version with sparse HTML formatting such as bold and italics is OK, but if you do not have to use an HTML design, save yourself the time and effort of HTML/email client testing and avoid it).

- Use an email marketing company as already mentioned (I haven't used MailChimp to send out any campaigns, but their web interface seems pretty easy to work with in terms of setup). These email marketing services deal with email delivery all day long, so their servers ideally will be fine-tuned and authenticated for optimal delivery, and will have a good reputation with the various ISPs. Also, their tools will give you much more data about bounces as mentioned already in the thread, and optimally will make things easier for you by automatically categorizing the types of bounces received. (If you do go with another company, be sure that you can see the raw bounce messages. Not all provide this option, but I believe MailChimp does.) Let your subscribers know about the move to the new company and have them confirm their subscriptions.

As mentioned earlier: with email marketing services, you will be able to track who has clicked the download link, if you have link tracking on. However, just a note that the number of email message "opens" is not always accurate because it depends on how the subscriber's email client works. Also, seeing the stats for actual inbox delivery (e.g. if the message did not bounce but went into the spam folder, it wouldn't fall under inbox delivery) will likely not be offered for free.

if I moved the list off my own server to GoogleGroups, would that be better/more reliable?

If you don't need a discussion list but just want to send out a newsletter, I still think a dedicated email marketing service will be more helpful for what you want to do. Having the automatic bounce analysis is going to be really helpful, and sending from servers with established, good reputations will be really beneficial, too. (On preview, what pharm said.)
posted by rangefinder 1.4 at 1:56 AM on March 29, 2011