Considerations when adding a periodic email update function to my website (with PHP's mail() function)

July 21, 2008 2:56 PM

I'm planning to add a feature to my website that will allow users to request periodic updates by email. I've cobbled together a little PHP script that uses the mail() function. It works perfectly when I test it myself, but I have no idea how this will work when released to a signficant number of users.

One problem - I don't know how many users I'll have, so I'm trying to keep my up-front costs to a minimum. Optimistically, I'll have a few hundred users. In my wildest dreams, I'd have a few thousand.

Another problem - the email will be dynamically generated based on user-entered criteria - i.e., the user will log in, and enter some parameters. A php page will pull the appropriate data from the MySQL database, assemble the email, and send it out. Also, the emails will be sent on demand (not according to some predetermined schedule). So, I think that rules out something like Campaign Monitor's newsletter service.

My site is hosted at Media Temple, on a Grid Server.

Is this a disaster waiting to happen, or is that kind of demand/load trivial?

If the former, is there a better way to handle this? A reliable third-party service?

Many thanks in advance for any advice!

Cheers,
Matt Stuehler

posted by stuehler to technology (8 comments total)

Clarifying question: do you mean that you would be sending an e-mail to everyone who subscribes at once?
posted by fogster at 3:43 PM on July 21

The load's trivial, but mass emails can be tricky - they're likely to get tagged as spam.

Is there any way you can push people towards an RSS feed of the same information? Offering both services will help lighten the load on the email service.
posted by Leon at 3:47 PM on July 21

fogster, Leon,

Many thanks for your quick replies.

Some clarification: the emails are totally personalized, so I don't think I can use an RSS feed, and they won't be published at once.

Specifically - the app is an "office football pool manager" (After years of trying the extant offerings, I've gotten fed up with their shortcomings and decided to build my own...)

Each week, participants will submit their picks for that week. Any time they like, they can have a record of their picks for that week emailed to them (e.g., as a confirmation, backup, audit-trail, whatever...)

Participants may submit picks, request an email confirmation, then later decide to change a pick or two, and request another email with the new picks.

So, each email will be different, and they won't all be sent at once (people can request them whenever they enter their picks, and the email should be sent promptly). It really needs to be "emailed on demand."

A problem I'm anticipating is that few people will request emails during the middle of the week, but in the few hours before Sunday's first game, I'd expect demand to be relatively high.

So, in short, would 500-1,000 separate executions of a PHP script that uses the mail function per hour cause any trouble? Or, is that trivial?

(Obviously, for a major site, it's infinitesimally trivial, but I don't know what to expect from my $20/month hosting provider...)

I've thought about queuing/batching the requests, but I don't know if that lessens or worsens the problem. Is there overhead associated with establishing a connection to the mail server that I could reduce if I did this in batches?

Again, many thanks for any advice and guidance!
posted by stuehler at 4:44 PM on July 21

I would personally use a mailing list (like mailman) and send an email to the mailing list from PHP. I wouldn't try to send 1000+ emails from mail(). But thats just me.
posted by SirStan at 6:32 PM on July 21

Build it, run it. If you get problems, shift from mail() to another method. If you're worried, mail support at your host, let them know you want a PHP script to send 500 emails, and ask them if that will trigger any alarm bells at their end. I think you'll be fine.
posted by Leon at 1:14 AM on July 22

I did a similar mass mailing thing last time on a shared server, and how I did it was:

1. Save all the emails (email_to, subject, message, etc) into a MySQL table. There could be thousand of rows, one for each email.

2. Using CRON, schedule for the mails to be sent out during off-peak hours to reduce server load. I set it to send emails between 8pm to 8am, or some odd off-work hours. For each mail delivered, just remove it from the MySQL table.

3. Instead of mail() - I used phpMailer, the only reason being that it offered a lot more customization functions that I needed over the long run.

Hope this helps!
posted by arrowhead at 2:21 AM on July 22

A few thousand emails in an hour is nothing, unless they're expensive to generate, or very big. I wouldn't worry too much about the load, certainly not if it's just the occasional spike.

I would be more concerned about making sure what you're sending out is Correct; making sure you verify emails before sending more than their signup/email change confirmation (using a secret URL, or reply address which only you and the recipient know); ensuring your From address is routable (even if you only send it to /dev/null because replies don't make sense, spam filters like addresses to work); using Return-Path to track bounces (optional, but might be a good idea); and of course, making it clear why the emails are being sent, who you are, how to make them stop, etc.

Also consider testing with common spam filters; SpamAssassin will give you a report about what it considers spammy, which can be useful. You might also consider setting up DomainKeys and/or SPF, which can make filters happier.

If your database is big, or your queries complex, you probably want to be wary of that too; a 4k email isn't going to be a problem, but if it takes 30 seconds of MySQL thrashing to make it, and at peak times you're making one every few seconds.. that might be a problem ;)
posted by Freaky at 9:44 AM on July 22

I just wanted to thank everyone for the excellent responses. You've raised some issues I hadn't thought about, and provided a few suggestions that are spot on.

I took Leon's advice, contacted Media Temple, and told them what I was planning.

They told me that, for Grid Server clients, there's a limit of 50 emails per minute, and 500 emails per hour. If I upgrade to their Dedicated Virtual (DV) product, I'd get higher limits.

For now, I think those limits are high enough for my plans. I'll take Freaky's advice, and make sure my queries are optimized, and that the emails are as streamlined and minimal as possible.

Thanks again.
posted by stuehler at 11:47 AM on July 23