die spam die.

February 24, 2006 9:04 AM

Help me deal with email: I have a couple domain names, and I get obscene amounts of spam and garbage email. I'd like to setup a system to help deal with all of it, but I'm not very familiar with server-side email setup beyond the basics.

Basically what I'd like to do is setup rules such that if an address matches (something like *@mydomain1.com or [e-mail]) it could be either sent through, dropped, or sent through a sender verification step, that would require someone with the FROM address to reply, or go to a URL and type in one of those captchas. It would also be nice if someone with a particular email address would only have to authorize once.

I've given up on the idea of AI-based spam filtering, which just doesn't work when you have a 10000:1 signal to noise ratio.

Following that step, it would be nice if I could filter mail into different folders and then read everything with IMAP and keep it on the server.

Now, I know the tools to do this exist somewhere, but where, and what, are they? I have a basic Debian installed on my virtual host. and the mail server is exim (And yes, obviously I'll clear it with my hosting provider before sending out large amounts of email :P)

posted by delmoi to computers & internet (15 comments total)

I've had my own domains for email since around 1998, and have had to go through a few changes during that time.

One of the best things I did was dumping the catch-all address. I know it's handy to have, but getting rid of it helped things immensely.

For a while I was using spamassassin, and training the filters quite regularly for both spam and non-spam messages. Most messages came straight into my inbox, so everything in the spam folder was shown to spamassassin as spam, and things in any other folder had been seen and verified by me, so were used to train spamassassin what not-spam looks like. SA works a lot better when trained, and this system didn't take any of my time after the scripts were written.

More recently I've simply dumped the storage part of the mail server, and have all my mail redirected straight to a gmail account. I read most of my mail from my cell phone through POP access to gmail, can archive and open up items on a computer that are too complex for my Sidekick, and gmail has a second layer of spam filtering that does a pretty good job. This spam filter is maintained by a company that receives a lot more messages than I do, and the entire userbase participates in training the spam filters.
posted by tkolstee at 9:55 AM on February 24, 2006

It doesn't support captchas, but tmda [Tagged Message Delivery Agent] will give you a challenge/response system that'll run on the same machine as your mail server. It requires python and it does work with exim.

You probably wouldn't need a captcha system anyway. Spam almost always has an un-reply-able address. I've been using tmda for years with great success. My inbox is spam free. 100% spam free.

I do have to drain my tmda-pending queue now and then to drain out the stuff that spamassassin hadn't caught for me, but it sure beats trying to weed out signal from noise in my inbox.

As for some sort of user verification system to determine which addresses used for your domain are valid, I would bet that exim has a solution built in. However, since I use postfix on the servers I run, I don't know offhand how that'd work for exim.
posted by ursus_comiter at 9:57 AM on February 24, 2006

I didn't implement it myself, my mailhost did, but greylisting cut my incoming spam significantly.

The mailserver keeps a list of e-mail addresses you've received mail from. If mail comes from a new address, it tells the originating mailserver that it's not accepting e-mail right now. A legit mailserver will try again, based on its own retry interval, and if a long enough interval has passed, the receiving mailserver will recognize the retry, let the mail through, and add the e-mail address to the whitelist.

The greylist approach works like a sender verification setup, but without actually requiring correspondants to take an extra step once you've rejected their e-mail. It works because servers obeying the SMTP protocol will always retry mail (though sometimes the delay is rather long) and most spammers don't want to keep a mailserver around long enough to resend messages, lest they end up on a real time blacklist instead. The originator of the concept talks more about it here It looks like the links page references some exim implementations.

Good luck.
posted by Good Brain at 9:59 AM on February 24, 2006

I like the sound of that Good Brain. I may adopt it for my server.
posted by ursus_comiter at 10:03 AM on February 24, 2006

That's an interesting idea Good Brain, unfortunately in my case a lot of the messages are actually bad bounce messages caused by spammers jo-jobbing one of my domains, so I don't think that would help in that case.
posted by delmoi at 10:07 AM on February 24, 2006

looking more closely at exim, it dosn't apear to have much documentation (they want you to buy a book). I'll probably get rid of it and install postfix
posted by delmoi at 10:43 AM on February 24, 2006

Postfix is a better way to go. Weitse, the author, is very active on the postfix-users mailing list. There are also a couple new books (Not the one by Blum) on Postfix that cover this kind of stuff.

You may end up putting tmda and greylisting in place to further cut down on your spam volume.
posted by drstein at 10:53 AM on February 24, 2006

or not... somehow postgresql depends on exim? Sou I would lose my databases if I uninstalled it. Seems pretty bizzare.
posted by delmoi at 10:54 AM on February 24, 2006

hrm, if I do apt-get install postfix, it claims it will remove exim but not postgre. Eh. I think I'll leave it alone for now.
posted by delmoi at 10:56 AM on February 24, 2006

delmoi: In my experience, you'll be much happier with Postfix than Exim. PostgreSQL doesn't depend on Exim in Debian, definitely.

What you're going to look at for greylisting with postfix is "postgrey". Extremely easy to set up, very capable. No email gets lost from email servers that follow the spec, but (some mailing lists etc.) that don't follow the spec can be added to a whitelist. It comes with a fairly expansive default list of those too.

If you run into problems setting it up, drop me an email and I'll give you a hand.
posted by Kickstart70 at 12:30 PM on February 24, 2006

One of the best things I did was dumping the catch-all address. I know it's handy to have, but getting rid of it helped things immensely.

I nth this. I was very very reluctant to shitcan it since it was an easy way to identify where mail came from based on who I gave what. However I realized that using SpamGourmet was equally effective and had the added payoff of being able to set a number of mails I'd take and having someone else deal with managing the filters.

Getting rid of it has also cut down on the number of baloney bounce messages from people claiming to be from my domain.
posted by phearlez at 1:20 PM on February 24, 2006

If you can have exim pass off to procmail for final delivery, you can use my procmail challenge-response script.
posted by nicwolff at 2:02 PM on February 24, 2006

PostgreSQL doesn't depend on Exim in Debian, definitely.

It shouldn't, but when I type "apt-get remove exim" the following shows up:

_{Reading Package Lists... Done
Building Dependency Tree... Done
The following packages will be REMOVED:
  exim mailx postgresql}

Very weird
posted by delmoi at 2:37 PM on February 24, 2006

Okay, I backed up my databases and did "apt-get install postfix" I think it's just that postgresql requires a mail server installed.
posted by delmoi at 2:49 PM on February 24, 2006

alright, my databases are still there :) I have postfix now I'll look up the documentation for that and procmail.
posted by delmoi at 2:52 PM on February 24, 2006