Fixing my spam problem
January 28, 2004 11:43 PM Subscribe
ok, so I've had it up to here with nefarious spammers and other lowlifes spoofing my email addresses, causing my inbox to be filled with bounces and besmirching my and my company's good name. since we're all relatively smart and tech savvy, what exactly has to happen in order to get an authenticating email system to replace what's currently out there? is it an impossible dream?
Btw, the most-often-quoted essay on a spam-free-future is Paul Graham's A Plan for Spam.
posted by gen at 12:50 AM on January 29, 2004
posted by gen at 12:50 AM on January 29, 2004
i've had some success in two different ways. First, using linux (or i suppose OSX if you don't mind command line,) you can do some really nice tricks with mutt. Specifically, i only allowed mail that came directly to me or was CC's directly to me, BCC was blocked (although i did have a whitelist which would allow it to pass from certain sources), i also maintained a detailed blacklist which killed just about everything else.
Stopped about 95% of the crap from reaching me. The downside is that it might catch some messages from strangers that you might actually want (assuming, of course that the stranger might have a good reason for BCCing you...)
[if anyone is _really_ interested, i would be more than happy to offer the salient points off my .muttrc so that they could do this themselves]
When using Windows, i like Mailwasher. It's a free, useful little program that aside from being pretty damn good at catching spam, also allows you to read your messages directly from your POP server without actually downloading anything other than the text. This is an excellent tool if you have a problem with viruses or big attachments and are forced to use one of the MS mail programs that are most susceptible to them.
posted by quin at 1:20 AM on January 29, 2004
Stopped about 95% of the crap from reaching me. The downside is that it might catch some messages from strangers that you might actually want (assuming, of course that the stranger might have a good reason for BCCing you...)
[if anyone is _really_ interested, i would be more than happy to offer the salient points off my .muttrc so that they could do this themselves]
When using Windows, i like Mailwasher. It's a free, useful little program that aside from being pretty damn good at catching spam, also allows you to read your messages directly from your POP server without actually downloading anything other than the text. This is an excellent tool if you have a problem with viruses or big attachments and are forced to use one of the MS mail programs that are most susceptible to them.
posted by quin at 1:20 AM on January 29, 2004
Oh, and with regard to your server side problems (read as: spoofing, etc) relax, Bill's on the case.
In all seriousness, the fact that MS is pursuing this problem is actually kind of interesting, fixing the spam problem is going to require one of two things, a complete restructuring of the worlds laws regarding unsolicited commercial email and a strong desire from every nation to enforce these new laws (and since none of us are likely to live long enough to see those particular pigs fly,) a complete restructuring of the way we deal with email on a day to day basis. While seeming equally unlikely on the surface, the fact that so many POP servers are (unfortunately?) MS powered, and the fact that the Apple and Opensource communities are clever enough to support some degree of compliance with MS means that if Gates is serious, we could see a dramatic downshift, or at least a significant change in the way UCE works.
And from the standpoint of a former professional in this field, i can't believe that is a Bad Thing.
posted by quin at 1:32 AM on January 29, 2004
In all seriousness, the fact that MS is pursuing this problem is actually kind of interesting, fixing the spam problem is going to require one of two things, a complete restructuring of the worlds laws regarding unsolicited commercial email and a strong desire from every nation to enforce these new laws (and since none of us are likely to live long enough to see those particular pigs fly,) a complete restructuring of the way we deal with email on a day to day basis. While seeming equally unlikely on the surface, the fact that so many POP servers are (unfortunately?) MS powered, and the fact that the Apple and Opensource communities are clever enough to support some degree of compliance with MS means that if Gates is serious, we could see a dramatic downshift, or at least a significant change in the way UCE works.
And from the standpoint of a former professional in this field, i can't believe that is a Bad Thing.
posted by quin at 1:32 AM on January 29, 2004
I got myself a new e-mail address and protected it mercilessly with a series of forwarding addresses (to give to humans) and sneakemail for computers.
I've had this current address for a year and a half, and I have NO spam problem whatsoever. Just keep yourself off the lists, and you're fine.
However, crunchland, you're talking about your "company", which makes it a bit more tricky to up sticks and leave. Could you start again with a new address on the same domain? My work is changing all of ours from name.surname to namesurname to give us a clean slate.
posted by bonaldi at 2:59 AM on January 29, 2004
I've had this current address for a year and a half, and I have NO spam problem whatsoever. Just keep yourself off the lists, and you're fine.
However, crunchland, you're talking about your "company", which makes it a bit more tricky to up sticks and leave. Could you start again with a new address on the same domain? My work is changing all of ours from name.surname to namesurname to give us a clean slate.
posted by bonaldi at 2:59 AM on January 29, 2004
Crap, sorry to post again so soon, but bonaldi just reminded me of another trick i use. A friend of mine turned me onto the concept of using the personalized domains that we owned as a source of spam management.
Note: this is a more ideal solution for knowledgeable individual users than businesses, so i apologize for adding this to crunchland's thread, who seems to be looking for enterprise-level solutions, but...
Basically, the idea is that if you own your own domain, and can set it so that any mail coming to that domain comes to you, use that feature to your advantage. Whenever you sign up for a site that requires an email addy, use the site name @ your domain, (ie: here i might use something like MeFi@mydomain.com)
The idea being that if any site sells your addy, you know exactly who it was and can turn them in to their upstream provider.
i can say, for the record, that this system has worked well for me, and because of this trick, i was able to close at least one spam source before it became problematic.
i admit, this is a fairly proactive response to spam, which requires both the desire to seek revenge as well as the money/ knowledge to keep the system operating, but i can tell you from personal experience, it's worth it to have an ISP's abuse dept tell you that the evidence that you provided was what was necessary to close a spammers account for good.
ok, i'm done.
posted by quin at 3:25 AM on January 29, 2004
Note: this is a more ideal solution for knowledgeable individual users than businesses, so i apologize for adding this to crunchland's thread, who seems to be looking for enterprise-level solutions, but...
Basically, the idea is that if you own your own domain, and can set it so that any mail coming to that domain comes to you, use that feature to your advantage. Whenever you sign up for a site that requires an email addy, use the site name @ your domain, (ie: here i might use something like MeFi@mydomain.com)
The idea being that if any site sells your addy, you know exactly who it was and can turn them in to their upstream provider.
i can say, for the record, that this system has worked well for me, and because of this trick, i was able to close at least one spam source before it became problematic.
i admit, this is a fairly proactive response to spam, which requires both the desire to seek revenge as well as the money/ knowledge to keep the system operating, but i can tell you from personal experience, it's worth it to have an ISP's abuse dept tell you that the evidence that you provided was what was necessary to close a spammers account for good.
ok, i'm done.
posted by quin at 3:25 AM on January 29, 2004
Response by poster: actually, I wasn't really trying to start a conversation about anti-spam tactics and technology.
I really wanted to start a serious discussion about what it would take to replace the current pop3 protocol so that email addresses were secure and verifiable. I can't imagine the reason why we don't have it yet is because we don't want to inconvenience the network admins and isps. Surely it's in their best interest to bring about the changes necessary to accomplish it.
Is it an impossible dream?
posted by crunchland at 3:30 AM on January 29, 2004
I really wanted to start a serious discussion about what it would take to replace the current pop3 protocol so that email addresses were secure and verifiable. I can't imagine the reason why we don't have it yet is because we don't want to inconvenience the network admins and isps. Surely it's in their best interest to bring about the changes necessary to accomplish it.
Is it an impossible dream?
posted by crunchland at 3:30 AM on January 29, 2004
Well, POP3 is just the protocol your mail tool uses to copy your mail down from the server and doesn't have anything to do with spoofing; SMTP is the "sending" protocol and the one whose openness permits spoofing of "From" addresses.
If all you want is to prevent spoofing, then the Sender Permitted From extension to DNS and SMTP is what you're looking for. AOL is testing it, so I guess it has a shot at being widely adopted.
posted by nicwolff at 4:05 AM on January 29, 2004
If all you want is to prevent spoofing, then the Sender Permitted From extension to DNS and SMTP is what you're looking for. AOL is testing it, so I guess it has a shot at being widely adopted.
posted by nicwolff at 4:05 AM on January 29, 2004
The only thing you can currently do to stop people spoofing your email is to use legislation or the offending ISP and make them stop.
However, there is a new technology which forces email from a specific domain to ACTUALLY come from that domain. See the SPF Website for more information. I can't see it working. Currently email from my domain is actually sent via fastmail.fm, and as such SPF enabled mailboxes would lose my email.
So - Impossible Dream. No, but it's unlikely that the technology will be in place for a very long time.
short term measures to help facilitate this...
make sure that your "From" address matches your From domain.
Implement SPF yourself, with a bounce back to the sending person explaining (a) what they can do to stop the bounce (b) how they can otherwise get in touch with you.
(c) talk SPF up to all and sundry.
posted by seanyboy at 4:13 AM on January 29, 2004
However, there is a new technology which forces email from a specific domain to ACTUALLY come from that domain. See the SPF Website for more information. I can't see it working. Currently email from my domain is actually sent via fastmail.fm, and as such SPF enabled mailboxes would lose my email.
So - Impossible Dream. No, but it's unlikely that the technology will be in place for a very long time.
short term measures to help facilitate this...
make sure that your "From" address matches your From domain.
Implement SPF yourself, with a bounce back to the sending person explaining (a) what they can do to stop the bounce (b) how they can otherwise get in touch with you.
(c) talk SPF up to all and sundry.
posted by seanyboy at 4:13 AM on January 29, 2004
Well, in that case.
You're not really talking about replacing POP3 - that's for collecting mail. You want to change SMTP - the delivery mechanism.
There's a very good plan out there for that just now. But since google stopped searching the web, I'm having trouble finding it. In summary, it's called reverse MX. What happens is this: Domain records say what servers are allowed to *send* mail for them. If an mail is received from an unregistered host for that domain it can either be discarded or marked as spam, depending on the receiving host's preferences.
No more e-mails purporting to be from hotmail.com for a start. It's a solid solution, and avoids the privacy issues of identifying actual users. As soon as I find that link I'll post it.
on preview: there it is! both those SPF links are what I wanted.
seanyboy: can't you just add fastmail.fm as an authorised sending host to your domain's SPF record? As far as I can see the only problem SPF could cause is where one mail application uses one SMTP server for various accounts, and that's generally just a local configuration error
posted by bonaldi at 4:18 AM on January 29, 2004
You're not really talking about replacing POP3 - that's for collecting mail. You want to change SMTP - the delivery mechanism.
There's a very good plan out there for that just now. But since google stopped searching the web, I'm having trouble finding it. In summary, it's called reverse MX. What happens is this: Domain records say what servers are allowed to *send* mail for them. If an mail is received from an unregistered host for that domain it can either be discarded or marked as spam, depending on the receiving host's preferences.
No more e-mails purporting to be from hotmail.com for a start. It's a solid solution, and avoids the privacy issues of identifying actual users. As soon as I find that link I'll post it.
on preview: there it is! both those SPF links are what I wanted.
seanyboy: can't you just add fastmail.fm as an authorised sending host to your domain's SPF record? As far as I can see the only problem SPF could cause is where one mail application uses one SMTP server for various accounts, and that's generally just a local configuration error
posted by bonaldi at 4:18 AM on January 29, 2004
bonaldi: probably. I'll admit to being a little bit confused by what SPF actually is and how it works.
posted by seanyboy at 4:33 AM on January 29, 2004
posted by seanyboy at 4:33 AM on January 29, 2004
I'm a bit unsure of it too, but here's what I understand of SPF:
1. Each domain has an file (stored on the global DNS servers) that tells callers how to connect to it, how to send mail to it and who handles backups when it's down.
2. SPF adds a note to that file telling anyone who asks what domains are allowed to send mail claiming to be from that domain.
3. Mail receivers will be customised to look for this record, and reject or otherwise handle mail claiming to be from X.com but actually coming from spam.com
4. Bye, anonymous spammers.
5. Profit
since you can put anything you like in your allowed-to-send entry, if you have seanyboy.com but send from fastmail.fm, just add fastmail.fm as an allowed sender. Then the only people who could spoof mail from seanyboy.com would have to send from a fastmail.fm server.
Of course, SPF doesn't solve the problem of throwaway domains, but later bolt-ons can check for this, including marking higher spam scores for newborn domains. And since the blacklists won't be bogged down with millions of incorrect entries, they'll be more useful and more trusted to get only the spammers.
posted by bonaldi at 4:57 AM on January 29, 2004
1. Each domain has an file (stored on the global DNS servers) that tells callers how to connect to it, how to send mail to it and who handles backups when it's down.
2. SPF adds a note to that file telling anyone who asks what domains are allowed to send mail claiming to be from that domain.
3. Mail receivers will be customised to look for this record, and reject or otherwise handle mail claiming to be from X.com but actually coming from spam.com
4. Bye, anonymous spammers.
5. Profit
since you can put anything you like in your allowed-to-send entry, if you have seanyboy.com but send from fastmail.fm, just add fastmail.fm as an allowed sender. Then the only people who could spoof mail from seanyboy.com would have to send from a fastmail.fm server.
Of course, SPF doesn't solve the problem of throwaway domains, but later bolt-ons can check for this, including marking higher spam scores for newborn domains. And since the blacklists won't be bogged down with millions of incorrect entries, they'll be more useful and more trusted to get only the spammers.
posted by bonaldi at 4:57 AM on January 29, 2004
My recent informal proposal for spam and virus free email. I think it would work.
Basically we drop POP and come up with a new protocol which requires mail servers to get a certificate similar to SSL certs which they must use to identify themselves during each mail hop. The certification authority then implements an automated system for tracking and enforcing opt-out requests. A server which doesn't honor opt-out requests loses their certification.
posted by y6y6y6 at 5:43 AM on January 29, 2004
Basically we drop POP and come up with a new protocol which requires mail servers to get a certificate similar to SSL certs which they must use to identify themselves during each mail hop. The certification authority then implements an automated system for tracking and enforcing opt-out requests. A server which doesn't honor opt-out requests loses their certification.
posted by y6y6y6 at 5:43 AM on January 29, 2004
Problems I see with that:
* Asking for all the world's clients to be updated. Massively unlikely to happen.
* Loss of anonymity with certification
* Having to pay for certificates - loss of free e-mail
posted by bonaldi at 5:51 AM on January 29, 2004
* Asking for all the world's clients to be updated. Massively unlikely to happen.
* Loss of anonymity with certification
* Having to pay for certificates - loss of free e-mail
posted by bonaldi at 5:51 AM on January 29, 2004
bonaldi - You need to read down in the comments. I address all those concerns.
"Asking for all the world's clients to be updated. Massively unlikely to happen."
People upgrade all the time. I'd guess most people upgrade at least once every two years. Most clients will now let you choose either pop or imap, just add another protocol choice. And no one has to switch over. The new protocol can exist next to the old stuff. But i think over time it would naturally take over. If we got rid of 90% of spam over two years that would be pretty amazing, yes?
"Loss of anonymity with certification"
The protocol doesn't say anything about the anonymity of the sender. You could still use this protocol with an anonymous hotmail account. The server needs the cert, not the end user. And users who *needed* anonymous relay systems for one reason or another could always use pop. But the vast majority of users don't want or need that level of privacy. Most users *want* the recipients to know where an email they send came from.
"Having to pay for certificates - loss of free e-mail"
Nope. The certs can be cheap. The process could be the bottleneck that will keep spammers from jumping from cert to cert. If it takes a week to get a certificate, And you an only send a few thousand spams before the cert gets pulled, then the profit motive is gone. Spammers need millions of spams to make money. And reducing the bandwidth and admin now eaten up by spam would reduce costs at the ISP level. There would be no increased cost to pass on.
posted by y6y6y6 at 6:13 AM on January 29, 2004
"Asking for all the world's clients to be updated. Massively unlikely to happen."
People upgrade all the time. I'd guess most people upgrade at least once every two years. Most clients will now let you choose either pop or imap, just add another protocol choice. And no one has to switch over. The new protocol can exist next to the old stuff. But i think over time it would naturally take over. If we got rid of 90% of spam over two years that would be pretty amazing, yes?
"Loss of anonymity with certification"
The protocol doesn't say anything about the anonymity of the sender. You could still use this protocol with an anonymous hotmail account. The server needs the cert, not the end user. And users who *needed* anonymous relay systems for one reason or another could always use pop. But the vast majority of users don't want or need that level of privacy. Most users *want* the recipients to know where an email they send came from.
"Having to pay for certificates - loss of free e-mail"
Nope. The certs can be cheap. The process could be the bottleneck that will keep spammers from jumping from cert to cert. If it takes a week to get a certificate, And you an only send a few thousand spams before the cert gets pulled, then the profit motive is gone. Spammers need millions of spams to make money. And reducing the bandwidth and admin now eaten up by spam would reduce costs at the ISP level. There would be no increased cost to pass on.
posted by y6y6y6 at 6:13 AM on January 29, 2004
spoofing my email addresses, causing my inbox to be filled with bounces
what you're seeing at the moment is junk from the MyDoom virus.
posted by andrew cooke at 6:27 AM on January 29, 2004
what you're seeing at the moment is junk from the MyDoom virus.
posted by andrew cooke at 6:27 AM on January 29, 2004
You just need digitally signed email. If you get something digitally signed, you can be (almost completely) sure it's from who it says it's from. The trick is you'll have to get everybody to start using it, or at least get everybody who receives email from you to understand that if it ain't signed, it ain't from you.
posted by callmejay at 9:16 AM on January 29, 2004
posted by callmejay at 9:16 AM on January 29, 2004
callmejay: you also have to download the entire message before you can check the signature. It's still a client-side solution and does not help the bandwidth-wastage problem. One advantage to SPF is that you need only download the headers to find out if the mail is valid, and any mailer along the chain can perform the check.
It wouldn't hurt to use both, I suppose...
posted by Mars Saxman at 9:23 AM on January 29, 2004
It wouldn't hurt to use both, I suppose...
posted by Mars Saxman at 9:23 AM on January 29, 2004
y6y6y6, you've got me on two of them. But I still don't think you'll upgrade the world's clients. Two years seems optimistic to me. My mother's still on Win95, and she's not for upgrading. My dad's on a 68k Mac, and can still use Claris Emailer. Both these people would be forced to buy new kit, just to stop spam (that doesn't affect them because they're careful with their addresses)? Unlikely.
Backward compatibility is pretty much essential for e-mail. There are too many computers out there working well enough that don't need upgrading -- and people wanting to communciate with them will be held back from moving to your SSLMail. And think of all those embedded email apps that couldn't be upgraded: mobile phones, for instance, and e-mail terminals, yada yada yada.
The solution has to come at the server-side, and be transparent to users, I think. My gran has to be able to use it. What don't you like about SPF?
posted by bonaldi at 9:31 AM on January 29, 2004
Backward compatibility is pretty much essential for e-mail. There are too many computers out there working well enough that don't need upgrading -- and people wanting to communciate with them will be held back from moving to your SSLMail. And think of all those embedded email apps that couldn't be upgraded: mobile phones, for instance, and e-mail terminals, yada yada yada.
The solution has to come at the server-side, and be transparent to users, I think. My gran has to be able to use it. What don't you like about SPF?
posted by bonaldi at 9:31 AM on January 29, 2004
Yep. The chicken and egg problem is real. Getting people to upgrade to a certified protocol would be a long process. Well....... Tough. Let's do it anyway. And of course Win95 clients could created. Last time I checked Outlook and Eudora still worked on Win95.
"What don't you like about SPF?"
SPF seems like a patch that will only fill in a few holes. We need a new structure.
And the obvious circumvention for spammers is already in the wild. The MyDoom worm will happily send spam from a legit IP. Or they could just use open wireless points. Etc.
We need to stop patching and fix the problem.
posted by y6y6y6 at 10:02 AM on January 29, 2004
"What don't you like about SPF?"
SPF seems like a patch that will only fill in a few holes. We need a new structure.
And the obvious circumvention for spammers is already in the wild. The MyDoom worm will happily send spam from a legit IP. Or they could just use open wireless points. Etc.
We need to stop patching and fix the problem.
posted by y6y6y6 at 10:02 AM on January 29, 2004
I agree with Jon that SPF is not a total solution, but that's not to say that it isn't worth implementing. Anything, anything to stem the tide of crap mail on the 'net is worth investigating, imo.
posted by gen at 3:44 PM on January 29, 2004
posted by gen at 3:44 PM on January 29, 2004
This thread is closed to new comments.
At the Internet infrastructure level, it's a big deal to ask ISPs and email providers to agree upon one path towards a spam-free future. There is no agreed-upon path towards a no-spam nirvana...yet.
At the personal level, you have a few options.
1) Limit the spam before it hits your computer (use an ISP that has spam-filtering software installed.)
- I'm installing Spam-Assassin this weekend to cull more spam for my mail.
2) Install spam filters on your local machine/email client.
- I have Mail.app for Mac OS X do this for me. Not completely effective by any measure but better than nothing.
3) Move to a whitelist progam. Such a system will only allow through email that you specify. Those who are not on your list have to do a "challenge-response" (usually type a random phrase or characters into a web page form) in order to have their mail go through. Most spammers won't take the time to do that.
- I am not willing to do this to people who email me...yet.
posted by gen at 12:48 AM on January 29, 2004