Declaring independence from spam attacks

I don’t like spammers. At all. In fact, as some may know, I have the dubious distinction of being the original spam fighter.

Spam has come a long way since then, but it’s still a tactic reserved to the lowest category of exploitive, thieving vermin. And we still deal with our share of spam issues at NearlyFreeSpeech.NET.

Every now and then, a spammer finds one of our members’ sites that has an exploitable PHP mail() form. When this happens, the result is always the same: 50,000 – 150,000 junk mail messages flood into our system destined for all eight corners of the world (half of them bounce).

That clogs up our whole email system, and because our network is distributed it often takes us awhile to figure out what’s going on. Meanwhile, our email systems are pretty fast so they are more than happy to pump out several hundred junk mail messages every second until we pull the plug. I don’t care how fast your reaction time is, that’s way too many.

To combat this, we’ve introduced two new features: a default rate limit (20 email messages per minute per site) and a point-based queuing system for mail. Each site has a “bank” of email sending points. Your site earns one point each minute, up to a maximum of 100. When the site sends a message via PHP, it will deduct a point from the bank. If it hits zero, email will go into a queue until more points are available. (So not to worry, nothing legitimate will be lost even if you send out a lot of email at once. At the worst, it’ll get slowed down a bit.)

If a sizable queue forms, we will see it and be able to respond before the bulk of the junk gets unleashed on the world. If we check it out, and it’s legitimate, we can pass it right through. If it’s not… it never gets off the launch pad. This is adaptable on an per-site basis, so people running mailing lists and other high-volume or “bursty” senders won’t be affected. (Sites that do send large volumes of email tend not to get exploited anyway.)

We’ve added a line to the site info panel showing the current/maximum email bank values and the number of messages currently queued to send, if any.

Right now, this does not affect CGI scripts that send mail via sendmail. Such scripts represent a small fraction of email sent, exploits targeting them are even more rare, and they are pretty easy for us to catch manually. Even so, we will be backfilling that in the future to provide complete protection.

We fight a constant battle to keep the handful of spammers, scammers, and fraudsters at bay without restricting what our huge majority of perfectly legitimate users are able to do. We’ve had to make some tough compromises to stop spammers, like blocking email from the ssh server to stop spammers from setting up free trial accounts and using them to spew garbage. That’s why it’s nice to have a solution like this that kicks the bad guys right where it hurts without impacting the members we care about.

Cool stuff.

7 Comments

RSS feed for comments on this post.

1. Your last name is “Wheelhouse”? For some reason I just didn’t expect that.

   I know, I know. Everyone expects my name to be Jeff Bofh for some reason. 😉 -jdw

   Comment by dsymonds — July 5, 2007 #

2. thanks jdw!

   your efforts past and present to stop those parasites that rob the ‘net of vitality and clog the communications arteries are much appreciated.

   Comment by maphew — July 7, 2007 #

3. For when I catch up on my website development: Does this alleviate your very reasonable concern about contact form security against spamming by unknown senders through a form? I’ve thought of installing a contact form and I’m not sure I know where to find one that meets your security needs. (What I do now is state my Yahoo email address on my pages and maybe I’ll let people email me via my websites through NFSNet forwarding, but a contact form still seems good if security can be assured.) If this does not alleviate your concerns because some abuses are still possible-to-likely, I don’t want to rush or risk it. Thoughts? Thanx.

   This feature mitigates the damage caused by spammers who exploit vulnerable PHP forms, but it is by no means a substitute for proper security and due diligence. We will still have to suspend a site’s ability to send email if a vulnerable form is discovered. -jdw

   Comment by Nick — July 14, 2007 #

4. Rate limiting is full of win.

   BTW, I had no idea that you’ve been involved with spamfighting since the very beginning. That is all sorts of awesome.

   Comment by Douglas Muth — July 17, 2007 #

5. Just curious how this affects email privacy, you mention you’ll check it out.. does this mean there’s a ‘small’ possibility that our email will get read by you?

   Not meaning to be a prick, I’m just curious, you guys run a great service 🙂

   P.S. Just an idea / suggestion, do you guys run outgoing mail through a spam filter, and if over X amount (or X%) is flagged then investigate? I think that’s what my ISP does IF I remember correctly and it seemed like a good idea.

   [If your site is suddenly sending out a huge volume of mail, then yes, there’s a small chance that one of them will get manually reviewed. However, similar to your suggestion, the first thing that’s done in such a case is that a random sample of the messages will get run through the open-source program SpamAssassin, and we’ll look first at the automated analysis generated by that. This step exists specifically to protect your privacy. Only then, if it does appear to be a spam issue, will we read one message to confirm and identify key spam phrases (e.g. “viagra” or “late Nigerian bank official”) and have matching messages automatically dequeued. SpamAssassin is a pretty effective program which lets us have a high degree of confidence that you’re being abused before we proceed.

   Note that this applies only to email messages sent scripts on your web site, not our email forwarding service. The content of messages that pass through the email forwarding servers is never manually reviewed.)

   It’s a fair question, but we have zero interest in violating your privacy by reading mail sent by your site, and take every precaution to avoid doing so unless apparent spammer activity demands it. -jdw]

   Comment by Peter Sandersen — July 31, 2007 #

6. This is completely awesome. It’s a novel solution to the issue.

   (Could you guys put this in the FAQ? I just sent you a service request on this after I didn’t find information about “email sending” in the FAQ.)

   Once we finish tweaking it, which should be soon, we will definitely add some information about it to the FAQ. -jdw

   Comment by Matthew — August 4, 2007 #

7. Haha, rock on! This is a great solution.

   Comment by Tim McCormack — August 12, 2007 #

Sorry, the comment form is closed at this time.