Email forwarding changes and UI downtime

Over the next few days, we will be making major changes to our email forwarding service in order to make sure that we are providing a consistent, high-quality forwarding service that meets our members’ expectations. This is something we have wanted to do for awhile, but recent events have forced us to pursue a change strategy that might not have been our first choice.

One of the major problems we have with our service is that we have two different email forwarding services, the differences between which are documented very poorly when they are documented at all.

The older one is called Legacy Forwarding. It is a service we provide using our own servers, and adheres to our draconian email acceptance policy.

The newer one is simply called Email Forwarding. It is a service we provide in conjunction with a partnership with a third-party company. It is supposed to work a little differently; we call it a “hybrid” forwarding system because it combines certain aspects of a mailbox with certain aspects of email forwarding. Their servers are willing to accept virtually any email, but they then perform content-filtering checks and are supposed to divert suspect messages into a special “spam quarantine” accessible via our member interface.

I say “supposed to” because at some point in the recent past, without consulting with or notifying us, the third-party company unilaterally changed their system to discard all spam-suspect messages instead of placing them into the quarantine folder. At first we thought this was isolated to a few domains, but we have since discovered that the change was global. This means that for all of our members who have “Email Forwarding,” any messages you receive that are believed by their system to be spam, including false positives, are currently being silently discarded. This is absolutely not acceptable to us, but our efforts to redress this with them have not been successful.

Ultimately, our email forwarding members are paying us for email forwarding services, not a third party company, so it is our responsibility to make sure our members get the service they are paying for. To that end, we are taking the following steps:

• We will stop billing for all email forwarding domains until we feel things are working the way we intend.
• We are deploying a new hybrid email forwarding system, described below, which will be applied to all members’ email forwarding domains.
• Because we need to make changes, we will be temporarily disabling the email forwarding portions of our UI. We are not disabling email forwarding, just the UI that controls it.

The new email forwarding system will apply all of our existing email acceptance policy criteria, but it will do so a little differently. As with our legacy forwarding system, messages that meet all of our criteria will sail through the system with no disruption or delay, as will messages sent by domains with proper SPF configurations, even if they have other problems. Also as with the legacy system, messages from blacklisted servers will be immediately rejected.

For messages sent by servers with improper or no SPF and one or more other configuration problems, instead of bouncing them as the legacy forwarding system does, we will be trying a technique called greylisting, which tells the server sending the message to try again in a few hours. If the sending server does that, the message will be accepted.

We will also be adding a spam/virus checking component that will reject email containing viruses and quarantine egregious spam that somehow passes all of the above filters. (The current email forwarding scheme quarantines both spam and viruses.) Also new will be a feature that places post-forwarding bounce messages into the same quarantine, so if there are problems with delivering forwarded messages to you, you will be better able to detect and resolve them.

We feel that this system will make the best trade-offs between meeting our members’ need for reliable forwarding, keeping our forwarding mail servers off of global blacklists, and making life absolutely as miserable and unrewarding as we possibly can for spammers. It will also let us have one forwarding service for all members, and document it appropriately.

All of our email forwarding uses MX records in the nearlyfreespeech.net domain, so we retain full control over how they are handled. The above system is still in final development, but we believe it can be completed and deployed without any major disruption to anyone’s forwarded email. Due to the way Internet email works, even if we do wind up needing to take forwarding offline for a few hours in the middle of the night, messages will simply queue and redeliver when it comes back up.

We need to disable the member UI to work on it because we need the overall configuration to hold still long enough for us to update the UI code and move everyone over to the new system. It is our goal to have the entire transition completed by Saturday. Since we won’t be billing for email forwarding until this is resolved, we will be very highly motivated to make it happen as quickly and accurately as possible. If for any reason you do need to make an urgent email forwarding change that just can’t wait, or if you want to remove it altogether, just drop us a Secure Support Request and we’ll do our best to take care of you.

Since this is still an evolving situation, the above reflects what we want to do, and what we reasonably believe we can do. If and when circumstances change, we will update you accordingly. These changes will not affect the cost of email forwarding, and while they may ultimately put us in a more flexible position with respect to offering email hosting in the future, that is not a goal of this project and nothing is changing on that front at this time.

I apologize for the short notice and rushed character of this change. We spent a lot of effort (probably too much) trying to come to a less dramatic solution, and we were entirely unsuccessful.

2 Comments

RSS feed for comments on this post.

1. I’m looking forward to having ALL mail delivered from legitimate companies even if they have a bad set up, to date important mail from legitimate companies had to use different addresses to insure delivery (it amazing how many companies take offence at the suggestion their mail servers might not be set up right.) Being in business I still need that mail even if they are screwing thing up, thanks for the continued efforts guys, you continue to impress me!

   I don’t want to disappoint, but we are not giving a free pass to email servers with problems. They will still have to meet reasonable standards to get email accepted, the same (or possibly somewhat more strict) than anywhere else. If a message can’t be distinguished from spam, it will be refused or quarantined. If you want maximum control over what email gets accepted for you, it will always be the case that you need to operate your own email server and not employ forwarding of any kind. -jdw

   Comment by Brian “Doc” Shank — November 20, 2008 #

2. Glad to hear about this. It sounds like the new solution will be much more in line with your goals in providing email service (that is, forwarding everything you can without getting blacklisted). I’ve heard great things about greylisting, so I hope that works out well for you. (Any chance we can see a list of recently-attempted, greylisted deliveries, so we know that an email will be delivered even if we don’t get it yet?)

   The jury is still out on greylisting; we’ll see how it goes. It’s not feasible to provide any sort of per-domain logs for email forwarding, because a great deal of connections are refused without ever finding out who they were for, and those are generally the cases people want to know about. Under some circumstances, that can include our greylisting setup. -jdw

   Comment by Thomas Tuttle — November 23, 2008 #

Sorry, the comment form is closed at this time.