Is MailChannels queuing Yahoo mail?

It should be known that we are not currently working with Yahoo! Inc. However, we have heard murmurings that they have enforced more stringent filtering policies in addition to their own home-grown traffic shaping solution.

After Ken Simpson’s MAAWG presentation, we were contacted by a concerned system administrator who wanted to find out if our email traffic shaping has something to do with Yahoo’s current issue of widespread queuing and backlogs when sending mail to Yahoo! Mail users. Postmaster mailing lists have been reporting two situations:

1. Yahoo is returning 421 Messages, temporarily deferring every message attempt. Apparently Yahoo never accepts the message, even after a day of retrying. Whether you use DomainKeys or SPF records, Yahoo is deferring the message as soon as your mail server connects, so it never gets a chance to see the header.
2. Good email are not being bounced, but tagged as Spam and filed directly to the Spam folder that Yahoo users rarely check. As worldwide spam volume rises, users become numb to the ballooning size of quarantines and check Spam folders less often.

Needless to say, this filtering policy equates to bad service for Yahoo users. Whether or not they also multiplex the ‘shaped’ connections is not known, but this would be a good time to shed some light on the topic of email queuing.

SMTP typically works in a queue where mail is delivered in the order it arrives. In most cases, administrators don’t know when and for how long their mail is being delayed by backlogged mail servers. But relying on the fact that “mail will get delivered whenever it can” is a bit of a cop out.

Queuing email on the receiving server is an unnecessary artifact of the history of slow email servers with limited connections. It’s better to just keep all the connections alive and process them according to priority. But queuing makes this difficult because receiving the message into the queue in the first place is done in the order of receipt not by priority. Then once the message is in the queue it is now the responsibility of the recipient despite the fact it has not yet been processed and delivered.

By deploying traffic shaping, good senders are no longer forced to retry when connection limits are hit because the connections are maintained. If Yahoo were to deploy reputation-based prioritization, only bad senders and unknown senders would be penalized. Traffic shaping prioritizes email on servers to reduce the delay and loss of non-spam emails due to queuing. This is not to be confused for anti-spam vendors who rate-limit on quantity of emails, rather than the speed of email delivery.

Traffic shaping on SMTP is often mistaken for connection rate-limiting. Rate limiting sets a quota on the maximum amount of messages a spamming source can send in a given time, i.e. 6 messages per minute. After which, all future messages are rejected. On the other hand, traffic shaping involves interfering with network traffic by reducing the bandwidth for SMTP connections. It performs and implements similar to a network load-balancing device to pool and reduce overall email volume.

Were Yahoo to be doing SMTP traffic shaping, no messages would be lost or deferred. In fact, traffic shaping prioritizes sources with good traffic and throttles sources that are sending spam, resulting in up to 70 per cent reduction in overall email volume.