New Botnets Surpass Storm and Mega-D

According to a report by Marshal Threat Research and Content Engineering (TRACE) team, "Storm" and "Mega-D" are no longer the leading spam botnets. Recent months have shown a shift away from stock pumping, towards spam promoting branded pharmaceuticals such as erectile dysfunction (impotence) and weight loss products. The two largest botnets account for 61 per cent of all spam volume. Worthy of note, the size of a botnet doesn't determine the amount of spam being distributed. Another key finding was that spammers' experimentation in 2007 with spam laced in PDF, Word and MP3 attachments appears to have gone away for now. Overloads caused by botnets is a major concern for administrators who rely soley on content filtering.

The real arms race is one of sheer volume between the amount of traffic spammers can send and the volume of traffic administrators can successfully deliver.

Effective filtering software does nothing to decrease volume (they only separate spam from good email). So, the more email volume servers receive, the more hardware needs to be scaled to match. Traffic shaping on SMTP, as part of a multi-tiered anti-spam architecture, reduces overall volume and gives filtering software additional time to catch up and process emails. Not to mention, it restricts the bandwidth and resources botnets need to hammer your mail servers, dropping volume immediately and keeping spam entirely off your network.

Introducing "the Dip"

Our presentation at the recent MAAWG meetings focused on the effectiveness of Inbound Traffic Control in dealing with spam from unknown senders that represent most of drops seen in anti-spam effectiveness.

Two parts of the presentation really stood out with the audience, the second was a look at what a 98% capture-rate really means to an anti-spam lab.

Introducing "the Dip"



Despite 98% long term capture rates leading anti-spam systems experience significant drops in effectiveness when both sender and content are unknown, the most common times being the use of botnets, targeted campaigns not passing through a central lab and new spam approaches.

Any anti-spam lab worth its salt has a display that looks something like this graph in their lab showing their capture rate over time. Most of the time the capture rate is acceptably high, but once in a while – typically several times a day – the spam starts flooding through and then it’s all hands on deck while the lab figures out where that mail is coming from and how to plug the dike this time.
Sometimes the fix is elegant and long lasting, and sometimes its not.
The new technique can be network oriented or content oriented, and in either case the dip is what results.


From an end users and a service provider’s perspective you can flip this curve upside down and the dips become peak traffic loads, spam outbreaks, help desk calls and flooded inboxes.

Dips happen because anti-spam companies cannot have perfect insight into the spamming world.

• It takes enormous visibility and time to turn a new attack into the actionable quantities of known content and known senders.
• It takes the best filters 10 minutes to widely deploy a new filter rule capable of really making a dent in a new spam campaign.
• The blacklists take between 15 and 30 minutes to set up and distribute a new IP block.

Wouldn’t it be great if we could make the unknown senders wait around for a while – at least until we’ve had a chance to set up a filter rule?

In fact we can, this is one of the benefits of Inbound Traffic Control, messages from unknown senders are forced to wait for better anti-spam information. Taking away the spammers head start.

Spammers are Less Patient than Legitimate Senders

Our presentation at the recent MAAWG meetings (Messaging Anti-Abuse Working Group, 12th General Meeting Feb 18-20, 2008, San Francisco, California) focused on the effectiveness of Inbound Traffic Control in dealing with spam from unknown senders that represent most of drops seen in anti-spam effectiveness.

Based on discussions afterward, two parts of the presentation really stood out with the audience, the first was the difference in spammer and legitimate sender behavior when faced with a slow connection.

Spammers as Less Patient than Legitimate Senders



What this graph shows is that spamming MTA’s are less patient than legitimate senders. The economics of spam means that if forced to wait by a slow MTA the spammers will abandon the connection, usually within 10 seconds and move on while legitimate senders will wait to complete message delivery.

• The SMTP RFC recommends that email servers wait at least three minutes for each chunk of data they send to be received by the receiving server and acknowledged via a TCP acknowledgment packet.
• Furthermore, the RFC recommends that senders wait at least ten minutes for the final message delivery acknowledgment.
• These long timeouts were established because in the early days of the Internet, the infrastructure was slow and unreliable, and the machines were easily overloaded, leading to frequent message delivery delays.
• Today, email servers and our networks are much faster, processing incoming messages in a matter of seconds. Delays still occur, but the timeouts defined in the RFC are vastly higher than what is required in today's world.
• For what we imagine are economic reasons, spammers set their SMTP timeouts on the order of seconds rather than the levels recommended within the RFCs.
  

This graph compares the timeouts for spam traffic versus legitimate traffic. 90% of spam connections are gone within the first 10 seconds, whereas legitimate senders hang on for at least a couple of minutes.

The gap between these two lines is one of the things that Inbound Traffic Control can help you to exploit to reduce spam levels.




Wired Magazine Says Spam Filters Suck

Now that I am in San Francisco for the MAAWG conference, I feel it is only appropriate that I give a plug to the magazine that made geekdome cool. Wired has a feature this month on "why things suck" (with the comedic input of Sarah Silverman). One of their top picks: Spam Filters. Brendan I. Koerner writes:

The most obvious problem is that it's simply not possible to update filtering software frequently enough to catch all of the spammers' multifarious innovations — disguising unsolicited messages by replacing the "i" in Viagra with a "1" or using images in lieu of text, for example. At the same time, an overly aggressive approach can be disastrous, trapping legitimate email as false positives.

I think the Viagra example is a simplification, but the idea that filters are too slow to adapt is accurate. In my talk later this week, I will posit a theory that even as filters become faster, there is always going to be a computational mismatch between the spammers and the filters -- one that the filters will always lose.

Comments are welcome -- they will help me to hone my pitch.

Ken Simpson talking at MAAWG

For those of you fortunate enough to be attending February's MAAWG (Messaging Anti-Abuse Working Group) conference in sunny San Francisco, I will be speaking on "why 98% accurate doesn't really mean 98% accurate." Spam filters have gotten pretty good "on average." The spam war is now all about dealing with those gaps in filter effectiveness and reputation system coverage that occur from time to time, showering end users with bursts of spam. This problem of "dips" is particularly acute for service providers, who suffer mass defections of users when filter accuracy dips.

I will be giving my talk on Tuesday February 19 at 2:30pm at the Mark Hopkins hotel on Nobb Hill. Details are on the MAAWG agenda for those of you who are members of the organization. Pre-talk suggestions are welcome - get in touch with me by leaving a comment.

Using Throttling and Traffic Shaping to Combat Botnet Spam (video)

Ken Simpson presents an overview of the history of spamming, his theory of spammer economics and spammer behavior, and introduces the concept of Connection Management to the USENIX LISA audience. Presentation running time is 37 mins, and concludes with a Q&A session.