Comparing Spamhaus with Proactive Connection Throttling

What follows is a geek-friendly post.

One of the questions we sometimes get from prospective customers is: how well does slowing down spammers actually work, versus other techniques like connection blocking? The answer: After a great deal of analysis, it works very well indeed.

To compare how well throttling (i.e., slowing down or “traffic shaping” – or sometimes “tar-pitting”) connections performs as an anti-spam measure we decided to compare it against the data we had on Spamhaus blocking effectiveness.

We analyzed roughly 66 days worth of data across several of our customer sites from 2010-02-01 to 2010-04-08.

Connections from 28,204,693 distinct IPs were analysed during this time. All of the connections analysed were checked against Spamhaus’ Zen RBL at the time of connection, using up-to-date Zen data. Of those that were not rejected by Spamhaus, throttling was enabled on many of these connections. Of those that were throttled, the client either aborted the connection (we will call this “rejected by throttling”, even though it is not technically correct), or the client remained connected long enough to properly complete the SMTP session. We did not look at any connections that were throttled if Spamhaus was not first queried.

We found that 24,812,576 of these IPs were always rejected by Spamhaus, implying that they must have either a) already been listed on Spamhaus prior to the time of our test (most likely) or b) detected immediately by Spamhaus (less likely).

1,947,976 IPs were rejected by throttling at some point. This means they were not rejected by Spamhaus, and so were throttled and disconnected due to being throttling.

951,637 IPs were never rejected by Spamhaus and also never rejected by throttling during this time-frame. Remember, all connections in our sample were either rejected by Spamhaus or subsequently throttled. Therefore, these unsuccessfully throttled connections were either legitimate senders who were throttled as a result of having a neutral or unknown reputation, or bad senders that were both not on Spamhaus during this time and yet able to handle throttling. 638,626 of these IPs were throttled less than 10 times, which indicates unknown IPs initially being throttled. 453,556 of these IPs were throttled no more than 4 times.

From the 1,947,976 IPs rejected by throttling 559,562 IPs were rejected by throttling first and then later rejected by Spamhaus. We will look at these IPs in more detail as these are the ones that will allow us to compare Spamhaus vs throttling.

These 559,562 IPs were rejected by throttling on average for 4 days (351,682 seconds) prior to being rejected by Spamhaus. This ranged from 1 second to almost the entire time-frame of this analysis of 66 days (5,771,484 seconds). Therefore it is not unreasonable to assume that a spambot could go undetected by Spamhaus for longer than this time frame.

These 559,562 IPs were rejected by throttling on average 30 times, ranging from once to 158,378 times per IP.

Once these 559,562 IPs were detected by Spamhaus, Spamhaus then rejected on average 116 times, ranging from once to 220,084 times per IP.

Disregarding throttling rejections, if we look at the 26,340,897 IPs that at some point were rejected by Spamhaus, Spamhaus rejected on average 30 times, ranging from once to 1,034,038 times per IP. This shows that the IPs that were first detected by throttling go on to be more actively rejected by Spamhaus : 116 times versus only 30 times for ones we did not first detect by throttling. Something to consider is that throttling is likely to see newer IPs, since Spamhaus has not recognised them yet. We can assume that newer IPs send more spam.

Since we only throttle reject on average for 4 days (see above) before Spamhaus takes over, our average of 30 rejections per IP is actually higher than Spamhaus’s average of 30 rejections per IP, since this is averaged over the entire time frame of 66 days.

Following is a graph showing the amount of time it took for Spamhaus to “catch up” to the IPs that we were able to reject successfully with throttling: