Defensio: Outsmarting Evil Spam

In the “traditional” world of blog spam filters, spam accumulates in your spambox, sometimes at an alarming rate.

Occasionally legitimate comments, left by the users of your blog, will be eaten by your filter — an expected and normal occurrence (if not somewhat annoying) as no filter can ever be perfect.

When such “false positives” occur, one of 3 things will happen:

1) The commenter may send you a belligerent polite email notifying you that his/her comment failed to make it through to your blog;

2) If you’re fastidious and eagle-eyed, you may notice the errant comment in your spambox and restore it before its absence can be noticed by too many readers;

3) You may fail to pick up on the comment as it becomes buried in spam, sucked into a black hole from which it will never emerge.

If you’ve spent any time blogging, it’s almost a given that #1 happens to you every other day. And as we all know, good blogging practice requires that you keep your garden clean and so you’ll probably perform #2 on a regular basis, typically digging out a handful of erroneously caught comments whenever you check.

But, the million dollar question: How often does #3 occur? With what frequency are legitimate tidbits of commentary/knowledge passed on by your readers but never made live to the web? How much conversation spirals down the dark recesses of the urinal drain?

Sadly, it’s impossible to say. Your spambox may be riddled with such false-positives, but it is probably so overflowing and poorly chronologically organized that seeking out each and every one of these errors is simply impractical if not done on a daily basis.

If any of this resonates for you, we think you’ll jump for joy for Defensio which puts an end to the blackhole problem by:

• Sorting your spambox by the “spaminess” of each comment, so that those least likely to be spam (i.e. most likely to be false positives) bubble up to the top for easy identification;
• Providing you with an RSS feed of spam (and legitimate) comments, so that you can easily and quickly identify mistakes without having to log into your blogging platform;
• Cranking up the dial on performance through individualized and continuous learning algorithms, so that less errors occur, period.

Download Defensio now, and stop losing comments to spam.

We here at Defensio HQ see a lot of spam; spam in all its flavors and incarnations. Occasionally we see new techniques that baffle the mind. URL-less spam (that is, spam not containing URLs) is one of these baffling new forms of spam we’ve seen cross our desk, so puzzling that it’s worth delving in to try to understand what in the world it means.

Example

URL-less spam looks like the following:

Notice that this commenter (i.e. spammer) has not left a URL with his/her credentials, nor has he/she supplied any URLs in the body of the comment.

The Issue

Why is this strange? Because the entire reason spammers typically hit blogs with their bogus comments is to populate the web with URLs that link back to their spammy sites, and thus manage to exploit the Google juice of the sites they breach with the goal of boosting their own search engine rank. And so, bombarding a blog with comments that do not contain URLs defeats the whole purpose, and results in no obvious net benefit to the spammer, other than the evil satisfaction of annoying the hell out of bloggers.

Motives

So if not to exploit Google juice, why do spammers go with a URL-less approach? Two theories:

1) To “train” spam filters to allow specific keywords.

Filters that use statistical filtering learn over time. By having legitimate-looking comments make it through the filter, while containing a handful of specifically-chosen keywords, spammers could be trying to tip statistical filters toward starting to consider such keywords as innocent, thus increasing the likelihood that future spam comments containing these words will bypass spam defenses.

2) To be whitelisted.

Some spam filters allow users that successfully post comments X number of times to be added to a whitelist, meaning they will bypass the filter in the future. Since URL-less spam typically looks fairly normal, spammers hope that bloggers will fail to identify their comment as spam enough times that auto-whitelisting might kick in.

These motives are simply our best guesses at what might be in spammers’ nefarious minds. Who knows, simple annoyance could be their sole, inexplicable, goal?

When working in the spam space (dare I say spamosphere?) you have to be careful not to disclose too much about what goes on “under the hood”, as spammers are always seeking to exploit any informational edge they can. At the same time, we think it’s important to make sure our community understands what Defensio is all about, and the philosophy behind our approach.

We believe, as most of you will, that comment spam is a big problem (and getting worse). The way we see it, there are two ways to attack the issue:

1) At the source: trying to shut down the spammers altogether
2) At the destination: trying to stop spam as it attempts to hit your blog

The first option is a valuable approach, and there are many initiatives going on to try to thwart spammers directly (such as pursuing them through legal/criminal action or developing new technical specifications that would make spam impossible). This is certainly the ideal way forward, but its Achilles heel is timing and consensus. Making these wheels turn is a long, slow process - and all the while, spam will continue to pour into the blogosphere like a meaty, raging waterfall, and the techniques used will continue to evolve. And let’s not forget that actively pursuing thousands of spammers all over the world is a logistical and jurisdictional nightmare, especially when you consider the grey zone in which spammers operate in many countries without spam legislation.

At Defensio, we adopt the second option, not because it’s better, but because it’s the pragmatist’s approach. We (reluctantly) accept that the spam firehose will likely continue to be aimed at our collective heads for some time to come, and so we feel the need to develop good tools to handle the onslaught. And by better tools we mean both improving how spam is caught and how spam is managed. Some may think it defeatist, (or a “leaky condom” as one colorful critic called it) but we call it practical and effective.

So what this means is that:

• Defensio’s web service is designed to filter all incoming comments and trackbacks to your blog, banishing spam to a quarantine (using of secret-sauce of counter-spammy intelligence)
• Provide you with a convenient and hassle-free way of sorting through quarantined comments
• Learn and improve over-time, in a way that is personalized to your individual blog

Hope that sheds some light on our philosophy and some of the high-level mechanics of the Defensio platform!

Of course, you’ll have a chance to see this in action for yourself once we start rolling out accounts to our eager beta-testers (soon, we promise!).