4 ms·
One of the ways Askimet works is shared-secret-key authentication using your email address. Notice how sites collect your email address when you submit a commen
by briansmith 18y ago
One of the ways Askimet works is shared-secret-key authentication using your email address. Notice how sites collect your email address when you submit a comment but they don't usually publish it. However, they do send your email address to Askimet. As a result, we can get a fairly good approximation of whether a comment is spam or not simply by checking to see if the email address was used for comments previously flagged as spam or ham. The more non-spam comments--and especially the more ham comments--that the email address has in the corpus, the less likely it is that the new message is spam.
Now, imagine I am a spammer. I need to submit some automated comments that don't look like spam so that my spammy comments wll later get accepted. There are a variety of ways I can do this. The easiest is to give the user a generic compliment like "Wow, this was a great blog post! Thanks for publishing it." The comment doesn't add to the discussion but the blog owner is often reluctant to delete it because (a) it strokes his ego, (b) there is no way for him to tell if it is automatically generated or not. In fact, if his spam filter preemptively classified it as spam, he might even override the filter and reclassify it as ham; this would be a huge win for the spammer-to-be. Another way to do this is to automatically scrape a comment from another discussion about the page like Reddit, Digg, Twitter, or FriendFeed and submit it as a blog comment on the original page. A third way is to pay somebody a (very) small amount of money to read the blog post and write a comment (possibly using some fill-in-the-blanks response template system like those used in call centers.)
It works the other way too. If I want your comments to start getting flagged as spam then I should start submitting spam-like comments in your name. Then, users will start classifying my forged spam comments as spam and the automatic classifiers will start automatically classifying your valid comments as spam.
- briansmith 18y agoI forgot to add the corollary. Much like computers will soon be able to solve CAPTCHAs better than people, automated spamming systems will eventually be able to generate comments that are as good or better than human-submitted comments. In other words, we can expect comments that are generated by spammers to eventually become valuable enough that we want to retain them instead of filtering them away. That isn't the reality today but it could be soon. You might have a discussion in your blog's comments and then realize that everybody in the conversation is a robot except you. And you will learn something from those robots and/or lose an argument with them, when they get really good. Eventually, you will be able to post a rough draft of your blog post, and the spam robots will start up a good conversation about it in the comments. Then, based on their feedback, you can revise it into the final draft (if the robots haven't already given you a link to a blog post that refuted your point or made it better than you can). As a final step, we might notice that not only are the best comments are written by spam robots, but the best blogs are spam blogs too. Right now, people are blogging-for-adsense manually; in the future, Google will be able to blog-for-adsense itself, cutting out the authors. Today, if a site wants its user-generated content (comments in this case) to retain its value, it needs to start filtering out the low-value content, regardless of whether it was generated by a spammer or a well-intentioned user. For example, If I add a comment to your blog "Great blog post!" on your blog, you should probably delete it, even if you know I am not a spammer. Unfortunately, if you do that, legitimate users who see their complimentary comments get deleted might react negatively (e.g. stop commenting or complain loudly, generating even worse comments). So, we can see that the spammers who are generating low-value comments already have the ability to drive away your valuable commentators if those commentators occasionally submit low-value comments.
- thepanister 18y agoThis looks so scary. we can expect comments that are generated by spammers to eventually become valuable enough that we want to retain them instead of filtering them away What is the point? Why spammers would submit a valuable comment? Just to be kind of a door for them to enter through later, and nobody will think it's a spam?
- briansmith 18y agoWhen you submit a comment to a blog, you usually get to include a link to a website along with the comment. Usually the commentator's name is linked to that website in the resulting comment page. If my spam comment is good, a very small number of people will click my username and go to the site I choose. The more comments I get published, the more clicks I get on my link. If that is a spam link, I get paid for every one of those clicks. Or, a spammer might generate some good comments just to get the comment classifier to mis-classify a blatantly spammy comment that will be submitted later.
- thepanister 18y agoYou are more than a genius! Do you have any publications or a blog? I am very interested in learning more from you... you look like a scientist! EDIT: But there is a point here that you did not really mention! You assumed that spammers will be smarter, but spam filters will still stupid? Don't you think that as spammers build a smarter bots, the spam-fighters will build a smarter spam filters?
- briansmith 18y agoSee, this was a very nice demonstration. How am I supposed to tell if thepanister is a robot or not? Based on the content of this message it could go either way.
- thepanister 18y agoLOOOL But I edited it to look a man-made. :) Please re-read it again.
- DaniFong 18y agoIt seems to work this way in the gmail spam filter too, but I didn't want to tell anyone for fear of it being exploited. Apparently the cat's out of the bag, however.