3 ms·

To publish 'a how we did it' article like this suggests to me that the Twitter engineering team are either justifiably confident in their spam fighting creation

by junto 12y ago

To publish 'a how we did it' article like this suggests to me that the Twitter engineering team are either justifiably confident in their spam fighting creation (because they are confident it is infallible), or they are naively supremely over-confident; thus publicizing how this works will come back to bite them in the butt. Only time will tell I guess. From the article it would appear that they aren't doing anything magically different with regards to classification of twitter-spam, but they have found a way to deal with the volume of classification tasks in a pertinent manner. It also gives them a way to quickly respond to new types of spam attacks. Very interesting. They should consider opening it up as an Askimet competitor. The difference between a blog comment and a twitter post is negligible.

thewarrior 12y ago
They haven't revealed the conditions they use to determine if something is spam.
possibilistic 12y ago
> publicizing how this works will come back to bite them in the butt I didn't see any rules or heuristics published. Merely that they employ a multi-stage filter (as any engineer could imagine), and that they codify their ruleset with a human-readable DSL (which is kind of interesting, but also kind of weird). > The difference between a blog comment and a twitter post is negligible. The difference in textual content between a tweet and blog comments falling within a similar character length are perhaps negligible, though there is often no such length constraint in place for blogs. Comment spam strategies are free to vary string length to optimize for evasion, click through, proliferation, and other criteria. I would also argue there is a different demographic distribution between Twitter and blogs with regard to readership and participation. That said, Twitter just outlined a number of reasons why their case is special. They are a high-availability, high-volume, low-latency service. Twitter's spam solution was designed to handle their very special set of constraints. I think a multi-stage filter complete with asynchronous post-processing jobs would be a bit much for your average Wordpress blog. People just starting out with (say, PHP) probably can't fathom a multiprocess deployment architecture. Not to say they couldn't, but the journey is a long one.
- benaiah 12y ago
  > I think a multi-stage filter complete with asynchronous post-processing jobs would be a bit much for your average Wordpress blog. People just starting out with (say, PHP) probably can't fathom a multiprocess deployment architecture. Not to say they couldn't, but the journey is a long one. That's why he suggested an Akismet competitor - in case you haven't used it, Akismet is a SAAS solution that filters your comments on their servers, not on your own, so you don't have to worry about the architecture or deployment.
  - possibilistic 12y ago
    My mistake. Thanks for the correction. :)
junto 12y ago
My point appears to have been missed and I was down voted to hell for it, but I merely trying to point out that when you are fighting a war (as they are against spammers) then you shouldn't give away any secrets, no matter how insignificant they may appear now.

by junto 12y ago

thewarrior 12y ago

They haven't revealed the conditions they use to determine if something is spam.

possibilistic 12y ago

> publicizing how this works will come back to bite them in the butt I didn't see any rules or heuristics published. Merely that they employ a multi-stage filter (as any engineer could imagine), and that they codify their ruleset with a human-readable DSL (which is kind of interesting, but also kind of weird). > The difference between a blog comment and a twitter post is negligible. The difference in textual content between a tweet and blog comments falling within a similar character length are perhaps negligible, though there is often no such length constraint in place for blogs. Comment spam strategies are free to vary string length to optimize for evasion, click through, proliferation, and other criteria. I would also argue there is a different demographic distribution between Twitter and blogs with regard to readership and participation. That said, Twitter just outlined a number of reasons why their case is special. They are a high-availability, high-volume, low-latency service. Twitter's spam solution was designed to handle their very special set of constraints. I think a multi-stage filter complete with asynchronous post-processing jobs would be a bit much for your average Wordpress blog. People just starting out with (say, PHP) probably can't fathom a multiprocess deployment architecture. Not to say they couldn't, but the journey is a long one.

benaiah 12y ago
> I think a multi-stage filter complete with asynchronous post-processing jobs would be a bit much for your average Wordpress blog. People just starting out with (say, PHP) probably can't fathom a multiprocess deployment architecture. Not to say they couldn't, but the journey is a long one. That's why he suggested an Akismet competitor - in case you haven't used it, Akismet is a SAAS solution that filters your comments on their servers, not on your own, so you don't have to worry about the architecture or deployment.
- possibilistic 12y ago
  My mistake. Thanks for the correction. :)

junto 12y ago

My point appears to have been missed and I was down voted to hell for it, but I merely trying to point out that when you are fighting a war (as they are against spammers) then you shouldn't give away any secrets, no matter how insignificant they may appear now.