5 ms·

I'm pretty surprised reddit's backend doesn't just automatically devalue this kind of vote cheating. In the general case, it's really easy to detect, and if you

I'm pretty surprised reddit's backend doesn't just automatically devalue this kind of vote cheating. In the general case, it's really easy to detect, and if your cheater is a bit more sophisticated, it's still not that hard, it takes a really serious person to have five accounts on five different browsers/vpcs that each come from different IP addresses consistently, even then, when you see them colluding together, you can still devalue those votes simply because it's an obvious clique taking early action consistently, so even then it should be pretty detectable. I don't see a reason to ban someone over this behavior as long as you can make sure that the fraudulent votes don't actually add any value to the content, which they shouldn't.

raverbashing 12y ago
I think the real issue is the relative weight between the first votes and the rest
deleted 12y ago
[deleted]
dang 12y ago
> In the general case, it's really easy to detect Why do you say that? I've worked hard on this problem and would not call it easy.
- codezero 12y ago
  When I say general case, I mean random person creates an account or two to upvote their own stuff. This is usually trivial to detect if you save a single cookie that's linked to each account, even easier if you are also using more tenacious tracking like flash cookies or localstorage or the dreaded evercookie. In the less general case, you can still determine that a particular group of accounts never take action in the same small time windows. The way I've approached this problem in the past relies heavily on meta data about the user. It's easy to determine if an IP is a proxy/VPN, which should immediately make a client more suspicious (when you're already suspicious about fraudulent voting). Initially, account ages are important but if someone is able to get away with it for a long time, they can get complacent and keep using the same accounts, and at that point you can look at things like upvote/downvote ratios (or generally the lack of diversity in voting activity) compared to some expected average activity. There's a lot of other tricks, but I don't know what your approach has been when working on this, and I know HN is a much different platform from what I worked with :P
- lliwta 12y ago
  >> In the general case, it's really easy to detect > Why do you say that? I've worked hard on this problem and would not call it easy. Are there reference data sets for this problem? If not, you or reddit should publish data sets.
  - minimaxir 12y ago
    The issue with giving information as to how a voting detector work is that it would make it easier to cheat.
    - lliwta 12y ago
      Not the algorithm, the data. Anonymized user ids and their votes, together with relevant meta-data. I suppose the fact that there's already a detection algorithm biases the data, but it'd still be better than nothing.
agent00f 12y ago
> I don't see a reason to ban someone over this behavior as long as you can make sure that the fraudulent votes don't actually add any value to the content, which they shouldn't. Visibility of the content is essentially predicated on early votes. The problem is that an initial sort is built on limited data (quality content or just easy to upvote produces same results and latter is more common), and the subsequent feedback loop is built on initial visibility decisions.
opendais 12y ago
No detection mechanism is 100% reliable so better to ban 100% of the ones you know aren't false positives than risk them trying again and being a false negative.
- codezero 12y ago
  Not really. Devaluing a single vote insitu once in a while shouldn't be that negative on the system as a whole, especially if there is suspected fraud. The problem with banning fraudulent users is that they are tenacious, thats's a big factor in their committing fraud in the first place, they are usually on a "mission," see: https://en.wikipedia.org/wiki/Wikipedia:Single-purpose_account https://en.wikipedia.org/wiki/Wikipedia:Single-purpose_accou... (not really the same as a fraudster, but this is a common archetype) They will just make another account or increase their efforts, better to keep them in their container within the system.
  - opendais 12y ago
    You seem to believe they won't to notice the effect on their "container within the system". They will eventually, just like a shadowban. At which point, they create a new account. > while shouldn't be that negative on the system as a whole, especially if there is suspected fraud. Assuming it isn't computationally expensive to do this on a per-vote basis. If you have to do any sophisticated analysis, that may not be true.
DanBC 12y ago
Imagine a group of 30 people who use the site regularly. When an URLs are posted the each roll a die. If it comes up 1 then they up vote. If it comes up 6 then they downvote. If it comes up 2, 3, 4, or 5 then they do nothing. That takes 30 people and it requires discipline to oney the die, but it'd be pretty hard to detect. I think, maybe I'm wrong?
- codezero 12y ago
  Yeah this would be hard to detect, but voting fraud is usually targeted, so you can look at the specific things they have historically voted on and determine if they only vote on X type of thing or posts from Y, or only vote at times when Z just posted a URL, etc...
raldi 12y ago
> I'm pretty surprised reddit's backend doesn't just automatically devalue this kind of vote cheating. What makes you so sure it doesn't?
- codezero 12y ago
  That they have to ban people who do this is a pretty good sign that it's not working well. Subreddits have pretty intense human moderation which also tends to negate the need for sophisticated detection since people are pretty good at spotting this sort of thing on their own. Also, isn't the reddit source public? I presume that if it had this, it'd be easy to find it there, but I haven't looked, maybe I should :)
  - raldi 12y ago
    It's public except for the large, sophisticated body of anti-cheating code.
    - codezero 12y ago
      Are there any write ups about this anywhere? Sounds interesting. Also, is any information from that system provided to moderators?
      - raldi 12y ago
        http://www.redditblog.com/2008/06/reddit-goes-open-source.html?m=1 http://www.redditblog.com/2008/06/reddit-goes-open-source.ht... https://github.com/reddit/reddit/blob/master/README.md https://github.com/reddit/reddit/blob/master/README.md http://www.reddit.com/wiki/moderation http://www.reddit.com/wiki/moderation
        codezero 12y ago
        Thanks!

raverbashing 12y ago

I think the real issue is the relative weight between the first votes and the rest

deleted 12y ago

[deleted]

dang 12y ago

> In the general case, it's really easy to detect Why do you say that? I've worked hard on this problem and would not call it easy.

codezero 12y ago
When I say general case, I mean random person creates an account or two to upvote their own stuff. This is usually trivial to detect if you save a single cookie that's linked to each account, even easier if you are also using more tenacious tracking like flash cookies or localstorage or the dreaded evercookie. In the less general case, you can still determine that a particular group of accounts never take action in the same small time windows. The way I've approached this problem in the past relies heavily on meta data about the user. It's easy to determine if an IP is a proxy/VPN, which should immediately make a client more suspicious (when you're already suspicious about fraudulent voting). Initially, account ages are important but if someone is able to get away with it for a long time, they can get complacent and keep using the same accounts, and at that point you can look at things like upvote/downvote ratios (or generally the lack of diversity in voting activity) compared to some expected average activity. There's a lot of other tricks, but I don't know what your approach has been when working on this, and I know HN is a much different platform from what I worked with :P
lliwta 12y ago
>> In the general case, it's really easy to detect > Why do you say that? I've worked hard on this problem and would not call it easy. Are there reference data sets for this problem? If not, you or reddit should publish data sets.
- minimaxir 12y ago
  The issue with giving information as to how a voting detector work is that it would make it easier to cheat.
  - lliwta 12y ago
    Not the algorithm, the data. Anonymized user ids and their votes, together with relevant meta-data. I suppose the fact that there's already a detection algorithm biases the data, but it'd still be better than nothing.

agent00f 12y ago

> I don't see a reason to ban someone over this behavior as long as you can make sure that the fraudulent votes don't actually add any value to the content, which they shouldn't. Visibility of the content is essentially predicated on early votes. The problem is that an initial sort is built on limited data (quality content or just easy to upvote produces same results and latter is more common), and the subsequent feedback loop is built on initial visibility decisions.

opendais 12y ago

No detection mechanism is 100% reliable so better to ban 100% of the ones you know aren't false positives than risk them trying again and being a false negative.

codezero 12y ago
Not really. Devaluing a single vote insitu once in a while shouldn't be that negative on the system as a whole, especially if there is suspected fraud. The problem with banning fraudulent users is that they are tenacious, thats's a big factor in their committing fraud in the first place, they are usually on a "mission," see: https://en.wikipedia.org/wiki/Wikipedia:Single-purpose_account https://en.wikipedia.org/wiki/Wikipedia:Single-purpose_accou... (not really the same as a fraudster, but this is a common archetype) They will just make another account or increase their efforts, better to keep them in their container within the system.
- opendais 12y ago
  You seem to believe they won't to notice the effect on their "container within the system". They will eventually, just like a shadowban. At which point, they create a new account. > while shouldn't be that negative on the system as a whole, especially if there is suspected fraud. Assuming it isn't computationally expensive to do this on a per-vote basis. If you have to do any sophisticated analysis, that may not be true.

DanBC 12y ago

Imagine a group of 30 people who use the site regularly. When an URLs are posted the each roll a die. If it comes up 1 then they up vote. If it comes up 6 then they downvote. If it comes up 2, 3, 4, or 5 then they do nothing. That takes 30 people and it requires discipline to oney the die, but it'd be pretty hard to detect. I think, maybe I'm wrong?

codezero 12y ago
Yeah this would be hard to detect, but voting fraud is usually targeted, so you can look at the specific things they have historically voted on and determine if they only vote on X type of thing or posts from Y, or only vote at times when Z just posted a URL, etc...

raldi 12y ago

> I'm pretty surprised reddit's backend doesn't just automatically devalue this kind of vote cheating. What makes you so sure it doesn't?

codezero 12y ago
That they have to ban people who do this is a pretty good sign that it's not working well. Subreddits have pretty intense human moderation which also tends to negate the need for sophisticated detection since people are pretty good at spotting this sort of thing on their own. Also, isn't the reddit source public? I presume that if it had this, it'd be easy to find it there, but I haven't looked, maybe I should :)
- raldi 12y ago
  It's public except for the large, sophisticated body of anti-cheating code.
  - codezero 12y ago
    Are there any write ups about this anywhere? Sounds interesting. Also, is any information from that system provided to moderators?
    - raldi 12y ago
      http://www.redditblog.com/2008/06/reddit-goes-open-source.html?m=1 http://www.redditblog.com/2008/06/reddit-goes-open-source.ht... https://github.com/reddit/reddit/blob/master/README.md https://github.com/reddit/reddit/blob/master/README.md http://www.reddit.com/wiki/moderation http://www.reddit.com/wiki/moderation
      - codezero 12y ago
        Thanks!