3 ms·

I don't know Arc, but I'm happy to write out the idea in simpler instructions than the blog post if it helps. The implicit approach is very practical. You basi

by tansey 15y ago

I don't know Arc, but I'm happy to write out the idea in simpler instructions than the blog post if it helps. The implicit approach is very practical. You basically just need to bootstrap it and then it will run itself. To bootstrap: - Track if each user has seen an article or not. - Track how many times each user flags any article, let's call this any_flagged - Add an admin-only "honeypot" button (or track articles flagged by admins) - When an admin marks an article as a honeypot: 1) Increment the honeypot_seen counter of anyone who sees (or has seen) the article. 2) Increment the honeypot_flagged counter of anyone who flags the article. 3) Increment the honeypot_upvoted counter of anyone who upvotes the article. - Repeat until you are happy with the number of honeypots and the coverage of the community. Intuition says if you focus on front-page articles, then you should be fine after 30 or so flagged articles. Then calculate your super flaggers: - Apply the h-formula to each user, h(u) = (honeypot_flagged - honeypot_upvoted) / (honeypot_seen * any_flagged) - Select the top N% to be super flaggers. Again, intuition would say 5-10% is reasonable, but that depends on the way the data looks. Set it to implicit mode. Now, each article has super flags tracked and when its super flag threshold (percentage of super flaggers) is crossed, you declare it a honeypot. Then you run a process analogous to the one in the bootstrapping phase.

SkyMarshal 15y ago
Clever idea in general, a few issues: 1. Sometimes people upvote without reading the article just because the comments are good, and want others to benefit from that as well. 2. Sometimes people upvote to save a submission, since there's no separate save function. For example, I want to send it to a friend later, but not right now, so I upvote it to make it easier to find (b/c HN search is hit-or-miss). I confess I'm guilty of these, but I doubt I'm the only one. By way of comparison, take Reddit. Upvoting and saving are separate functions. You can save submissions you think you might want to revisit later. Reasons for doing so: 1. Scanning the headlines quickly, but don't have time to actually read everything, want to save the article and comments for later perusal (lunch break, after work, whatever). 2. Subreddits reduce the cost of upvoting. For example, every time I consider whether to upvote a Bitcoin story on HN, I consider whether it's front-page worthy. On Reddit, that's not a problem, I can just assume it won't hit the general front page b/c it's a relatively niche subject, but the upvote might help it within its own subreddit. One more potential problem with the idea of superflaggers. If 'social media experts', or spammers, or whatever the people are who game sites like Digg and Reddit got wind of the fact that flagging honeypots could increase the weight of their flags and/or votes, mightn't they also figure out how to abuse that? A professional, as some of them seem to be (eg, able to spend all day every day doing this), might be able to achieve a denominator very close to 1, and a numerator close to the total actual honeypots.
- tansey 15y ago
  > One more potential problem with the idea of superflaggers. If 'social media experts', or spammers, or whatever the people are who game sites like Digg and Reddit got wind of the fact that flagging honeypots could increase the weight of their flags and/or votes, mightn't they also figure out how to abuse that? A professional, as some of them seem to be (eg, able to spend all day every day doing this), might be able to achieve a denominator very close to 1, and a numerator close to the total actual honeypots. Super flaggers receive no special powers other than the ability to contribute to the honeypot score of a given article. Their votes and flags are counted the same as a normal user. I addressed this in more detail in another comment below, but basically the ability to and utility of gaming this system is minuscule.
  - pbhjpbhj 15y ago
    >Super flaggers receive no special powers other than the ability to contribute to the honeypot score of a given article. // No special powers other than helping to ensure their submissions are not 'honeypotted' and submissions contrary to their view are 'honeypotted'? Wouldn't that also downgrade those who hold contrary views to them - as the contrarians would be more likely to upvote the stories that the gamers are helping to get marked as honeypots - thus ensuring that the gamers keep those with opposing views from gaining a position in the quality control caucus? I notice you're in AI, have you run some formalised tests on how such a voting system would play out? My personal (untested) preference is towards making voting plain and all scores open and then letting users somehow create their own metric for filtering. Perhaps that won't work on the scale of a successful site though. Are there actually upvoting and flagging cabals in operation on this site now?
    - tansey 15y ago
      > No special powers other than helping to ensure their submissions are not 'honeypotted' and submissions contrary to their view are 'honeypotted'? Please see my detailed explaination of why this is not a problem [1]. > I notice you're in AI, have you run some formalised tests on how such a voting system would play out? No. If there is a top-tier conference publication in it, I would be happy to do some MC runs. That being said, this is not really a publishable idea unless I can actually implement it and measure the results somehow on a real site. :) > Are there actually upvoting and flagging cabals in operation on this site now? Probably not. There definitely are such rings on Digg and reddit (I know for a fact). This is a general system, so it could be useful on any social news site. [1] http://news.ycombinator.com/item?id=3166548 http://news.ycombinator.com/item?id=3166548
chollida1 15y ago
> Track how many times each user flags any article, let's call this any_flagged This is a pretty fatal flaw in your plan. Not everyone can flag and it seems if you flag too much you loose the ability to flag. This restriction would have to be relaxed before your plan could be tested.

by tansey 15y ago

SkyMarshal 15y ago

Clever idea in general, a few issues: 1. Sometimes people upvote without reading the article just because the comments are good, and want others to benefit from that as well. 2. Sometimes people upvote to save a submission, since there's no separate save function. For example, I want to send it to a friend later, but not right now, so I upvote it to make it easier to find (b/c HN search is hit-or-miss). I confess I'm guilty of these, but I doubt I'm the only one. By way of comparison, take Reddit. Upvoting and saving are separate functions. You can save submissions you think you might want to revisit later. Reasons for doing so: 1. Scanning the headlines quickly, but don't have time to actually read everything, want to save the article and comments for later perusal (lunch break, after work, whatever). 2. Subreddits reduce the cost of upvoting. For example, every time I consider whether to upvote a Bitcoin story on HN, I consider whether it's front-page worthy. On Reddit, that's not a problem, I can just assume it won't hit the general front page b/c it's a relatively niche subject, but the upvote might help it within its own subreddit. One more potential problem with the idea of superflaggers. If 'social media experts', or spammers, or whatever the people are who game sites like Digg and Reddit got wind of the fact that flagging honeypots could increase the weight of their flags and/or votes, mightn't they also figure out how to abuse that? A professional, as some of them seem to be (eg, able to spend all day every day doing this), might be able to achieve a denominator very close to 1, and a numerator close to the total actual honeypots.

tansey 15y ago
> One more potential problem with the idea of superflaggers. If 'social media experts', or spammers, or whatever the people are who game sites like Digg and Reddit got wind of the fact that flagging honeypots could increase the weight of their flags and/or votes, mightn't they also figure out how to abuse that? A professional, as some of them seem to be (eg, able to spend all day every day doing this), might be able to achieve a denominator very close to 1, and a numerator close to the total actual honeypots. Super flaggers receive no special powers other than the ability to contribute to the honeypot score of a given article. Their votes and flags are counted the same as a normal user. I addressed this in more detail in another comment below, but basically the ability to and utility of gaming this system is minuscule.
- pbhjpbhj 15y ago
  >Super flaggers receive no special powers other than the ability to contribute to the honeypot score of a given article. // No special powers other than helping to ensure their submissions are not 'honeypotted' and submissions contrary to their view are 'honeypotted'? Wouldn't that also downgrade those who hold contrary views to them - as the contrarians would be more likely to upvote the stories that the gamers are helping to get marked as honeypots - thus ensuring that the gamers keep those with opposing views from gaining a position in the quality control caucus? I notice you're in AI, have you run some formalised tests on how such a voting system would play out? My personal (untested) preference is towards making voting plain and all scores open and then letting users somehow create their own metric for filtering. Perhaps that won't work on the scale of a successful site though. Are there actually upvoting and flagging cabals in operation on this site now?
  - tansey 15y ago
    > No special powers other than helping to ensure their submissions are not 'honeypotted' and submissions contrary to their view are 'honeypotted'? Please see my detailed explaination of why this is not a problem [1]. > I notice you're in AI, have you run some formalised tests on how such a voting system would play out? No. If there is a top-tier conference publication in it, I would be happy to do some MC runs. That being said, this is not really a publishable idea unless I can actually implement it and measure the results somehow on a real site. :) > Are there actually upvoting and flagging cabals in operation on this site now? Probably not. There definitely are such rings on Digg and reddit (I know for a fact). This is a general system, so it could be useful on any social news site. [1] http://news.ycombinator.com/item?id=3166548 http://news.ycombinator.com/item?id=3166548

chollida1 15y ago

> Track how many times each user flags any article, let's call this any_flagged This is a pretty fatal flaw in your plan. Not everyone can flag and it seems if you flag too much you loose the ability to flag. This restriction would have to be relaxed before your plan could be tested.