3 ms·

the study itself is a bit "pnas'y". In fact, a lot of work studies information aggregation in different kinds of settings. As you, I think, suspect, things can

by zwaps 5y ago

the study itself is a bit "pnas'y". In fact, a lot of work studies information aggregation in different kinds of settings. As you, I think, suspect, things can also easily go the other way. Well known is the fact that extensive information / interaction networks can decrease diversity (a general fact of averaging mechanisms in network settings that holds even in Games on networks). A decrease in diversity in information aggregation is often detrimental (for example through correlation of errors, canceling out one mechanism of wisdom of crowds, or through conformism). Further, wisdom of crowds only exists (relative to individual guesses) if in growing networks the opinions are not controlled by influential groups or echo chambers (Jackson did work on this). This of course, is what ends up happening a lot, in part due due to aforementioned factors. Since the proposed mechanism relies on weighting accurate individuals higher, it is even more susceptible to such biases than just wisdom of crowds in general.

AstralStorm 5y ago
Funny. The rating mechanism makes this equivalent to a reputation based prediction market. These can also fail and are gameable. The key assumptions are that people form social networks according to DeGroot process, which is not actually true. People tend to maximize objective gain not information/influence gain - usually objective is not stated and there are various ones, from knowledge, entertainment, desirability, clout/influence and power, even financial ones. Unfortunately, this does result in a divergent scenario. As for feedback, current social networks have simplistic "like" system that only weights engagement in a hidden matter and generic desirability points with partial feedback scenario. Trying to get full feedback would require someone to be able to read likes of every user, not feasible. What could be feasible is guessing how "likeable" a post is - gaming the system. That said, the paper is rather high quality considering its limitations. Interesting that essentially listening to a moderate number of curated but not isolated experts had best results. (Top 5 interesting enough, this being ~33% of group.) The trick is in identifying these top performers in a real multiobjective scenario, and whether the selection strategy still works.
- forgotmypw17 5y ago
  >"Trying to get full feedback would require someone to be able to read likes of every user, not feasible." I think that with a fully transparent system this is not only feasible, but easy.
- sysadm1n 5y ago
  > These can also fail and are gameable Gameable perhaps, but in proper meritocratic[0] systems, voting rings etc would be detected and the perpetrators hopefully banned and they would also lose the ability to participate again at a later stage, giving an incentive to play fairly next time. [0] https://en.wikipedia.org/wiki/Meritocracy https://en.wikipedia.org/wiki/Meritocracy
- machinehermiter 5y ago
  Thanks for this interaction. Going to think about this while I go on a walk.

by zwaps 5y ago

AstralStorm 5y ago

Funny. The rating mechanism makes this equivalent to a reputation based prediction market. These can also fail and are gameable. The key assumptions are that people form social networks according to DeGroot process, which is not actually true. People tend to maximize objective gain not information/influence gain - usually objective is not stated and there are various ones, from knowledge, entertainment, desirability, clout/influence and power, even financial ones. Unfortunately, this does result in a divergent scenario. As for feedback, current social networks have simplistic "like" system that only weights engagement in a hidden matter and generic desirability points with partial feedback scenario. Trying to get full feedback would require someone to be able to read likes of every user, not feasible. What could be feasible is guessing how "likeable" a post is - gaming the system. That said, the paper is rather high quality considering its limitations. Interesting that essentially listening to a moderate number of curated but not isolated experts had best results. (Top 5 interesting enough, this being ~33% of group.) The trick is in identifying these top performers in a real multiobjective scenario, and whether the selection strategy still works.

forgotmypw17 5y ago
>"Trying to get full feedback would require someone to be able to read likes of every user, not feasible." I think that with a fully transparent system this is not only feasible, but easy.
sysadm1n 5y ago
> These can also fail and are gameable Gameable perhaps, but in proper meritocratic[0] systems, voting rings etc would be detected and the perpetrators hopefully banned and they would also lose the ability to participate again at a later stage, giving an incentive to play fairly next time. [0] https://en.wikipedia.org/wiki/Meritocracy https://en.wikipedia.org/wiki/Meritocracy
machinehermiter 5y ago
Thanks for this interaction. Going to think about this while I go on a walk.