14 ms·
Trust-based moderation systems
- curtisblaine 3y agoAs far as I understand, every user must know the trust network of all other users, which 1) doesn't scale much 2) has terrible privacy implications.
- wanderingbit 3y agoProblem (1) can likely be addressed by restricting the number of iterations to some small constant, like 3 to 6 (which they suggest in the article). You can also restrict the number of peers-of-peers you fetch during each iteration to a small random subset. So if we only iterate 6 times and choose 10 peers for each iterations, we’ll get 1 million (10^6) trust scores needing to be pulled to calculate a trust score. That is an upper bound and will likely be less because it assumes each peer is distinct. At 32-bit floating points, that’s 32 million bits or 4 MB necessary to be fetched. I can imagine this being reduced by at least a factor of 10 without much impact on the trust score. But note the “random subset” means different people will have different trust scores for the same peer :shrug: For (2) yeah we probably need to lower our expectations on privacy for the time being; it’s a masters thesis and privacy in open distributed systems is very tricky.
- _heimdall 3y ago> For (2) yeah we probably need to lower our expectations on privacy for the time being; it’s a masters thesis and privacy in open distributed systems is very tricky. If we do care about privacy, any moderation system should first be designed to meet that goal. Its not worth designing a moderation system if we don't first know it will work with one of the core requirements.
- _factor 3y agoIt’s needs a republic structure. Every person joins a small default group of random strangers, their moderation group. Small enough that bad actors can be fished out. This is part of a larger group where these individual silos get their own trust relationship. If a small group starts misbehaving, the individuals in that group get reassigned. If the individuals who have moved also correlate with a lack of trust in their next group, they get flagged and put on probation. We can’t have global trust until local trust has been established.
- defrost 3y agoAs a former moderator of some relatively large channels | forums semi frequently subjected to both edge lord raids and the glacial perverse machinations of the slow troll I have one question: To find the most trusted peers, we use Appleseed, a peer-reviewed algorithm and trust metric which was proposed in the mid 2000s by Cai-Nicolas Ziegler and Georg Lausen from the University of Freiburg. Appleseed operates on precisely the kind of weighted graph structure we have described, and it is also guaranteed to converge after a variable (but finite) number of iterations. Appleseed, once converged, produces a ranking of the most trusted peers. Appleseed produces its rankings by effectively releasing a predefined amount of energy at the trust source, or starting node, and letting that energy flow through the trust relations. The energy pools up in the nodes of the graph, with more energy remaining with the more trusted peers. After the computation has converged, each peer has captured a portion of the initial energy, where the peer with the most energy is regarded as the most trusted. Now that this mechanism is out in the open, how robust is it in the face of a determined attempt to deliberate game the system? ie: Can I become the most trusted for that one glorious moment of ripping the table cloth out from under everybody?
- camgunz 3y agoYeah systems like this essentially just centralize power. For a failure case, see spam IP address blacklists. I think generally it's not a good model.
- cheschire 3y agoLet’s raise the stakes even further than discussion forums. Search for “eve online betrayal” or “eve online heist” and determine if this algorithm could’ve prevented those situations. Then raise the stakes more and consider the US presidential candidate nominations process. Push it to extreme situations, and finally in the end compare it to the Chinese social scoring system. This kind of journey of morals is what ultimately led to me transferring ownership of my own online community and walking away for good. It’s tough and I don’t envy the folks who keep up with these things.
- _heimdall 3y ago
- nonrandomstring 3y agoTransitivity of trust is controversial, so I am sceptical of systems that aim to unburden most participants from having to manage individual trust relations and emerging a set of leader/deciders and a larger passive group of followers. Not sure how Appleseed or the proposed TrustNet overlay solves this. Also I don't think there's much hope for one-size-fits-all solutions to trust tracking. Some applications are slow and iterative, like the evolution of reputation in communities, and they must allow for redemption. Others are critical, where even a single, brief defection would be a disaster. I guess this one is aimed at social media chat.
- jll29 3y agoTack, Alexander! I'll add your thesis to my library of papers & books on online trust.
- verisimi 3y agois that list somewhere available to view?
- steelframe 3y agoWhy would you trust a list of papers and books on online trust from some rando online?
- verisimi 3y agoI'm interested in the topic, and its just a collection of some interesting info that this chap has found. Trusting the information has nothing to do with it - I can discern whether the info is valuable for myself.
- mnd999 3y agoI always thought slashdot’s community moderation and meta-moderation was excellent. I always thought it curious that nobody copied it. Of course, dang based moderation also works well but you need a dang for that.
- layer8 3y agoLet’s hope human cloning will arrive within dang‘s lifetime.
- capableweb 3y agoMeh, fine-tune a LLM on dang's comments and call it a day. Ship early and ship often, adjustments can be made as we discover it doing the wrong thing.
- pavel_lishin 3y agoFunny, but taking it seriously: it should be trained on dang's moderation actions, not comments.
- capableweb 3y agoMost of dang's comments (that I've came across at least) are moderation actions, like telling people how they're not following the guidelines and so on. But yeah, also the actual backend moderation actions should obviously be included in the training set.
- andrepd 3y agoA sort of Dang Acevedo? :)
- MichaelZuo 3y agoMetafilter also seems to work fine. Just charge money for each account creation, eventually the repeat trolls will get tired of paying over and over again.
- 3y ago
- jgalt212 3y ago[flagged]
- beebeepka 3y ago"Trickle down moderation"
- codingclaws 3y agoSlightly off topic: I am trying to innovate on moderation systems and I run/code a whitelist moderated forum [0]. You can only see posts and comments from users that you follow. It's a very simple system and there really aren't any gaming vectors. One implication is that if a new user signs up and posts, no one will see it unless they follow. I've actually never used any typical censorship moderation. [0] https://www.commentcastles.org https://www.commentcastles.org
- brlewis 3y agoI just signed up. The front page seems to show lots of posts though I haven't followed anyone yet. Do most users avoid the front page? (I think your comment is on topic for a post about a moderation system.)
- naasking 3y ago> You can only see posts and comments from users that you follow. I don't get it. How do you even find users to follow if you can't see their posts or comments?
- photochemsyn 3y agoA central problem in all online communities that this post doesn't address is the definition of malicious behavior. E.g. a forum run by the marketing division of Coca-Cola might define comments on the negative health effects of soda consumption or just how great Pepsi is as malicious behavior. Explicit definitions of malicious behavior in the forum guidelines may or may not be enforced if the forum is controlled by interests seeking to covertly amplify certain narratives while suppressing others, even if those narratives do not explicitly conflict with site guidelines. One plausible approach to this situation is to use a LLM agent as the forum moderator - one which only uses a publicly-available explicit set of moderation rules to flag comments and submissions. Something like this is almost certainly being used at Youtube, X, etc., with the caveat that the rules being used are mostly hidden from the public (e.g. X feeds don't seem to have much interest in amplifying stories about UAW's efforts to unionize Tesla, etc.). This could lead to a regulatory approach to social media in which the moderation rules being fed to the LLM must be made publicly available.
- Pixie_Dust 3y ago“How do you remove malicious participants from a chat?” You can't. Inevitably, the forum is slowly taken over by some self appointed dictator and cohorts and the more saner voiced are driven out.
- remram 3y agoIs there any way to express distrust? This seems like level 0 of moderation, way to "report" bad behavior. It seems here you can only "trust" someone into being a moderator, and then they have to do this part.
- kmeisthax 3y agoIt's mentioned in the post link, but I suspect in practice distrust is less useful than you'd think as long as fresh identities are free. You can't punish someone without any "skin in the game". Centralized systems have an advantage here: they can refuse to issue new accounts or make it cost money to register, which puts some cost on spam. Distributed systems can be spammed and sock-puppeted for free. In practice most central systems don't explicitly charge money for accounts, but instead require verification of something that would make it inconvenient to register large numbers of accounts all at once. For example, if you want a Gmail account, you need to verify your phone number with SMS. Phone numbers cost money to obtain, which means that you can distrust ones used to create spam accounts and the spammers actually lose something. This is also why Fediverse moderation puts so much emphasis on defederating instances rather than banning individual accounts. In the Identica/OStatus era of the Fediverse, defederation was actually very controversial! But here in the Mastodon era, the only way to actually punish bad instance operators (and there are plenty of them) is to defederate their instance. This works because instances are referred to by domain name, and DNS is a centralized[0] system that costs money to register, so you can distrust a domain and actually cost the abuser money. [0] The distribution of domain records is decentralized, and you can delegate subdomains forever, but you have to have a chain of delegation leading back to the root servers. Top level delegations cost lots of money, second-level delegations less so, and subdomain delegations are basically not worth anything and can be distrusted with wildcards on the first private zone in the domain (e.g. ban .evil.co.uk, .evil.net, etc).
- beefman 3y agoAppleseed sounds a lot like PageRank. Is it? The link for it returns 404. It looks like this[1] is the original paper. It does cite the PageRank paper... [1] https://link.springer.com/article/10.1007/s10796-005-4807-3 https://link.springer.com/article/10.1007/s10796-005-4807-3
- philipwhiuk 3y agoI'm not sure that the general population would understand three different similar actions of trust, when the difference in effect to them personally is 0.
- Animats 3y agoThe usual answer: The “Why your anti-spam idea won’t work” checklist. [1] If you can create identities at zero or modest cost, no majority-vote scheme will work. Amusingly, what does work is Second Life. Space keeps everything from being in the same place. You can shout at most 100 meters, and the 3D world is the size of Los Angeles. There's no broadcast system built in. Jerks are a local problem. Local landowners can kick people off their land. Spam consists of buying small land parcels and putting up billboards, and is rarely profitable. Influencers have small circles of influence. Everything is local. If it's hard for one person to reach large numbers of people at low cost, moderation becomes far less of a problem. This is alien to the concept of social networks of course. It does raise the question, do you need to give everybody a bullhorn? [1] https://trog.qgl.org/20081217/the-why-your-anti-spam-idea-wont-work-checklist/ https://trog.qgl.org/20081217/the-why-your-anti-spam-idea-wo...