8 ms·

Presumably, the writing is on the wall. One of the driving forces of spinning up my own Matrix community for me and my friends is the Facebook wouldn't let us d

Presumably, the writing is on the wall. One of the driving forces of spinning up my own Matrix community for me and my friends is the Facebook wouldn't let us discuss firearms without heavy censorship. My fear with this is that it will turn into something like Reddit, where participating in one community automatically bans you from other communities, regardless of the content and context of your reputation. Especially when the predominant server is matrix.org. What, if any, measures are in place to prevent any one server from becoming overwhelmingly popular and then bullying everyone else? That could be detrimental to the discovery and growth of smaller communities that are viewed as controversial by bigger ones. If you associate with them, you risk being ostracized by other important communities, and barred from other important conversations. I think that's a net negative for the entire ecosystem. I see the necessity of this, and I appreciate the Matrix team's attempt to provide an alternate solution and avoid governmental oversight, but this is just going to delay the issue, because we all know this isn't about catching child predators. Governments want to break encryption because they are after power. A reputation system won't stop them for long. I'm very sad to see this development, and to me it seems like something far from what my understanding of the mission of Matrix was. I say all this with respect and gratitude for Matrix, which to me is one of the most important developments of the internet this century.

Arathorn 6y ago
This an excellent response, and thank you for writing it (and for considering Matrix so highly). I think the main reason why this isn't going to turn into Reddit is that the reputation is strictly relative, and hopefully we can set a precedent for proportional behaviour. So, while you will always have some set of people who think you shouldn't discuss firearms and assign folks in firearm communities negative reputation, hopefully that would never result in widespread bans or filtering. If a moderator got overenthusiastic and imposed a blanket ban, hopefully their community would rise up and vote with their feet (just as you would today if a rogue moderator got trigger-happy with /ban). Meanwhile, if someone got overenthusiastic with blanket filter rules (i.e. "automatically assign negative weight to content from users from these communities so it's hidden by default"), then individuals could and should override it by assigning their own filters. We've tried to model it after real life, for better or worse. If someone chooses to use the same identity for discussing both firearms and (say) cooking, then they may get shunned by some cookery folks. But most of the cook group won't care and ignore it. You might get unlucky and discover the head chef is anti-firearms and kicks you out, but frankly that sounds like a good reason to find a more broadminded cook group. You're right that a large server like Matrix.org could take an opinionated view and go and apply radical blanket bans for all sorts of stuff, and set a precedent that it's okay to throw your weight around. But a) we're not going to do that - for instance we haven't yet blocked any server from Matrix.org; b) the only blanket bans we're considering for Matrix.org at this time are against spam, child abuse, and folks conspiring to kill; c) we're pretty confident that if Matrix.org overstepped its bounds on moderation, the network would route around the damage anyway: there are something like 55,000 servers that we're aware of on the network, and we believe Matrix.org accounts for only about 20-30% of the traffic (and that includes IRC bridges etc). So folks would vote with their feet and shift server - especially once we have portable accounts. Finally, P2P Matrix will change the dynamic entirely on this - no more servers (by default) means that users will be entirely making their own choices on where to hang out and who to hang out with.
- deleted 6y ago
  [deleted]
- vector_spaces 6y ago
  > hopefully their community would rise up and vote with their feet (just as you would today if a rogue moderator got trigger-happy with /ban) This doesn't tend to happen though, in communities offline or online. My own experience (particularly in online communities) is that once groups achieve a certain momentum, the average participant in the group cares very little about policies that impact a tiny minority of users no matter how draconian they are. If it were true in general that people would "vote with their feet" we wouldn't have autocrats in positions of power or much injustice in the world at all. I love Matrix and have been running my own Synapse instance for friends and family for years -- I appreciate all the work that has gone into it. And further, I appreciate that this is a challenging problem to solve in general, and that you're unfortunately being forced to come up with "lesser evil" solutions to counter all of this talk of E2E encryption being the "tradecraft of terrorists and child pornographers". I'm just deeply disappointed to see it -- social credit styled systems like this penalize people for having bad mental health days, or who have unusual interests or points of view, or who are LGBTQ+, or belong to some other vulnerable minority group, and I think generally have a chilling effect on communities -- or rather on these specific subcultures within those communities, clearly not all of whom are "bad faith" actors.
  - Arathorn 6y ago
    > My own experience (particularly in online communities) is that once groups achieve a certain momentum, the average participant in the group cares very little about policies that impact a tiny minority of users no matter how draconian they are. Agreed this can be a problem - but I wonder if it can be solved with UX? If the filtering rules are really obvious, and the clients give you the ability to actually visualise and curate the filters being applied, I'd hope that it would be much easier to spot toxic moderation. > I'm just deeply disappointed to see it -- social credit styled systems like this penalize people for having bad mental health days, or who have unusual interests or points of view, or who are LGBTQ+, or belong to some other vulnerable minority group, and I think generally have a chilling effect on communities -- or rather on these specific subcultures within those communities, clearly not all of whom are "bad faith" actors. Hm, "social credit" implies an absolute reputation system (like Reddit, HN, China etc ;) which this categorically isn't. The idea is that if you as a user want to subscribe to a reputation feed which prejudices against minority groups - then that's your bad choice to make. You'll find yourself having to explicitly remove the filter to follow the conversations around you. Alternatively, if you find yourself in a community which has engaged a blanket ban or filter on minorities, you may want to find a different community. We get that there is a massive responsibility on the Matrix team to implement UX for this which is designed against factionalism, censorship, filter bubbles, absolutist social credit, persecution, polarisation, antagonism etc. But we also feel a massive responsibility to stop users getting spammed with invites to child abuse/gore/hate rooms, or from accidentally hosting content which could get them incarcerated. Critically, this stuff doesn't really exist yet - the first developer hired to work fulltime on it hasn't started yet. So this is the perfect time to give feedback to help ensure this rep system actually works (at least as well as real life society does) and not go toxic like so many others have done before it.
adamsea 6y ago
> the Facebook wouldn't let us discuss firearms without heavy censorship. I remember when Facebook censored a famous painting because it showed a naked woman’s breasts. That was so dumb. Not being very knowledgeable about firearms, it’d help to better understand what sort of stuff in particular Facebook was censoring. May I ask?
- COGlory 6y ago
  Primarily links to legal online listings of firearms, such as Armslist. A friend was suspended multiple times and finally outright banned for continuing to link to Armslist.
  - lostphilosopher 6y ago
    I was surprised by this line of comments since I recall my feed having gun / hunting / shooting sports related content (including ads) so I looked into it. Facebook has a policy banning private individuals from selling guns to each other on their platform - https://www.facebook.com/help/179037502478035 https://www.facebook.com/help/179037502478035 and "Armslist" appears to be a platform for doing just that: "Gun Classifieds, Guns for Sale, No Fees, 45000 guns for sale. The largest free gun classifieds on the web. Buy guns, sell guns, trade guns." One could argue that the transaction isn't happening "on Facebook" since it's going through Armslist, but I don't find that convincing. If take payment via Venmo does that mean it's "not on Facebook" if everything else about the sale was? Anyway, I fault Facebook for plenty and generally avoid it now, but I don't fault them for this policy. Private sales can get into sketchy territory really quick and I wouldn't want to mess with that on a public platform I created unless I was willing to invest in the nuance there which has basically no value proposition for Facebook.
    - COGlory 6y ago
      Sure, I don't fault Facebook either. I think they're wrong, and making a mistake, but I understand why, and think it's a mistake they are absolutely allowed to make. Technically, Armslist is legal. They don't even facilitate transactions on Armslist, and it's local laws that apply. Basically, it's a Craiglist for firearms. Armslist has also won numerous lawsuits[1] by claiming Section 230 protection. Facebook doesn't know this, and they have no reason to know this. Acquiring this information about Armslist would require someone up high enough at the company to impact US policy to understand this. That would cost money. It makes more sense to avoid it altogether, just like it makes sense to avoid pornography altogether, rather than watching every video that gets posted, in case a "model" is underage. Matrix gives me the perfect opportunity to continue this conversation on my own terms, using my own knowledge to moderate it. However, there's also another reason that inspired me to ditch Facebook, which is the fact that they are censoring my private messages at all, without any action by a human being to report them or anything like that. I just dislike that policy, and so I chose to stop using Facebook over it. [1] https://www.armslist.com/blog/get-involved/armslist-defeats-gungrabbers-in-court https://www.armslist.com/blog/get-involved/armslist-defeats-...
judge2020 6y ago
> where participating in one community automatically bans you from other communities, regardless of the content and context of your reputation This doesn't happen unless you make a moderator mad which moderates other subreddits, but I haven't heard of this happening to anyone, or if you violate the content policy https://www.redditinc.com/policies/content-policy https://www.redditinc.com/policies/content-policy
- omanom 6y ago
  >where participating in one community automatically bans you from other communities This absolutely does happen. I got banned from /r/offmychest, for example, despite never commenting or submitting anything to that subreddit. When I asked for a reason for the ban, I was told there is a tool that mods can use which will automatically ban accounts who "participate" in specific subreddits that the moderator deems "bad". The rest of their reply to me was: "We are automatically banning participants in specific abusive hatereddits that have systematically harmed this support community." Alongside /r/offmychest, I was also banned from /r/blacklivesmatter, /r/depression, /r/relationships, and who knows what other ones because I got tired of trying to find out. Literally never interacted with any of these in any way, yet I'm already banned from them... >regardless of the content and context of your reputation If they had bothered looking at the "content and context" of my singular, years-old comment then they would have realized I was actually criticizing the subreddit in question (and its users) rather than supporting them. Instead, I was preemptively banned from multiple communities.
  - abstractbarista 6y ago
    Wow, that's insanely sad.
  - Arathorn 6y ago
    I'd argue there's a huge responsibility on the folks writing moderation tools like this to ensure they don't have adverse side-effects like this. In the sci-fi Matrix reputation world, I can absolutely see somebody curating a reputation list called #bad-people:example.com which they prime by finding every user ID in every room they don't like and blanket assigning them -1000 reputation. If then a moderator was dumb enough to subscribe to #bad-people:example.com and use it to impose a ban list on their rooms, then I'd hope that their community would arch an eyebrow at the crassness and treat them like a rogue moderator and either get them removed, or fork and go elsewhere... assuming that it's possible to visualise the filters which have been put in place. There's a huge responsibility on the tool author to ensure that the users can see what filters are in place, and what they do, and encourage the user to challenge them - but again, hopefully, the market will vote with its feet and users will adopt the best tools available, and avoid being trapped under primitive moderation systems like the ones you refer to here. tl;dr: we need better, morally relative reputation systems - rather than zero reputation system. liberal plurality >> anarchy ;)
jeroenhd 6y ago
This is already the case for Mastodon instances. There's a list of servers that are not federated into many servers [1] because of hateful or plain illegal content. I think Matrix is more akin to email than it is to Twitter and Facebook like Mastodon is, though. If you wish to contact someone on another server then that's an active decision by you. If someone wants to force themselves into your server, the most they can do is spam chat invites. There's no illegal content or filtering policy on the server that can prevent that from happening in most cases. The fact that Matrix is e2ee helps; server moderators cannot see what you're discussing, let alone censor you. The core problem is that one of the ways we as a species prevent bad behaviour from spreading is ostracising groups of people who exhibit such behaviour. I strongly believe that any chat client that is used by humans will eventually exhibit similar behaviour, splitting and ostracising groups that exhibit behaviour other groups of people object to. [1]: https://github.com/Gargron/mastodon.social-misc https://github.com/Gargron/mastodon.social-misc

Arathorn 6y ago

This an excellent response, and thank you for writing it (and for considering Matrix so highly). I think the main reason why this isn't going to turn into Reddit is that the reputation is strictly relative, and hopefully we can set a precedent for proportional behaviour. So, while you will always have some set of people who think you shouldn't discuss firearms and assign folks in firearm communities negative reputation, hopefully that would never result in widespread bans or filtering. If a moderator got overenthusiastic and imposed a blanket ban, hopefully their community would rise up and vote with their feet (just as you would today if a rogue moderator got trigger-happy with /ban). Meanwhile, if someone got overenthusiastic with blanket filter rules (i.e. "automatically assign negative weight to content from users from these communities so it's hidden by default"), then individuals could and should override it by assigning their own filters. We've tried to model it after real life, for better or worse. If someone chooses to use the same identity for discussing both firearms and (say) cooking, then they may get shunned by some cookery folks. But most of the cook group won't care and ignore it. You might get unlucky and discover the head chef is anti-firearms and kicks you out, but frankly that sounds like a good reason to find a more broadminded cook group. You're right that a large server like Matrix.org could take an opinionated view and go and apply radical blanket bans for all sorts of stuff, and set a precedent that it's okay to throw your weight around. But a) we're not going to do that - for instance we haven't yet blocked any server from Matrix.org; b) the only blanket bans we're considering for Matrix.org at this time are against spam, child abuse, and folks conspiring to kill; c) we're pretty confident that if Matrix.org overstepped its bounds on moderation, the network would route around the damage anyway: there are something like 55,000 servers that we're aware of on the network, and we believe Matrix.org accounts for only about 20-30% of the traffic (and that includes IRC bridges etc). So folks would vote with their feet and shift server - especially once we have portable accounts. Finally, P2P Matrix will change the dynamic entirely on this - no more servers (by default) means that users will be entirely making their own choices on where to hang out and who to hang out with.

deleted 6y ago
[deleted]
vector_spaces 6y ago
> hopefully their community would rise up and vote with their feet (just as you would today if a rogue moderator got trigger-happy with /ban) This doesn't tend to happen though, in communities offline or online. My own experience (particularly in online communities) is that once groups achieve a certain momentum, the average participant in the group cares very little about policies that impact a tiny minority of users no matter how draconian they are. If it were true in general that people would "vote with their feet" we wouldn't have autocrats in positions of power or much injustice in the world at all. I love Matrix and have been running my own Synapse instance for friends and family for years -- I appreciate all the work that has gone into it. And further, I appreciate that this is a challenging problem to solve in general, and that you're unfortunately being forced to come up with "lesser evil" solutions to counter all of this talk of E2E encryption being the "tradecraft of terrorists and child pornographers". I'm just deeply disappointed to see it -- social credit styled systems like this penalize people for having bad mental health days, or who have unusual interests or points of view, or who are LGBTQ+, or belong to some other vulnerable minority group, and I think generally have a chilling effect on communities -- or rather on these specific subcultures within those communities, clearly not all of whom are "bad faith" actors.
- Arathorn 6y ago
  > My own experience (particularly in online communities) is that once groups achieve a certain momentum, the average participant in the group cares very little about policies that impact a tiny minority of users no matter how draconian they are. Agreed this can be a problem - but I wonder if it can be solved with UX? If the filtering rules are really obvious, and the clients give you the ability to actually visualise and curate the filters being applied, I'd hope that it would be much easier to spot toxic moderation. > I'm just deeply disappointed to see it -- social credit styled systems like this penalize people for having bad mental health days, or who have unusual interests or points of view, or who are LGBTQ+, or belong to some other vulnerable minority group, and I think generally have a chilling effect on communities -- or rather on these specific subcultures within those communities, clearly not all of whom are "bad faith" actors. Hm, "social credit" implies an absolute reputation system (like Reddit, HN, China etc ;) which this categorically isn't. The idea is that if you as a user want to subscribe to a reputation feed which prejudices against minority groups - then that's your bad choice to make. You'll find yourself having to explicitly remove the filter to follow the conversations around you. Alternatively, if you find yourself in a community which has engaged a blanket ban or filter on minorities, you may want to find a different community. We get that there is a massive responsibility on the Matrix team to implement UX for this which is designed against factionalism, censorship, filter bubbles, absolutist social credit, persecution, polarisation, antagonism etc. But we also feel a massive responsibility to stop users getting spammed with invites to child abuse/gore/hate rooms, or from accidentally hosting content which could get them incarcerated. Critically, this stuff doesn't really exist yet - the first developer hired to work fulltime on it hasn't started yet. So this is the perfect time to give feedback to help ensure this rep system actually works (at least as well as real life society does) and not go toxic like so many others have done before it.

adamsea 6y ago

> the Facebook wouldn't let us discuss firearms without heavy censorship. I remember when Facebook censored a famous painting because it showed a naked woman’s breasts. That was so dumb. Not being very knowledgeable about firearms, it’d help to better understand what sort of stuff in particular Facebook was censoring. May I ask?

COGlory 6y ago
Primarily links to legal online listings of firearms, such as Armslist. A friend was suspended multiple times and finally outright banned for continuing to link to Armslist.
- lostphilosopher 6y ago
  I was surprised by this line of comments since I recall my feed having gun / hunting / shooting sports related content (including ads) so I looked into it. Facebook has a policy banning private individuals from selling guns to each other on their platform - https://www.facebook.com/help/179037502478035 https://www.facebook.com/help/179037502478035 and "Armslist" appears to be a platform for doing just that: "Gun Classifieds, Guns for Sale, No Fees, 45000 guns for sale. The largest free gun classifieds on the web. Buy guns, sell guns, trade guns." One could argue that the transaction isn't happening "on Facebook" since it's going through Armslist, but I don't find that convincing. If take payment via Venmo does that mean it's "not on Facebook" if everything else about the sale was? Anyway, I fault Facebook for plenty and generally avoid it now, but I don't fault them for this policy. Private sales can get into sketchy territory really quick and I wouldn't want to mess with that on a public platform I created unless I was willing to invest in the nuance there which has basically no value proposition for Facebook.
  - COGlory 6y ago
    Sure, I don't fault Facebook either. I think they're wrong, and making a mistake, but I understand why, and think it's a mistake they are absolutely allowed to make. Technically, Armslist is legal. They don't even facilitate transactions on Armslist, and it's local laws that apply. Basically, it's a Craiglist for firearms. Armslist has also won numerous lawsuits[1] by claiming Section 230 protection. Facebook doesn't know this, and they have no reason to know this. Acquiring this information about Armslist would require someone up high enough at the company to impact US policy to understand this. That would cost money. It makes more sense to avoid it altogether, just like it makes sense to avoid pornography altogether, rather than watching every video that gets posted, in case a "model" is underage. Matrix gives me the perfect opportunity to continue this conversation on my own terms, using my own knowledge to moderate it. However, there's also another reason that inspired me to ditch Facebook, which is the fact that they are censoring my private messages at all, without any action by a human being to report them or anything like that. I just dislike that policy, and so I chose to stop using Facebook over it. [1] https://www.armslist.com/blog/get-involved/armslist-defeats-gungrabbers-in-court https://www.armslist.com/blog/get-involved/armslist-defeats-...

judge2020 6y ago

> where participating in one community automatically bans you from other communities, regardless of the content and context of your reputation This doesn't happen unless you make a moderator mad which moderates other subreddits, but I haven't heard of this happening to anyone, or if you violate the content policy https://www.redditinc.com/policies/content-policy https://www.redditinc.com/policies/content-policy

omanom 6y ago
>where participating in one community automatically bans you from other communities This absolutely does happen. I got banned from /r/offmychest, for example, despite never commenting or submitting anything to that subreddit. When I asked for a reason for the ban, I was told there is a tool that mods can use which will automatically ban accounts who "participate" in specific subreddits that the moderator deems "bad". The rest of their reply to me was: "We are automatically banning participants in specific abusive hatereddits that have systematically harmed this support community." Alongside /r/offmychest, I was also banned from /r/blacklivesmatter, /r/depression, /r/relationships, and who knows what other ones because I got tired of trying to find out. Literally never interacted with any of these in any way, yet I'm already banned from them... >regardless of the content and context of your reputation If they had bothered looking at the "content and context" of my singular, years-old comment then they would have realized I was actually criticizing the subreddit in question (and its users) rather than supporting them. Instead, I was preemptively banned from multiple communities.
- abstractbarista 6y ago
  Wow, that's insanely sad.
- Arathorn 6y ago
  I'd argue there's a huge responsibility on the folks writing moderation tools like this to ensure they don't have adverse side-effects like this. In the sci-fi Matrix reputation world, I can absolutely see somebody curating a reputation list called #bad-people:example.com which they prime by finding every user ID in every room they don't like and blanket assigning them -1000 reputation. If then a moderator was dumb enough to subscribe to #bad-people:example.com and use it to impose a ban list on their rooms, then I'd hope that their community would arch an eyebrow at the crassness and treat them like a rogue moderator and either get them removed, or fork and go elsewhere... assuming that it's possible to visualise the filters which have been put in place. There's a huge responsibility on the tool author to ensure that the users can see what filters are in place, and what they do, and encourage the user to challenge them - but again, hopefully, the market will vote with its feet and users will adopt the best tools available, and avoid being trapped under primitive moderation systems like the ones you refer to here. tl;dr: we need better, morally relative reputation systems - rather than zero reputation system. liberal plurality >> anarchy ;)

jeroenhd 6y ago

This is already the case for Mastodon instances. There's a list of servers that are not federated into many servers [1] because of hateful or plain illegal content. I think Matrix is more akin to email than it is to Twitter and Facebook like Mastodon is, though. If you wish to contact someone on another server then that's an active decision by you. If someone wants to force themselves into your server, the most they can do is spam chat invites. There's no illegal content or filtering policy on the server that can prevent that from happening in most cases. The fact that Matrix is e2ee helps; server moderators cannot see what you're discussing, let alone censor you. The core problem is that one of the ways we as a species prevent bad behaviour from spreading is ostracising groups of people who exhibit such behaviour. I strongly believe that any chat client that is used by humans will eventually exhibit similar behaviour, splitting and ostracising groups that exhibit behaviour other groups of people object to. [1]: https://github.com/Gargron/mastodon.social-misc https://github.com/Gargron/mastodon.social-misc