6 ms·
I've never been involved with implementing large-scale moderation or content controls, but it seems pretty standard that underlying automated rules aren't gener
by ianbicking 3y ago
I've never been involved with implementing large-scale moderation or content controls, but it seems pretty standard that underlying automated rules aren't generally public, and I've always assumed this is because there's a kind of necessary "security through obscurity" aspect to them. E.g., publish a word blocklist and people can easily find how to express problematic things using words that aren't on the list. Things like shadowbans exist for the same purpose; if you make it clear where the limits are then people will quickly get around them.
I know this is frustrating, we just literally don't seem to have better approaches at this time. But if someone can point to open approaches that work at scale, that would be a great start...
- verisimi 3y ago[flagged]
- CrazyStat 3y agoThis feels more like a personal attack than a response to the argument made.
- SturgeonsLaw 3y agoIt's does, but as someone who is staunchly anti-censorship, I understand the frustration. There are sharks out there who want to control speech for their own ends - governments seeking to control populations, corporations wanting docile consumers, hostile nations wishing to stir dissent, individuals trying to cover up their misdeeds, and enabling censorship helps those hostile parties achieve their ends. In this worldview, regular people who say a variation of "censorship is good, actually" are perhaps seen as useful idiots. A better approach would be building up the critical thinking skills of the population so they can better process information, however that transfers a measure of power to the people and is a multigenerational investment, and removes a justification for censorship, which is politically unappealing.
- nonrandomstring 3y ago> better approach would be building up the critical thinking skills of the population The term for that is "Intellectual self-defence" > transfers a measure of power to the people and is a multigenerational investment The challenge to that literacy comes not from power riding on censorship, but from the people themselves who are now conditioned into an apathetic need for "convenience". Critical thinking is hard work, and is never rewarded except in the long run. Also, anti-censorship absolutism must be tempered with what we call "information hazards". Some of these are genuine, although admittedly very rare, such as easy instructions to create nuclear weapons or synthesise a deadly virus. There are just too many idiots in the world not to want to put a brake on that stuff.
- serial_dev 3y agoThere is no need to implement large scale censorship and moderation in this case. Where is the security concern? That I can generate images of white people in various situations for my five minutes of entertainment? The whole premise of your argument doesn't make sense. I'm talking to a computer, nobody gets hurt. It's like censoring what I write in my notes app vs. what I write on someone's Facebook wall. In one case, I expect no moderation, whereas in the other case, I get that there needs to be some checks.
- Ecoste 3y agoBut what if little timmy asks it how to make a bomb? What if it's racist? Then what?
- withinboredom 3y agoNone of this should be a mystery. Making a bomb is literally something you can figure out with very little research (my friends and I used to blow up cow pastures for fun!). Racism is a totally different and sadder issue. I don’t have a good answer for that one, but knowledge shouldn’t be withheld because someone thinks it is “dangerous”
- lobocinza 3y ago> but knowledge shouldn’t be withheld because someone thinks it is “dangerous” Those that think that are the truly dangerous. They think they know better and want to remove agency from people.
- pizzafeelsright 3y agoIt's fine. We have laws against blowing people up. Racism is fine as well. I won't date out of my race and if you think there should be a law that I must that's not really freedom. As for hiring or not based upon race there's already a law against that. The cure is often worse than navigating uneasy waters. Every time you pass a law you give a gun to a bureaucrat.
- EricE 3y agoYeah, there's absolutely no need for transparency /s https://youtu.be/THZM4D1Lndg?si=0QQuLlH7JebSa6w3&t=485 https://youtu.be/THZM4D1Lndg?si=0QQuLlH7JebSa6w3&t=485 If it doesn't start 8 minutes in, go to the 8 minute mark. Then again I can see why some wouldn't want transparency.
- nonrandomstring 3y ago> publish a word blocklist and people can easily find how to express problematic things using words that aren't on the list. I'd love to explore that further. It's not the words that are "problematic" but the ideas, however expressed? Seems like a "problematic" idea, no ?
- UberFly 3y agoA word blocklist just serves to apply guardrails. It just slows down common abuse. Very far from perfect, but the alternatives are anything goes or total lockdown. Perfect solutions are pretty damn rare.
- nonrandomstring 3y agoYou mention "solutions" Perhaps you mistake me for someone who cares about suggesting solutions for the prolems suffered by giant technopolies, as if they were my problems.
- lp0_on_fire 3y agoThis is basically my view. These tech companies have produced some pretty amazing stuff. Solved problems and built things at scale that were unthinkable a decade ago. They can solve this problem. They choose not to because a) they're already shielded from legal liability for certain things that happen/are said on their platforms and b) it doesn't make them any money.
- wruza 3y agoYes, but the implied problems may not need be approached at all. It's a uniform ideology push, with which people agree differently at different levels. If companies don't want to reveal the full set of measures, they could at least summarize them. I believe even these summaries would be what subj tweet refers to as "ashamed". We cannot discuss or be aware of the problems-and-approaches, unless they are explicitly stated. Your analogy with content moderation is a little off, because it's not a set of measures that is hidden, but the "forum rules" themselves. One thing is AI refusing with an explanation. That makes it partially useless, but it's their right to do so. Another thing if it silently avoids or directs topics due to these restrictions. Pretty sure authors are unable to clearly separate the two cases, and also maintain the same quality as the raw model. At the end of the day people will eventually give up and use Chinese AI instead, cause who cares if it refuses to draw CCP people while doing everything else better.
- verticalscaler 3y agoMy dear fellow, some believe the ends justify the means and play games. Read history, have some decency. The danger of being captured by such people far outweighs any other "problematic things". First and foremost any system must defend against that. You love guardrails so much - put them on the self annointed guard railers. Otherwise, if You Want a Picture of the Future, Imagine a Boot Stamping on a Human Face – for Ever.
- cryptonector 3y agoAnd to add insult to injury the jackbooted thug is an AI bot.
- advael 3y agoThis is simply a bad approach and a bad argument. Security through obscurity is a term whose only usage in security circles is derogatory. People figure out how to get around these auto-censors just fine, and not publishing them creates more problems for legitimate users and more plausible deniability for bad policy hidden in them. Doing the same thing but with public policy would already be better, albeit still bad. The only real solution to the problem of there being an enormous public square controlled by private corporations is to end this situation
- kansface 3y agoIs a content moderation policy the same thing as "security"? Do we get to apply the best practices of the one to the other because they overlap to a smaller or larger degree?
- cryptonector 3y agoYes, since content moderation is a form of authorization policy.
- advael 3y agoThe use of "security through obscurity" invited the comparison. It is a better comparison when using automated tools instead of human decision-makers. That said, even if we are talking about policy rather than security, policies that are unknown by and hidden from the people they bind is probably the most recognizable and one of the more onerous features of despotism. We have the term "Kafkaesque" because a whole famous writer literally spent his career pointing out how they don't work and harm the people they affect
- lamontcg 3y agoI mean people can get through your front door using any number of attacks that either exploit weaknesses in the locking mechanism, or circumvent your locks via a carefully placed brick through a window or something like that. Even if you have alarms and other measures, that doesn't stop someone from doing a quick smash and grab, etc. All of these security flaws doesn't mean that you should leave your door open and unlocked all the time. Imperfect security isn't useless security. And the purpose of things like bad-word filters is to make a best effort at blocking stuff which violates the platform TOC and makes plausible deniability much less likely when someone is deliberately circumventing the filters. The existence of false positives and false negatives is considered acceptable in an imperfect world. The filters themselves also only block the action to change a username or whatever and don't punish the user or deny use of the platform entirely (they're much less punitive than the AI abuse algorithms that auto-ban people off of Google/GitHub/etc).
- cryptonector 3y agoRepeat after me: security by obscurity is weak. Clearly people can work out what some of the rules are, so why not just publish them. If you need to alter them when people figure out how to get around them, well, you already had to anyways.
- UncleMeat 3y agoKerckoff's principle only applies to crypto. "Security by obscurity" is used in oodles of systems security applications and broader contexts involving human behavior. Very very ordinary security best practices rely on obscurity. ALSR is a good example. It is defeated by a data exfiltration vulnerability but remains a useful thing to add to your binaries. Because outside of the crypto space security is an onion and layers add additional cost to attackers.
- cryptonector 3y agoASLR is not security by obscurity. The addresses into which all the various things are mapped are secret in the same sort of way as cryptographic secret and private keys are secret, but the mechanism is not secret.
- UncleMeat 3y agoSure it is. The layout randomization is not at all like a cryptographic secret because it can be extracted from the contents of the binary. Kerckoff's principle means that adversary gets access to literally everything except the private key. If you've got access to the contents of the binary then you can get around aslr.
- cryptonector 3y agoWhat? No, the actual addresses as-loaded at run-time are randomized, that's the point of ASLR, and those addresses are secret-like because we don't want an attacker to be able to craft an exploit that works against a running instance of the victim code -- we want whatever exploit they have that needs loaded addresses to fail because it doesn't know those address. ASLR itself is not secret, but the run-time addresses are. In order to be able to randomize addresses of loaded libraries at run-time... the shared objects need to be built as position independent code, so you can't extract the actual addresses "from the contents of the binary". I suspect you're referring to the fact that in ELF the executable itself is not subject to ASLR unless it's built as a PIE.
- u32480932048 3y agoMost legal systems operate at the nation-state scale and aren't made of hidden mystery laws. There are lots of reasons for that. We've already had this argument with cryptocurrency, where we've basically decided that the existing legal system (although external) provides a sufficient toolset to go after bad actors. Finally, based on the illiberal nature of most AI Safety Sycophants' internet writings, I don't like who they are as people and I don't trust them to implement this.
- observationist 3y agoIf you can't afford to pay a sufficient number of people to moderate a group, you need to reduce the size of the group or increase the number of moderators. Your speculation implies no responsibility for taking on more than can be handled responsibly, and externalizes the consequences to society at large. There are responsible ways to have very clear, bright, easily understood, well communicated rules and sufficient staff to manage a community. I don't know why it's simply accepted that giant social networks get to play these games when it's calculated, cold economics driving the bad decisions. They make enough money to afford responsible moderation. They just don't have to spend that money, and they beg off responsibility for user misbehavior and automated abuses, wring their hands, and claim "we do the best we can!" If they honestly can't use their billions of adtech revenue to responsibly moderate communities, then maybe they shouldn't exist. Maybe we need to legislate something to the effect of "get as big as you want, as long as you can do it responsibly, and here are the guidelines for responsible community management..." Absent such legislation, there's no possible change until AI is able to reasonably do the moderation work of a human. Which may be sooner than any efforts at legislation, at this rate.
- fallingknife 3y agoWhat guideline for community management would possibly not be a flagrant 1A violation?
- opportune 3y agoI think this is a fair approach when things work well enough that a typical user doesn’t need to worry about whether they’ll trigger some kind of special content/moderation logic. If you shadowban spammers and real users almost never get flagged as spammers, the benefits of being tight-lipped outweigh those of the very few users who get improperly flagged or are just curious. With some of these models the guardrails are so clumsy and forced that I think almost any typical user will notice them. Because they include outright work-refusal it’s a very frustrating UX to have to “discover” the policy for yourself through trial and error. And because they’re more about brand management than preventing fraud/bad UX for other users, the failure modes are “someone deliberately engineered a way to get objectionable content generated in spite of our policies.” Obviously some kinds of content are objectionable enough for this to be worth it still, but those are mostly in the porn area - if somebody figures out a way to generate an image that’s just not PC, despite all the safety features, shouldn’t that be on them rather than the provider? Even tuning the model for political correctness is not the end of the world in my opinion, a lot of LLMs do a perfectly reasonable job for my regular use cases. With image generators they are going so far as to obviously (there’s no other way that makes sense) insert diversity sub prompts for some fraction of images which is simply confusing and amateur. Everybody who uses these products just a little bit will notice it. It’s also so cautious that even mild stuff (I tried to do the “now make it even more X” with “American” and it stopped at one iteration) gets caught in the filters. You’re going to find out the policies anyway because they’re so broad an likely to be encountered while using the product innocently - anything a real non-malicious user is likely to get blocked by should be documented.