4 ms·

This is simply a bad approach and a bad argument. Security through obscurity is a term whose only usage in security circles is derogatory. People figure out how

by advael 3y ago

This is simply a bad approach and a bad argument. Security through obscurity is a term whose only usage in security circles is derogatory. People figure out how to get around these auto-censors just fine, and not publishing them creates more problems for legitimate users and more plausible deniability for bad policy hidden in them. Doing the same thing but with public policy would already be better, albeit still bad. The only real solution to the problem of there being an enormous public square controlled by private corporations is to end this situation

kansface 3y ago
Is a content moderation policy the same thing as "security"? Do we get to apply the best practices of the one to the other because they overlap to a smaller or larger degree?
- cryptonector 3y ago
  Yes, since content moderation is a form of authorization policy.
- advael 3y ago
  The use of "security through obscurity" invited the comparison. It is a better comparison when using automated tools instead of human decision-makers. That said, even if we are talking about policy rather than security, policies that are unknown by and hidden from the people they bind is probably the most recognizable and one of the more onerous features of despotism. We have the term "Kafkaesque" because a whole famous writer literally spent his career pointing out how they don't work and harm the people they affect
lamontcg 3y ago
I mean people can get through your front door using any number of attacks that either exploit weaknesses in the locking mechanism, or circumvent your locks via a carefully placed brick through a window or something like that. Even if you have alarms and other measures, that doesn't stop someone from doing a quick smash and grab, etc. All of these security flaws doesn't mean that you should leave your door open and unlocked all the time. Imperfect security isn't useless security. And the purpose of things like bad-word filters is to make a best effort at blocking stuff which violates the platform TOC and makes plausible deniability much less likely when someone is deliberately circumventing the filters. The existence of false positives and false negatives is considered acceptable in an imperfect world. The filters themselves also only block the action to change a username or whatever and don't punish the user or deny use of the platform entirely (they're much less punitive than the AI abuse algorithms that auto-ban people off of Google/GitHub/etc).
- advael 3y ago
  Your analogy is also bad. I agree that perfect security is impossible and that is completely irrelevant here. What platforms like this do by not publishing their policies is more akin to insisting you use their special locks on your door that they claim protect you better because no one knows how they work. Maybe they're operated by an AI working with a Ring camera or something? Very fancy stuff. With this kind of tech, you may be locked out of your home for reasons you don't understand. An independent locksmith might have a hard time figuring out what's going wrong with the door if it fails. You have no idea if some burglars are authorized to enter your home trivially by a deal with the company. If the company decides you are in the wrong in any context, they have the unilateral power to deny you access with no clear recourse. They may get you arrested for trying to get into your own home
  - lamontcg 3y ago
    > They may get you arrested for trying to get into your own home The AI algorithms that ban people from platforms like Google and GitHub do that, which I explicitly called out as needing more oversight. That is different from algorithms which just prevent you from doing something on a platform like using n-bombs in your username, or the LLM guardrails that just give you mangled answers or tell you that they can't do that. That isn't analogous to getting arrested. And in these cases the analogy really falls apart because it isn't your home, and it isn't critical for your life.
    - advael 3y ago
      Friend, it was your analogy
      - lamontcg 3y ago
        I was intending to make a point about security through obscurity, not to make an analogy with those systems.
        advael 3y ago
chillfox 3y ago
And yet, every single security system on the net relies on an element of obscurity to work. Passwords are secret (obscure), as are private keys for SSL/TLS.
- advael 3y ago
  This misunderstands what is meant by the concept. The mechanisms and the public policies that dictate how they are used are not obscured. To borrow and improve another commenter's analogy about locks and keys, the lock on your door is more secure because every locksmith in the world knows how it works, which doesn't mean they have your key
batch12 3y ago
Security through obscurity can have a place as part of a larger defense in depth strategy. Alone it's a joke. Source: in security circles
- advael 3y ago
  Even granting that there may be some nuance to whether and to what degree secrecy is valuable in some contexts, the policies used for automated content moderation on large platforms and the policies by which AI systems are aligned are not good candidates for this secrecy having even a beneficial effect, let alone being necessary

by advael 3y ago

kansface 3y ago

Is a content moderation policy the same thing as "security"? Do we get to apply the best practices of the one to the other because they overlap to a smaller or larger degree?

cryptonector 3y ago
Yes, since content moderation is a form of authorization policy.
advael 3y ago
The use of "security through obscurity" invited the comparison. It is a better comparison when using automated tools instead of human decision-makers. That said, even if we are talking about policy rather than security, policies that are unknown by and hidden from the people they bind is probably the most recognizable and one of the more onerous features of despotism. We have the term "Kafkaesque" because a whole famous writer literally spent his career pointing out how they don't work and harm the people they affect

lamontcg 3y ago

I mean people can get through your front door using any number of attacks that either exploit weaknesses in the locking mechanism, or circumvent your locks via a carefully placed brick through a window or something like that. Even if you have alarms and other measures, that doesn't stop someone from doing a quick smash and grab, etc. All of these security flaws doesn't mean that you should leave your door open and unlocked all the time. Imperfect security isn't useless security. And the purpose of things like bad-word filters is to make a best effort at blocking stuff which violates the platform TOC and makes plausible deniability much less likely when someone is deliberately circumventing the filters. The existence of false positives and false negatives is considered acceptable in an imperfect world. The filters themselves also only block the action to change a username or whatever and don't punish the user or deny use of the platform entirely (they're much less punitive than the AI abuse algorithms that auto-ban people off of Google/GitHub/etc).

advael 3y ago
Your analogy is also bad. I agree that perfect security is impossible and that is completely irrelevant here. What platforms like this do by not publishing their policies is more akin to insisting you use their special locks on your door that they claim protect you better because no one knows how they work. Maybe they're operated by an AI working with a Ring camera or something? Very fancy stuff. With this kind of tech, you may be locked out of your home for reasons you don't understand. An independent locksmith might have a hard time figuring out what's going wrong with the door if it fails. You have no idea if some burglars are authorized to enter your home trivially by a deal with the company. If the company decides you are in the wrong in any context, they have the unilateral power to deny you access with no clear recourse. They may get you arrested for trying to get into your own home
- lamontcg 3y ago
  > They may get you arrested for trying to get into your own home The AI algorithms that ban people from platforms like Google and GitHub do that, which I explicitly called out as needing more oversight. That is different from algorithms which just prevent you from doing something on a platform like using n-bombs in your username, or the LLM guardrails that just give you mangled answers or tell you that they can't do that. That isn't analogous to getting arrested. And in these cases the analogy really falls apart because it isn't your home, and it isn't critical for your life.
  - advael 3y ago
    Friend, it was your analogy
    - lamontcg 3y ago
      I was intending to make a point about security through obscurity, not to make an analogy with those systems.
      - advael 3y ago

chillfox 3y ago

And yet, every single security system on the net relies on an element of obscurity to work. Passwords are secret (obscure), as are private keys for SSL/TLS.

advael 3y ago
This misunderstands what is meant by the concept. The mechanisms and the public policies that dictate how they are used are not obscured. To borrow and improve another commenter's analogy about locks and keys, the lock on your door is more secure because every locksmith in the world knows how it works, which doesn't mean they have your key

batch12 3y ago

Security through obscurity can have a place as part of a larger defense in depth strategy. Alone it's a joke. Source: in security circles

advael 3y ago
Even granting that there may be some nuance to whether and to what degree secrecy is valuable in some contexts, the policies used for automated content moderation on large platforms and the policies by which AI systems are aligned are not good candidates for this secrecy having even a beneficial effect, let alone being necessary