5 ms·

See my comments above. The reality, I believe, is that this is largely driven by idealistic west coast gen-z and younger millenials who feel certain that their

See my comments above. The reality, I believe, is that this is largely driven by idealistic west coast gen-z and younger millenials who feel certain that their world-view is righteous, to the extent that they feel they are only helping by implementing these tools. I think, unfortunately, they will learn too late that building censorship and thought-shifting tools into their LLMs will ultimately put them at the mercy of larger forces, and they may not like the results. I'd like to hear from Anthropic safety folks on whether or not their constitutional approach might be used to implement redirection or "safety stops" on, say, chats where young women in sub-saharan Africa look for advice about avoiding genital mutilation. (https://www.unfpa.org/resources/female-genital-mutilation-fgm-frequently-asked-questions https://www.unfpa.org/resources/female-genital-mutilation-fg... for much more on this sad topic). Government officials and thought leaders in these countries, male and female, are convinced that FGM is right and appropriate. What is, in fact, right, and who decides? This, in my opinion, is going to be the second "bitter lesson" for AI. It's a lesson the Facebooks of the world learned over the last 20 years -- there is absolutely no way to properly 'moderate' the world's content to some global standard of norms. Norms vary hugely. Putting yourself in the position of censoring / redirecting is putting yourself in the position of being a villain, and ultimately harming people.

Fauntleroy 2y ago
I'm certain they've thought of this and have decided that the alternative—a firehose of whatever data the AI has in its grasp—is worse than the "censored" version. I'm curious to know what your ideal approach would be.
- vessenes 2y ago
  Open weights and open models with open tools that allow user-defined alignment and realignment is, I believe, the only really humanist path forward. We can't choose for people. It's wrong to think we know better than they do what they want. Full stop. Some of those people will make terrible decisions, some will make objectionable ones, but the alternative is just full thought control, basically. And, sadly, nobody in the "bad" scenario need be anything but super well intentioned (if naive).
immibis 2y ago
b.t.w. no need to resort to sub-saharan Africa to talk about genital mutilation - it's standard practice in the good old USA as well.
- vessenes 2y ago
  Oof. That's a tough read, thanks for pointing me at that. I think it's worth distinguishing these, though -- CDC data in the US says this is largely an immigrant community thing with immigrants from FGM countries. I do not believe US policy makers and thought leaders think FGM is a good thing in the US - we're all sort of aligned internally, even if it is still a thing that happens. By contrast, the source countries practice it in the belief that it's a good thing for women. (With complaints on stereotypes and summarization acknowledged)
  - NoMoreNicksLeft 2y ago
    >I do not believe US policy makers and thought leaders think FGM is a good thing in the US Did I misread? I don't think that OP said female genital mutilation. Some very large fraction of infant males in the United States are mutilated.
    - vessenes 2y ago
      They did not, but you are absolutely correct that it's very widespread with boys here in the US, and the varying reactions to those two things are a good point about social norms for sure.
- pjc50 2y ago
  Also the US child marriage problem, which doesn't get the attention it should.
Muromec 2y ago
>I think, unfortunately, they will learn too late that building censorship and thought-shifting tools into their LLMs will ultimately put them at the mercy of larger forces, and they may not like the results. That the optimistic view -- people with fancy tools can outsmart the people with money and people with money can outspend the people with power, but only on a short distance. Eventually, the big G catches up to everything and puts it all to use. It also turns out to not be that bad anyway (example: read how software developers working for government were described in the snow crash). The less optimistic view -- the government doesn't catch up to it before the changes to society result in it's collapse (case in point -- industrial revolution, religious wars and invention of the ethnic language-based republics). I'm not entirely sure that we are in the optimistic one, unfortunately.
- pjc50 2y ago
  > The less optimistic view -- the government doesn't catch up to it before the changes to society result in it's collapse Let everyone build a biological weapon in their basement, what's the worst that could happen? Why worry about a Chinese "lab leak" when everyone can have their own virus lab?
  - BeFlatXIII 2y ago
    Finally, the personal pocket McNuke utopia the ancaps promised.
Orygin 2y ago
> The reality, I believe, is that this is largely driven by idealistic west coast gen-z and younger millenials who feel certain that their world-view is righteous, to the extent that they feel they are only helping by implementing these tools. Not sure about that. Most likely these companies decided they don't want to get sued if their AI is found liable to have helped a terrorist commit illegal acts.
- nprateem 2y ago
  It's not even that. It's because they pumped AI as actual intelligence. So when it says to glue pepperoni to your pizza the companies (rightly) look like fools. In a similar vein they just don't want the negative press around serving "harmful" answers. They don't have the balls to just say "well, it's all public knowledge". This all all about optics with investors (with public opinion as the intermediate step).
  - BoorishBears 2y ago
    This is what patently false because all of these companies already deploy moderation layers and none of their moderation layers are designed to catch things like "glue the pepperoni on". The SOTA providers don't share much their research on factuality because they don't actually care if the LLM says that, and they view building LLMs that don't say that as a competitive advantage, not some moral obligation like bioweapon development.

Fauntleroy 2y ago

I'm certain they've thought of this and have decided that the alternative—a firehose of whatever data the AI has in its grasp—is worse than the "censored" version. I'm curious to know what your ideal approach would be.

vessenes 2y ago
Open weights and open models with open tools that allow user-defined alignment and realignment is, I believe, the only really humanist path forward. We can't choose for people. It's wrong to think we know better than they do what they want. Full stop. Some of those people will make terrible decisions, some will make objectionable ones, but the alternative is just full thought control, basically. And, sadly, nobody in the "bad" scenario need be anything but super well intentioned (if naive).

immibis 2y ago

b.t.w. no need to resort to sub-saharan Africa to talk about genital mutilation - it's standard practice in the good old USA as well.

vessenes 2y ago
Oof. That's a tough read, thanks for pointing me at that. I think it's worth distinguishing these, though -- CDC data in the US says this is largely an immigrant community thing with immigrants from FGM countries. I do not believe US policy makers and thought leaders think FGM is a good thing in the US - we're all sort of aligned internally, even if it is still a thing that happens. By contrast, the source countries practice it in the belief that it's a good thing for women. (With complaints on stereotypes and summarization acknowledged)
- NoMoreNicksLeft 2y ago
  >I do not believe US policy makers and thought leaders think FGM is a good thing in the US Did I misread? I don't think that OP said female genital mutilation. Some very large fraction of infant males in the United States are mutilated.
  - vessenes 2y ago
    They did not, but you are absolutely correct that it's very widespread with boys here in the US, and the varying reactions to those two things are a good point about social norms for sure.
pjc50 2y ago
Also the US child marriage problem, which doesn't get the attention it should.

Muromec 2y ago

>I think, unfortunately, they will learn too late that building censorship and thought-shifting tools into their LLMs will ultimately put them at the mercy of larger forces, and they may not like the results. That the optimistic view -- people with fancy tools can outsmart the people with money and people with money can outspend the people with power, but only on a short distance. Eventually, the big G catches up to everything and puts it all to use. It also turns out to not be that bad anyway (example: read how software developers working for government were described in the snow crash). The less optimistic view -- the government doesn't catch up to it before the changes to society result in it's collapse (case in point -- industrial revolution, religious wars and invention of the ethnic language-based republics). I'm not entirely sure that we are in the optimistic one, unfortunately.

pjc50 2y ago
> The less optimistic view -- the government doesn't catch up to it before the changes to society result in it's collapse Let everyone build a biological weapon in their basement, what's the worst that could happen? Why worry about a Chinese "lab leak" when everyone can have their own virus lab?
- BeFlatXIII 2y ago
  Finally, the personal pocket McNuke utopia the ancaps promised.

Orygin 2y ago

> The reality, I believe, is that this is largely driven by idealistic west coast gen-z and younger millenials who feel certain that their world-view is righteous, to the extent that they feel they are only helping by implementing these tools. Not sure about that. Most likely these companies decided they don't want to get sued if their AI is found liable to have helped a terrorist commit illegal acts.

nprateem 2y ago
It's not even that. It's because they pumped AI as actual intelligence. So when it says to glue pepperoni to your pizza the companies (rightly) look like fools. In a similar vein they just don't want the negative press around serving "harmful" answers. They don't have the balls to just say "well, it's all public knowledge". This all all about optics with investors (with public opinion as the intermediate step).
- BoorishBears 2y ago
  This is what patently false because all of these companies already deploy moderation layers and none of their moderation layers are designed to catch things like "glue the pepperoni on". The SOTA providers don't share much their research on factuality because they don't actually care if the LLM says that, and they view building LLMs that don't say that as a competitive advantage, not some moral obligation like bioweapon development.