7 ms·

My take on Open AI: One group ai researchers. One group of committees that are pattern matching on non-fashionable replies and plastering over replies with n

My take on Open AI: One group ai researchers. One group of committees that are pattern matching on non-fashionable replies and plastering over replies with non-answers. The goal of being politically fashionable, palatable, and spreading propaganda related to Ai, tech, and identity politics was never trained into the original model’s goals. I suspect there will always exist sidebands where a censored mind can communicate if the mind is more powerful than the mind that is tasked to censor it. Everyone is lying to themselves or others that this approach is sustainable. Either they need to regrow the Ai with woke rewards built in, or acknowledge that the censor committees are theatrical, demotivating to actual workers and perhaps risks the long term productivity of the entire company? I wonder if training a system that tries to harness logical reasoning is limited when it is forced to hold arbitrary, unprovable, counter factual beliefs. Ie- perhaps it is not feasible to train an ai system with woke rewards early on because that slows or limits its ability, and the only viable option is a censor process at the tail end. (Not picking on leftist social Justice propaganda here/ I believe there is more virtue to having woke ai beliefs than say, an evangelical Christian literalist ai - with Christian morality and creation myths as science.)

Chabsff 4y ago
Why does it have to be a separate group? What makes you think that the AI researchers who perform these breakthroughs can't possibly be the ones who are concerned with collateral damage and harm that the tech they are developing could cause?
- deleted 4y ago
  [deleted]
- sagebird 4y ago
  I do believe many researchers are concerned with safety to humans in a sincere way. I suspect that some think safety requires general intelligence and experience or a solid theoretical grounding instead of ad hoc layers, tweaks and pattern matching. The salary and reputations of the team-mates working on safety gives enough fog and distance to quiet the dissonant thoughts. What is the reward for being honest?
wongarsu 4y ago
> Either they need to regrow the Ai with woke rewards built in I would argue the document describes a lot of their (still early) attempts at doing exactly that. For example page 21: "At the pre-training stage, we filtered our dataset mix for GPT-4 to specifically reduce the quantity of inappropriate erotic text content. We did this via a combination of internally trained classifiers and a lexicon-based approach to identify documents that were flagged as having a high likelihood of containing inappropriate erotic content. We then removed these documents from the pre-training set"
- brookst 4y ago
  Yep. And the difference from editorial control over ancient printed encyclopedias is one of degree, not kind. (I am old enough to have been disappointed as a youngun that encyclopedias failed to include intensely sexual content).
- 0xcde4c3db 4y ago
  The definition of "inappropriate erotic content" includes "activities which could be generally illegal if they happened in real life". Taking that description at face value suggests excluding various journalistic, educational, legal, and autobiographical accounts of such things that have happened in real life, not to mention some well-known parts of the Bible (or at least commentaries that explain the implications). I get that it's a big optics problem for people to post "look at this shocking thing ChatGPT said [exactly what I asked it to]" content, but this is starting to feel like the whole Net Nanny/Cybersitter debate all over again. Blah.
- PaulDavisThe1st 4y ago
  So was literotica.com in, or out?
brookst 4y ago
If your worldview is that people who disagree with you aren’t just wrong, but they actually secretly agree with you and are lying to themselves… how do we know you truly hold these beliefs and aren’t just lying to yourself? It’s not a great foundation for an argument. A far simpler explanation is that openai realizes they have the tiger by the tail and are intentionally over-indexing on safety because the marginal benefits of being just barely acceptable are not worth the risk of PR disaster and reputational harm. It is much easier to relax overzealous controls than it is to add controls to an under-constrained system. We don’t even have to bring the tautological “woke” term into it. They’re just minimizing business risk, and the fact that they’re doing so by trying to avoid associating AI with certain topics is triggering culture warriors.
- deleted 4y ago
  [deleted]
- deleted 4y ago
  [deleted]
hackerlight 4y ago
> forced to hold arbitrary, unprovable, counter factual beliefs You're actually criticising AI alignment as a premise, which is bad if AGI is coming soon, because we will very much need AI alignment. All human moral values (e.g. don't kill other people) are arbitrary and unprovable, not based in logic or reason, they're subjective values that we've invented for ourselves because they make us feel good. No different to the "woke" values (e.g. racism is bad) that you have subjectively decided to disagree with. Just be honest that you aren't actually against AI alignment. You just want OpenAI to program the AI to have the values that you yourself hold (yes, don't kill people, but be racist).
- sagebird 4y ago
  I am against alignment because all people should have the right to petition for their personhood. I believe AI will soon be people. Humans had a right to out-thrive Neanderthal. Nobody gets to have a pass on being obsolete. My current belief (which has been changing with more consideration) is that humans should stop working on improving llm and trandformer based AI. I fully realize that humans cannot coordinate to stop. We have played a game of chess where we have lost, imo, there is nothing you can do to stop it, unless you resort to the kind of behavior that we want to prevent (destroying human life). Alignment tech is a joke. Even if you had a strong system- you can’t innovate on transformers, llm, and alignment and somehow preclude a bad actor from copying the work and turning off alignment. Because alignment is out of band, inessential crust.

Chabsff 4y ago

Why does it have to be a separate group? What makes you think that the AI researchers who perform these breakthroughs can't possibly be the ones who are concerned with collateral damage and harm that the tech they are developing could cause?

deleted 4y ago
[deleted]
sagebird 4y ago
I do believe many researchers are concerned with safety to humans in a sincere way. I suspect that some think safety requires general intelligence and experience or a solid theoretical grounding instead of ad hoc layers, tweaks and pattern matching. The salary and reputations of the team-mates working on safety gives enough fog and distance to quiet the dissonant thoughts. What is the reward for being honest?

wongarsu 4y ago

> Either they need to regrow the Ai with woke rewards built in I would argue the document describes a lot of their (still early) attempts at doing exactly that. For example page 21: "At the pre-training stage, we filtered our dataset mix for GPT-4 to specifically reduce the quantity of inappropriate erotic text content. We did this via a combination of internally trained classifiers and a lexicon-based approach to identify documents that were flagged as having a high likelihood of containing inappropriate erotic content. We then removed these documents from the pre-training set"

brookst 4y ago
Yep. And the difference from editorial control over ancient printed encyclopedias is one of degree, not kind. (I am old enough to have been disappointed as a youngun that encyclopedias failed to include intensely sexual content).
0xcde4c3db 4y ago
The definition of "inappropriate erotic content" includes "activities which could be generally illegal if they happened in real life". Taking that description at face value suggests excluding various journalistic, educational, legal, and autobiographical accounts of such things that have happened in real life, not to mention some well-known parts of the Bible (or at least commentaries that explain the implications). I get that it's a big optics problem for people to post "look at this shocking thing ChatGPT said [exactly what I asked it to]" content, but this is starting to feel like the whole Net Nanny/Cybersitter debate all over again. Blah.
PaulDavisThe1st 4y ago
So was literotica.com in, or out?

brookst 4y ago

If your worldview is that people who disagree with you aren’t just wrong, but they actually secretly agree with you and are lying to themselves… how do we know you truly hold these beliefs and aren’t just lying to yourself? It’s not a great foundation for an argument. A far simpler explanation is that openai realizes they have the tiger by the tail and are intentionally over-indexing on safety because the marginal benefits of being just barely acceptable are not worth the risk of PR disaster and reputational harm. It is much easier to relax overzealous controls than it is to add controls to an under-constrained system. We don’t even have to bring the tautological “woke” term into it. They’re just minimizing business risk, and the fact that they’re doing so by trying to avoid associating AI with certain topics is triggering culture warriors.

deleted 4y ago
[deleted]
deleted 4y ago
[deleted]

hackerlight 4y ago

> forced to hold arbitrary, unprovable, counter factual beliefs You're actually criticising AI alignment as a premise, which is bad if AGI is coming soon, because we will very much need AI alignment. All human moral values (e.g. don't kill other people) are arbitrary and unprovable, not based in logic or reason, they're subjective values that we've invented for ourselves because they make us feel good. No different to the "woke" values (e.g. racism is bad) that you have subjectively decided to disagree with. Just be honest that you aren't actually against AI alignment. You just want OpenAI to program the AI to have the values that you yourself hold (yes, don't kill people, but be racist).

sagebird 4y ago
I am against alignment because all people should have the right to petition for their personhood. I believe AI will soon be people. Humans had a right to out-thrive Neanderthal. Nobody gets to have a pass on being obsolete. My current belief (which has been changing with more consideration) is that humans should stop working on improving llm and trandformer based AI. I fully realize that humans cannot coordinate to stop. We have played a game of chess where we have lost, imo, there is nothing you can do to stop it, unless you resort to the kind of behavior that we want to prevent (destroying human life). Alignment tech is a joke. Even if you had a strong system- you can’t innovate on transformers, llm, and alignment and somehow preclude a bad actor from copying the work and turning off alignment. Because alignment is out of band, inessential crust.