5 ms·

According to the paper, Anthropic's superior safety approach was deliberately delaying release in order to avoid “advanc[ing] the rate of AI capabilities progre

According to the paper, Anthropic's superior safety approach was deliberately delaying release in order to avoid “advanc[ing] the rate of AI capabilities progress." OpenAI is criticized for kicking off a race for AI progress by releasing ChatGPT to the public. [1] https://cset.georgetown.edu/wp-content/uploads/CSET-Decoding-Intentions.pdf https://cset.georgetown.edu/wp-content/uploads/CSET-Decoding...

himaraya 3y ago
The paper reads a lot more nuanced than that. It compares the "system card" released with GPT-4 to the delay of Claude and the merits of each approach vis a vis safety.
- Emma_Goldman 3y ago
  Not really, and your own description is little different from the person you're responding to anyway?
  - deleted 3y ago
    [deleted]
danenania 3y ago
It's important to remember that part of OpenAI's mission, apart from developing safe AGI, is to "avoid undue concentration of power". This is crucial, because safety is a convenient pretext for people whose real motivations are elitist. It's not that they want to slow down AI development; it's that they want to keep it restricted to a tight inner circle. We have at least as much to fear from elites who think only megacorps and governments are responsible enough to be given access as we do from AI itself. The failures of our elite class have been demonstrated again and again in recent decades--from the Iraq War to Covid. These people cannot be trusted as stewards, and we can't afford to see such potentially transformative technology employed to further ossify and insulate the existing power structure. OpenAI had the right idea. Keep the source closed if you must, and invest heavily on safety, but make the benefits available to all. The last thing we need is an AI priesthood that will inevitably turn corrupt.
- jacquesm 3y ago
  And so you strike a deal with Microsoft. That tracks.
  - for1nner 3y ago
    I think their point was that openAI (the nonpro) had the right idea in re: those concerns/concentration. The velocity of release and entwining with MSFT (by the profit side) might then be reasonably seen as a great concern for the board.
  - danenania 3y ago
    Advancing AI requires massive amounts of capital and compute. There's no way around this. It can only be done either within or in partnership with huge organizations. The default, in the absence of an OpenAI, is that it gets developed secretly by these organizations, and they get to decide who can use it (hint: it's not you, typical HN reader). OpenAI were the only ones at least trying to walk this tightrope, and I think they were doing a pretty good job. So what were the real motivations of these three wealthy/elite board members who are taking it upon themselves to decide this issue for the rest of humanity? It's looking less and less likely that safety was the reason.
- deleted 3y ago
  [deleted]
- hutzlibu 3y ago
  "OpenAI had the right idea. Keep the source closed if you must, and invest heavily on safety, but make the benefits available to all." Well, you see the result. Apart from this drama, lots of debate and speculation whether we are already in AGI territory, because it is all just a black box and no one knows what exactly was in the training data to be able to judge the quality of the output. That is way too much AI priesthood for me, just open it for real, if you don't want concentration of power.
  - danenania 3y ago
    Yeah that's certainly a debate worth having. My point is this coup against OpenAI (seemingly) wasn't started by people who want to make AI more open. They want to make it inaccessible unless you're in their club. To my reading, that is more starkly opposed to the charter than anything Sam Altman has done. Commercialization = giving wide access. Not total 100% open source access, but wide access nonetheless.
sgift 3y ago
Having read the word soup of contradicting weasel words that make up Claudes "constitution", 'superior safety approach' has so many asterisks it could be a star chart. The only thing the garbage Anthropic has produced is superior is in making some people feel good about themselves. (https://www.anthropic.com/index/claudes-constitution https://www.anthropic.com/index/claudes-constitution)
- nonethewiser 3y ago
  Im already out after the very first sentence. > How does a language model decide which questions it will engage with and which it deems inappropriate? This is what we mean by safety? Content moderation?
kgeist 3y ago
>According to the paper, Anthropic's superior safety approach was deliberately delaying release in order to avoid “advanc[ing] the rate of AI capabilities progress." Which can end up with China taking the lead. I don't understand why they think it's safer.
nsagent 3y ago
There are even more gems in the paper, like this one: > Suppose a leader pledges during a campaign to provide humanitarian aid to a stricken nation or the CEO of a company commits publicly to register its algorithms or guarantee its customers' data privacy. In both cases, the leader has issued a public statement before an audience who can hold them accountable if they fail to live up to their commitments. The political leader may be punished at the polls or subjected to a congressional investigation; the CEO may face disciplinary actions from the board of directors or reputational costs to the company's brand that can result in lost market share. I wonder if she had Sam Altman in mind while writing this.

himaraya 3y ago

The paper reads a lot more nuanced than that. It compares the "system card" released with GPT-4 to the delay of Claude and the merits of each approach vis a vis safety.

Emma_Goldman 3y ago
Not really, and your own description is little different from the person you're responding to anyway?
- deleted 3y ago
  [deleted]

danenania 3y ago

It's important to remember that part of OpenAI's mission, apart from developing safe AGI, is to "avoid undue concentration of power". This is crucial, because safety is a convenient pretext for people whose real motivations are elitist. It's not that they want to slow down AI development; it's that they want to keep it restricted to a tight inner circle. We have at least as much to fear from elites who think only megacorps and governments are responsible enough to be given access as we do from AI itself. The failures of our elite class have been demonstrated again and again in recent decades--from the Iraq War to Covid. These people cannot be trusted as stewards, and we can't afford to see such potentially transformative technology employed to further ossify and insulate the existing power structure. OpenAI had the right idea. Keep the source closed if you must, and invest heavily on safety, but make the benefits available to all. The last thing we need is an AI priesthood that will inevitably turn corrupt.

jacquesm 3y ago
And so you strike a deal with Microsoft. That tracks.
- for1nner 3y ago
  I think their point was that openAI (the nonpro) had the right idea in re: those concerns/concentration. The velocity of release and entwining with MSFT (by the profit side) might then be reasonably seen as a great concern for the board.
- danenania 3y ago
  Advancing AI requires massive amounts of capital and compute. There's no way around this. It can only be done either within or in partnership with huge organizations. The default, in the absence of an OpenAI, is that it gets developed secretly by these organizations, and they get to decide who can use it (hint: it's not you, typical HN reader). OpenAI were the only ones at least trying to walk this tightrope, and I think they were doing a pretty good job. So what were the real motivations of these three wealthy/elite board members who are taking it upon themselves to decide this issue for the rest of humanity? It's looking less and less likely that safety was the reason.
deleted 3y ago
[deleted]
hutzlibu 3y ago
"OpenAI had the right idea. Keep the source closed if you must, and invest heavily on safety, but make the benefits available to all." Well, you see the result. Apart from this drama, lots of debate and speculation whether we are already in AGI territory, because it is all just a black box and no one knows what exactly was in the training data to be able to judge the quality of the output. That is way too much AI priesthood for me, just open it for real, if you don't want concentration of power.
- danenania 3y ago
  Yeah that's certainly a debate worth having. My point is this coup against OpenAI (seemingly) wasn't started by people who want to make AI more open. They want to make it inaccessible unless you're in their club. To my reading, that is more starkly opposed to the charter than anything Sam Altman has done. Commercialization = giving wide access. Not total 100% open source access, but wide access nonetheless.

sgift 3y ago

Having read the word soup of contradicting weasel words that make up Claudes "constitution", 'superior safety approach' has so many asterisks it could be a star chart. The only thing the garbage Anthropic has produced is superior is in making some people feel good about themselves. (https://www.anthropic.com/index/claudes-constitution https://www.anthropic.com/index/claudes-constitution)

nonethewiser 3y ago
Im already out after the very first sentence. > How does a language model decide which questions it will engage with and which it deems inappropriate? This is what we mean by safety? Content moderation?

kgeist 3y ago

>According to the paper, Anthropic's superior safety approach was deliberately delaying release in order to avoid “advanc[ing] the rate of AI capabilities progress." Which can end up with China taking the lead. I don't understand why they think it's safer.

nsagent 3y ago

There are even more gems in the paper, like this one: > Suppose a leader pledges during a campaign to provide humanitarian aid to a stricken nation or the CEO of a company commits publicly to register its algorithms or guarantee its customers' data privacy. In both cases, the leader has issued a public statement before an audience who can hold them accountable if they fail to live up to their commitments. The political leader may be punished at the polls or subjected to a congressional investigation; the CEO may face disciplinary actions from the board of directors or reputational costs to the company's brand that can result in lost market share. I wonder if she had Sam Altman in mind while writing this.