5 ms·

We need to push the notion that "closed-source LLMs are super dangerous, an existential risk to humanity". Basically we need to equate "safety" in LLMs to mean

We need to push the notion that "closed-source LLMs are super dangerous, an existential risk to humanity". Basically we need to equate "safety" in LLMs to mean "being open-source". OpenAI keeps talking about "safety" as the most important goal. If we define it to mean "open-source" then they will be pushed into a corner.

jackvezkovic 4y ago
It's "Security Through AI Obscurity"
michaelmior 4y ago
> Basically we need to equate "safety" in LLMs to mean "being open-source". I think open source is a reasonable component to safety, but I wouldn't want to make them equal. Open source may be necessary for safety, but I wouldn't call it sufficient. For example, assume the source code, the model, the training data, and all the model weights are open source. How do you know that the model was actually trained using that training data? Very few organizations have the capacity to train models at this scale themselves.
wseqyrku 4y ago
> Basically we need to equate "safety" in LLMs to mean "being open-source". Another way to put it is to make it more accessible to everyone, right? The opposite of that is happening to nuclear power. They're actually trying to stop any more countries to have the technology at their disposal. So no, make it "open source" doesn't make it safe by any stretch of imagination.
- blibble 4y ago
  "nuclear power" is open source, and this is one of the fundamental ideas behind the NPT reactor blueprints have been accessible to IAEA members for something like 50 years
tel 4y ago
Why do you believe that "open source" would imply greater safety? Here, I'll loosely define "safety" to be "avoidance of harm to individuals or society that would have otherwise not occurred without the use of LLM technology". Feel free to modify that definition as you see fit, but I'm genuinely curious what the argument is that open source is necessary, sufficient, or even a major component of achieving safety.
seydor 4y ago
Good luck after a decade of convincing the public about the opposite (walled gardens)
LouisSayers 4y ago
> OpenAI keeps talking about "safety" Whenever I see this I simply think "monopoly". It smells of anti-competitiveness and is a kind of open forum lobbying to restrict who gets to lead the AI wave (and make a shit tonne of money in the process).

jackvezkovic 4y ago

It's "Security Through AI Obscurity"

michaelmior 4y ago

> Basically we need to equate "safety" in LLMs to mean "being open-source". I think open source is a reasonable component to safety, but I wouldn't want to make them equal. Open source may be necessary for safety, but I wouldn't call it sufficient. For example, assume the source code, the model, the training data, and all the model weights are open source. How do you know that the model was actually trained using that training data? Very few organizations have the capacity to train models at this scale themselves.

wseqyrku 4y ago

> Basically we need to equate "safety" in LLMs to mean "being open-source". Another way to put it is to make it more accessible to everyone, right? The opposite of that is happening to nuclear power. They're actually trying to stop any more countries to have the technology at their disposal. So no, make it "open source" doesn't make it safe by any stretch of imagination.

blibble 4y ago
"nuclear power" is open source, and this is one of the fundamental ideas behind the NPT reactor blueprints have been accessible to IAEA members for something like 50 years

tel 4y ago

Why do you believe that "open source" would imply greater safety? Here, I'll loosely define "safety" to be "avoidance of harm to individuals or society that would have otherwise not occurred without the use of LLM technology". Feel free to modify that definition as you see fit, but I'm genuinely curious what the argument is that open source is necessary, sufficient, or even a major component of achieving safety.

seydor 4y ago

Good luck after a decade of convincing the public about the opposite (walled gardens)

LouisSayers 4y ago

> OpenAI keeps talking about "safety" Whenever I see this I simply think "monopoly". It smells of anti-competitiveness and is a kind of open forum lobbying to restrict who gets to lead the AI wave (and make a shit tonne of money in the process).