7 ms·

Remember when OAI said: "Oh no we're not going to release GPT-2 because its so advanced that it's a threat to humankind" meanwhile it was dumb as rocks. Scare

Remember when OAI said: "Oh no we're not going to release GPT-2 because its so advanced that it's a threat to humankind" meanwhile it was dumb as rocks. Scaremongering purely for the sake of it. The only remotely possible "safety" part I would acknowledge is that it should be balanced against biases if used in systems like loans, grants, etc.

drcode 2y ago
It's always easy to make fun of people who are trying to be safe after the fact "trying to be safe" means you sometimes don't do something, even if there's only a 10% chance something bad will happen Why bother checking if there's a bullet in the chamber of a gun before handling it? It looks so foolish every time you check and don't find a bullet.
- deleted 2y ago
  [deleted]
- lannisterstark 2y ago
  the problem is that there's a very real danger in one thing, and on the other hand, the danger is "omg haven't you read this scifi novel or seen this movie?!?!" Bullets kill people when fired by firearms. I fail to see how LLMs do.
padolsey 2y ago
The thing is, such prophecies are all very wrong until they're very right. The idea of an LLM (with capabilities of e.g. <1 yr away) being given access to a VM and spinning up others without oversight, IMHO, is real enough. Biases like "omg it's gonna prefer western names in CVs" is a bit meh. The real stuff is not evident yet.
- lannisterstark 2y ago
  >. The idea of an LLM (with capabilities of e.g. <1 yr away) being given access to a VM and spinning up others without oversight, IMHO, is real enough. Is that really a danger? I can shut off a machine or VMs.
  - kalkin 2y ago
    This line of argument indicates a basic refusal to take the threat model seriously, I think. Should Google worry about Chinese state-backed attackers using attacking its systems to target dissidents or for corporate or military espionage? "Why, when they're using machines or VMs, and you can just shut those off?" At a sophisticated-human level of capability, there are many established techniques to circumvent people trying to shut off your access to compute in general, or even to specific systems. It's certainly possible that AI will never reach a sophisticated-human level of capability at this task—it hasn't yet—but the fact that computers have off switches gives no information about the likelihood or proximity of reaching that threshold.
ben_w 2y ago
People have bad memories. I keep going back to the actual announcement because what they actually say is: """This decision, as well as our discussion of it, is an experiment: while we are not sure that it is the right decision today, we believe that the AI community will eventually need to tackle the issue of publication norms in a thoughtful way in certain research areas. Other disciplines such as biotechnology and cybersecurity have long had active debates about responsible publication in cases with clear misuse potential, and we hope that our experiment will serve as a case study for more nuanced discussions of model and code release decisions in the AI community. We are aware that some researchers have the technical capacity to reproduce and open source our results. We believe our release strategy limits the initial set of organizations who may choose to do this, and gives the AI community more time to have a discussion about the implications of such systems.""" - https://openai.com/index/better-language-models/ https://openai.com/index/better-language-models/ > The only remotely possible "safety" part I would acknowledge is that it should be balanced against biases if used in systems like loans, grants, etc. That's a very mid-1990s view of algorithmic risk, given models like this are already being used for scams and propaganda.
- LegionMammal978 2y ago
  I'd imagine there's a wide spectrum between "release the latest model immediately to everyone with no idea what it's capable of" and OpenAI's apparent "release the model (or increasingly, any information about it) literally never, not even when it's long been left in the dust".
  - ben_w 2y ago
    Yes, indeed. However, given the capacity for some of the more capable downloadable models to enable automation of fraud, I am not convinced OpenAI is incorrect here. If OpenAI and Facebook both get sued out of existence due to their models being used for fraud and them being deemed liable for that fraud, the OpenAI models become unavailable, the Facebook models remain circulating forever
- modeless 2y ago
  > we hope that our experiment will serve as a case study for more nuanced discussions People trot this out every time this comes up, but this actually makes it even worse. This was only part of the reason, the other part was that they seemed to legitimately think there could be a real reason to withhold the model ("we are not sure"). In hindsight this looks silly, and I don't believe it improved the "discussion" in any way. If anything it seems to give ammunition to the people who say the concerns are overblown and self-serving, which I'm sure is not what OpenAI intended. So to me this is a failure on both counts, and this was foreseeable at the time.
ugh123 2y ago
I think theres a big difference between a model that is "dumb" and a model that can cause harm by running loose with ill-thought actions.

drcode 2y ago

It's always easy to make fun of people who are trying to be safe after the fact "trying to be safe" means you sometimes don't do something, even if there's only a 10% chance something bad will happen Why bother checking if there's a bullet in the chamber of a gun before handling it? It looks so foolish every time you check and don't find a bullet.

deleted 2y ago
[deleted]
lannisterstark 2y ago
the problem is that there's a very real danger in one thing, and on the other hand, the danger is "omg haven't you read this scifi novel or seen this movie?!?!" Bullets kill people when fired by firearms. I fail to see how LLMs do.

padolsey 2y ago

The thing is, such prophecies are all very wrong until they're very right. The idea of an LLM (with capabilities of e.g. <1 yr away) being given access to a VM and spinning up others without oversight, IMHO, is real enough. Biases like "omg it's gonna prefer western names in CVs" is a bit meh. The real stuff is not evident yet.

lannisterstark 2y ago
>. The idea of an LLM (with capabilities of e.g. <1 yr away) being given access to a VM and spinning up others without oversight, IMHO, is real enough. Is that really a danger? I can shut off a machine or VMs.
- kalkin 2y ago
  This line of argument indicates a basic refusal to take the threat model seriously, I think. Should Google worry about Chinese state-backed attackers using attacking its systems to target dissidents or for corporate or military espionage? "Why, when they're using machines or VMs, and you can just shut those off?" At a sophisticated-human level of capability, there are many established techniques to circumvent people trying to shut off your access to compute in general, or even to specific systems. It's certainly possible that AI will never reach a sophisticated-human level of capability at this task—it hasn't yet—but the fact that computers have off switches gives no information about the likelihood or proximity of reaching that threshold.

ben_w 2y ago

People have bad memories. I keep going back to the actual announcement because what they actually say is: """This decision, as well as our discussion of it, is an experiment: while we are not sure that it is the right decision today, we believe that the AI community will eventually need to tackle the issue of publication norms in a thoughtful way in certain research areas. Other disciplines such as biotechnology and cybersecurity have long had active debates about responsible publication in cases with clear misuse potential, and we hope that our experiment will serve as a case study for more nuanced discussions of model and code release decisions in the AI community. We are aware that some researchers have the technical capacity to reproduce and open source our results. We believe our release strategy limits the initial set of organizations who may choose to do this, and gives the AI community more time to have a discussion about the implications of such systems.""" - https://openai.com/index/better-language-models/ https://openai.com/index/better-language-models/ > The only remotely possible "safety" part I would acknowledge is that it should be balanced against biases if used in systems like loans, grants, etc. That's a very mid-1990s view of algorithmic risk, given models like this are already being used for scams and propaganda.

LegionMammal978 2y ago
I'd imagine there's a wide spectrum between "release the latest model immediately to everyone with no idea what it's capable of" and OpenAI's apparent "release the model (or increasingly, any information about it) literally never, not even when it's long been left in the dust".
- ben_w 2y ago
  Yes, indeed. However, given the capacity for some of the more capable downloadable models to enable automation of fraud, I am not convinced OpenAI is incorrect here. If OpenAI and Facebook both get sued out of existence due to their models being used for fraud and them being deemed liable for that fraud, the OpenAI models become unavailable, the Facebook models remain circulating forever
modeless 2y ago
> we hope that our experiment will serve as a case study for more nuanced discussions People trot this out every time this comes up, but this actually makes it even worse. This was only part of the reason, the other part was that they seemed to legitimately think there could be a real reason to withhold the model ("we are not sure"). In hindsight this looks silly, and I don't believe it improved the "discussion" in any way. If anything it seems to give ammunition to the people who say the concerns are overblown and self-serving, which I'm sure is not what OpenAI intended. So to me this is a failure on both counts, and this was foreseeable at the time.

ugh123 2y ago

I think theres a big difference between a model that is "dumb" and a model that can cause harm by running loose with ill-thought actions.