3 ms·

> Altman doesn’t know how powerful AI will become, or what its ascendance will mean for the average person, or whether it will put humanity at risk. In all ser

> Altman doesn’t know how powerful AI will become, or what its ascendance will mean for the average person, or whether it will put humanity at risk. In all seriousness, if practical AI development is potentially an extinction-level risk, it should simply cease. It may be a technology that mankind is not ready for, not equipped to handle. In this case, those who recklessly seek to commercialize it are, in a very real sense, privatizing profits and socializing losses. ("Heads, I become as rich as Croesus. Tails, we all die -- or, at the very least, we all experience unprecedented social upheaval.")

kmeisthax 3y ago
Heads and tails sound like the exact same scenario.
amelius 3y ago
Otoh, if we don't create it, some nation state actor will.
- haswell 3y ago
  It would require a military threat to shut this down globally, i.e. willingness to go to war to prevent further development. Which is not necessarily a deterrent anyway.
  - flangola7 3y ago
    There was a Time article detailing exactly this necessity.
- entropicdrifter 3y ago
  Just like with nukes, it's an international 200+ actor version of the prisoner's dilemma
- adventured 3y ago
  They're going to anyway. And of course it won't just be nation state actors. It'll be criminal organizations out of Russia, or terrorist organizations, or some clandestine team inside a three letter US agency. They're proliferating rapidly. The hardware will keep getting better. A lot more people will learn how to build them, in a lot of different locations around the globe. The genie is never going back into the bottle. There will be extremely nefarious, highly potent, alt LLMs created, and soon. Focused on variations of social engineering. They will cause tens of billions of dollars in damages globally and lead to tight regulations on the proliferation of LLMs. Most likely all LLMs or the equivalent will have to be registered with a host nation and include details about its capabilities.
  - cubefox 3y ago
    There could be systems far more powerful than LLMs. Systems that roughly work like animal brains, while being much more intelligent than any human. These systems probably can't be controlled once they are deployed, so the only hope is to create their goal function such that they are motivated to act in humanity's best interests. Nobody knows how to do that though.
mrcheesebreeze 3y ago
[dead]
cubefox 3y ago
The problem is that it can't realistically be stopped. In terms of game theory, it's a prisoner's dilemma. All nations are incentivized to race ahead with AI, since the benefits are enormous before they turn catastrophic. Moreover, international agreements (similar to a nuclear test treaty) can't be checked or enforced, since countries can just build AI supercomputers under ground where they can't be seen from satellites nor destroyed by rockets. So the only hope is that the first organization which develops a superintelligence manages to align it with human values, or, alternatively, they manage to build a superintelligent "task AI" (non-agentic AI) which just follows instructions, and then use it to enforce the global stop in AI development until the alignment problem is solved.
- A_D_E_P_T 3y ago
  > All nations are incentivized to race ahead with AI But what's happening, to all appearances, is that it isn't "nations" driving this. It's commerce and the investor class. If AI development were de facto restricted to, e.g., the US Military and the Chinese military, that would slow everything down by years -- perhaps even longer than that -- and the most disruptive effects might never become manifest. I'll grant that AI research probably cannot be stopped at this point. But with Altman agitating for "regulations" it should really be apparent that the best form of regulation in this case is the ultimate one: A total ban on above-ground AI development.
  - cubefox 3y ago
    Governments can and do pay private companies to develop things. OpenAI would basically be a defense contractor with tons of funding. I don't see this slowing down AI development by a lot.
version_five 3y ago
> In all seriousness, if practical AI development is potentially an extinction-level risk, it should simply cease. In all seriousness, it isn't, at least not in a way that's different from the risks of general purpose computing. Social media (anything that let's humanity interact at a scale we aren't prepared for), biological (disease) research, nuclear weapons, inadequate asteroid and tsunami preparedness, etc are actual risks, it's ridiculous and dangerous to put computers in the same category, this is a manufactured panic to sell chatbots.

kmeisthax 3y ago

Heads and tails sound like the exact same scenario.

amelius 3y ago

Otoh, if we don't create it, some nation state actor will.

haswell 3y ago
It would require a military threat to shut this down globally, i.e. willingness to go to war to prevent further development. Which is not necessarily a deterrent anyway.
- flangola7 3y ago
  There was a Time article detailing exactly this necessity.
entropicdrifter 3y ago
Just like with nukes, it's an international 200+ actor version of the prisoner's dilemma
adventured 3y ago
They're going to anyway. And of course it won't just be nation state actors. It'll be criminal organizations out of Russia, or terrorist organizations, or some clandestine team inside a three letter US agency. They're proliferating rapidly. The hardware will keep getting better. A lot more people will learn how to build them, in a lot of different locations around the globe. The genie is never going back into the bottle. There will be extremely nefarious, highly potent, alt LLMs created, and soon. Focused on variations of social engineering. They will cause tens of billions of dollars in damages globally and lead to tight regulations on the proliferation of LLMs. Most likely all LLMs or the equivalent will have to be registered with a host nation and include details about its capabilities.
- cubefox 3y ago
  There could be systems far more powerful than LLMs. Systems that roughly work like animal brains, while being much more intelligent than any human. These systems probably can't be controlled once they are deployed, so the only hope is to create their goal function such that they are motivated to act in humanity's best interests. Nobody knows how to do that though.

mrcheesebreeze 3y ago

[dead]

cubefox 3y ago

The problem is that it can't realistically be stopped. In terms of game theory, it's a prisoner's dilemma. All nations are incentivized to race ahead with AI, since the benefits are enormous before they turn catastrophic. Moreover, international agreements (similar to a nuclear test treaty) can't be checked or enforced, since countries can just build AI supercomputers under ground where they can't be seen from satellites nor destroyed by rockets. So the only hope is that the first organization which develops a superintelligence manages to align it with human values, or, alternatively, they manage to build a superintelligent "task AI" (non-agentic AI) which just follows instructions, and then use it to enforce the global stop in AI development until the alignment problem is solved.

A_D_E_P_T 3y ago
> All nations are incentivized to race ahead with AI But what's happening, to all appearances, is that it isn't "nations" driving this. It's commerce and the investor class. If AI development were de facto restricted to, e.g., the US Military and the Chinese military, that would slow everything down by years -- perhaps even longer than that -- and the most disruptive effects might never become manifest. I'll grant that AI research probably cannot be stopped at this point. But with Altman agitating for "regulations" it should really be apparent that the best form of regulation in this case is the ultimate one: A total ban on above-ground AI development.
- cubefox 3y ago
  Governments can and do pay private companies to develop things. OpenAI would basically be a defense contractor with tons of funding. I don't see this slowing down AI development by a lot.

version_five 3y ago

> In all seriousness, if practical AI development is potentially an extinction-level risk, it should simply cease. In all seriousness, it isn't, at least not in a way that's different from the risks of general purpose computing. Social media (anything that let's humanity interact at a scale we aren't prepared for), biological (disease) research, nuclear weapons, inadequate asteroid and tsunami preparedness, etc are actual risks, it's ridiculous and dangerous to put computers in the same category, this is a manufactured panic to sell chatbots.