7 ms·

Ask HN: Should the US pass sweeping restrictions on AI development?

https://time.com/6898967/ai-extinction-national-security-risks-report/ According to this government-commissioned report, AI at its current SOTA frontier represents an unprecedented existential risk to humanity. The best course of action, according to its authors, is for the United States to restrict compute on training models and punish open-sourcing weights with jail time. > The new AI agency should require AI companies on the “frontier” of the industry to obtain government permission to train and deploy new models above a certain lower threshold, the report adds. > Authorities should also “urgently” consider outlawing the publication of the “weights,” or inner workings, of powerful AI models, for example under open-source licenses, with violations possibly punishable by jail time, the report says. > And the government should further tighten controls on the manufacture and export of AI chips, and channel federal funding toward “alignment” research that seeks to make advanced AI safer, it recommends Do you believe, if the risks of AI are true, that this is the best course of action? Would this still allow AI development to reach a state of usefulness that has been promised for years, or will it completely destroy the momentum and lead to an AI winter?

lambdaba 3y ago
Even following all of this very closely it's hard to tell where the ceiling is, there are constant improvements, in 1 year or so we've seen at least 3 generations of significant performance upgrades. I have not read the report, what are the reasons they recommend such drastic measures?
- atleastoptimal 3y ago
  In summary, AI implemented in weapons systems present a tangible threat, along with AI trained past a certain order of magnitude potentially being adversarial to humans by default while not revealing its alignment.
  - smoldesu 3y ago
    Lethal autonomy is more than 20 years old in the United States: https://en.wikipedia.org/wiki/MIM-104_Patriot https://en.wikipedia.org/wiki/MIM-104_Patriot At this point, I think we can safely say that making a system that gives AI enough control to fuck up is a "skill issue" in earnest. The overwhelmingly obvious solution to this problem is to just not design systems that trust AI whatsoever. Problem solved, alignment crisis averted.
shrimp_emoji 3y ago
I think we're probably heading toward another AI winter anyway, so this would be like the government covering it up until things thaw again. But anyway, there's no way to enforce not running certain programs without some kind of Orwellian surveillance baked in to hardware that I would abhor more than Roko's basilisk, and you now make the people at the forefront of the technology criminals and geopolitical adversaries. It would be an ultra-bonehead move. As always, technology and law don't mix well.
smoldesu 3y ago
> According to this government-commissioned report I feel like I can guess their stance on the topic without even knowing what it is.
rhelz 3y ago
The more we are getting ourselves all worked up about how dangerous AI is, the less we are thinking about the fact that LLMs are sucking up all the intellectual property we created in the information age, and then selling it back to us. Astounding. IMHO the biggest achievement wasn't technical at all--it was how they were all able to do this gigantic end-run around copywrite, trademark, or patent protections. Essentially, there is no such thing as intellectual property anymore.
- atleastoptimal 3y ago
  AI training doesn't constitute copyright infringement. It's not the same thing. Could I host a Hollywood movie on my personal website? Could I take the text of "Harry Potter" and sell it as my own novel? If not, then intellectual property obviously still exists. Inaccurate hyperbole doesn't help your side. It just makes claims about AI training overstepping IP boundaries seem illegitimate.
  - rhelz 3y ago
    > AI training doesn't constitute copyright infringement. It's not the same thing. Why not? E.g., chagGpt 3.5, if you ask what the first paragraph of "Moby Dick" is, it prints it out verbatim. How is that not reselling somebody else's IP? If you ask it to print out the entire book of "Green Eggs and Ham" it dutifully says it can't because of copywrite restrictions. HOWEVER, you can ask it to print out the nth page, and it will happily print it out, for all 1 <= n<= #pages in book. It snorked up Dr. Suisses IP, and is reselling it to us. > Could I take the text of "Harry Potter" and sell it as my own novel? Well, I asked chatGtp to print out the first paragraph of Harry Potter, and it did, but said that it might violate copywright.... LLMs are quite "self aware" (for want of a better word) that they have appropriated copywrited IP. It will still sell it back to us. LLMs are a compressed representation of the training set. So...in answer to your question, no, you cannot take the text of "Harry Potter" and sell it as your own content. But chatGtp sure can!!!!! > Inaccurate hyperbole doesn't help your side. sigh yes, its important not to overstate your case. Let me try again: Intellectual Property Laws are for plebs like us. OpenAI and its cohorts, however, have found a way to rip off, repackage, and resell any IP they want to. And all this fuss over how dangerous AI, and how it should be regulated, etc, is just a false flag to distract us from the real issues.
romanhn 3y ago
The genie is out of the bottle, you can't just undo the last year of progress and knowledge. Even if the US took this drastic step, there will be other countries that will push forward. Or private interests. Or well-funded terrorist organizations. Etc, etc, you get the point. Much better, in my opinion, to stay ahead of the curve and tackle whatever issues head-on rather than burying your head in the sand, hoping for the best, and pulling out the surprised Pikachu face when someone else inevitably pushes state of the art forward.
artninja1988 3y ago
It's just a bunch of cranks who did the report. I hope no important politican takes this "report" seriously.
- reducesuffering 3y ago
  You're about 1 year behind the White House: https://twitter.com/kristjanmoore/status/1663860424100413440 https://twitter.com/kristjanmoore/status/1663860424100413440
lulznews 3y ago
That’s quite lulzy. But never estimate the stupidity of USG …
hollerith 3y ago
Yes. The US government should ban the training of new foundational models and should do whatever it can to slow down progress in GPUs because the way things are going now, we are very probably doomed. Even if China or some other country does not impose similar restrictions, a ban in the US and Great Britain would buy humanity a decade or maybe even 2 decades in which we might find a way out of the terrible situation we are in. (A decade or 2 is about how far the US and Great Britain are ahead in frontier AI research.) Also, buying an extra decade or 2 of life for everyone on Earth is a valuable goal in its own right even if after that Chinese AI researchers kill everyone. If humanity can survive somehow for a few more centuries, our prospects start improving because in a few centuries, the smartest among us might be smart enough to create powerful AI that doesn't quickly kill us all. It is not impossible that the current generation of humans can do it, but the probability is low enough (3.5 percent would be my current estimate) that I'm going to do anything I can to slow down AI research or preferably stop it altogether -- preferably for at least a few centuries.
- MichaelMug 3y ago
  How are we doomed? Or what is the doomsday scenario you have in mind? From what I read we have: massive unemployment, AI making military decisions. What else?
  - hollerith 3y ago
    Massive unemployment is not the main problem. No one has a good idea on how to control an AI that is more capable than people are over a broad range of pursuits. (Actually we don't need to be able to control such an AI, it suffices for it to be aligned with our long-term interests. No one knows how to do that either.) The Machine Intelligence Research Institute has been paying researchers to think up ideas for over 20 years. The executive director of that institute and its most senior researcher (Nate Soares and Eliezer Yudkowsky) have been publicly saying now for a few years that it is hopeless and that there is no way any of their institute's research directions will bear fruit before AI kills us all. They say that people just aren't smart enough to figure out how to control or align the powerful AIs that will be created soon if things keep on going the way they are going; consequently some sort of pause or halt on AI research -- almost certainly for at least a few decades -- is going to be necessary for us to survive. The people in charge of the AI labs care enough about their own survival and the survival of their families that they could probably be persuaded to adopt an effective alignment method if someone or some group can come up with one, but it is highly unlikely that anyone will do so in time. Note that whether the effective alignment method is used to align the AI with the broad interests of the entire human population or whether it is used to align the AI to the particular interests of only the leaders of the AI lab is not a particularly important consideration at the current time: it only becomes important in the very unlikely event that someone somewhere finds an effective alignment method. AI is essentially a subfield of the broader field of optimization; an AI applies an impressive amount of optimization pressure. (The human brain can, too.) When you (or a team of researchers) apply tons of optimization pressure, or create an AI that does so, you steer reality towards some "optimization point" or goal. Most optimization points are incompatible with continued human survival when enough optimization pressure is applied towards that point. And no one knows how to aim a powerful optimization process at the few points that are consistent with human survival and human flourishing. No one has even provided a mathematically precise specification of an optimization point consistent with human survival (and mathematical precision would be quite useful and I suspect also necessary for the specification to be useful to a team creating an AI). The closest anyone has gotten is a 2004 publication by Yudkowsky called "coherent extrapolated volition". "The hidden complexity of wishes" is one way in which applying tremendous optimization pressure can go wrong, like in the story of the powerful genie who grants 3 wishes to the person (robber?) who found a magic lamp, but the hidden complexity of wishes is not the main snag. If an AI is better at discovering scientific laws and inventing technologies than people are, it doesn't have to gain control over a nuclear arsenal: it can create its own arsenal that is better than nukes out of materials that people did not even suspect could be used as weapons (just as before about 1930 no one suspected that uranium could be turned into a bomb).
hollerith 3y ago
>Would this still allow AI development to reach a state of usefulness that has been promised for years It is better that the ban start now because no one has a method for determining which frontier models will become dangerously capable just as no one at OpenAI had any way of determining at the start of training that GPT-4 would be able to get a 92 on the bar exam. That is a pity because the benefits of creating more capable models is considerable: they could've made us all wealthier and saved lives through medical advances.
thoughtstheseus 3y ago
Throwing large amounts of compute, power, and learning cycles is building fascinating outcomes. That alone means strategic competitors should be secret.
BriggyDwiggs42 3y ago
“Outlaw the publication of weights” is this regulatory capture or just the usual gov stupidity?
animal_spirits 3y ago
I genuinely can't understand the doom. Can someone provide me with a realistic step by step scenario where language models destroy human civilization? How can it counter each individual step we take to correct/secure our computer systems? I feel like I'm lost here. There is beautiful mental gymnastics we have to make to conclude that AI will cause the civilization destroying dangers that we all think(want?) to happen. The only thing that I can think of is somehow a language model accidentally launches all the nukes in the world. But, that assumes that literally every step of the process can be actioned upon by typing text into a terminal. There's many armed people and physical keys that are in between GPT4 and the launch codes...
- p1esk 3y ago
  Two likely scenarios: 1. Bad people will use powerful models to do bad things. 2. Many jobs will be automated, and many people will get laid off.
  - animal_spirits 3y ago
    1. Bad people people have been doing bad things literally each and every day since the dawn of man. How does ChatGPT make this different? Do good guys not get the same tools to make the world better and safer? 2. This scenario I can understand causing _upheaval_ and significant societal change, but I can't see it causing the death of the human project like some predict. Will there be more food in this new society where it's incredibly cheap and efficient to make it? I presume so. Is it impossible for us to figure out how to distribute the food? No. Why will humanity come to it's end because people are not required to crunch numbers in excel day after day anymore? I'm not trying to be an ass, I just haven't seen an explanation that makes actual realistic conclusions.
    - p1esk 3y ago
      I thought we were talking about why would someone be concerned with recent progress in AI. That’s the question I tried to answer.
  - 3y ago
__rito__ 3y ago
What that genuinely achieves are: - Power being consolidated to only some small number of companies that can lobby and spend well. This leads market capture and oligopoly. And bars smaller players from entering the market. This leads to poor conditions for consumers similar to aviation industry, cellular networks and broadband market. - Innovation stops, and growth stagnates. Consumers and developers are left with poorer and higher paid products. The development of the technology stagnates. - I guess government will also monitor development of weapons, etc. China, Russia won't have similar restrictions, and will get ahead.
aeternum 3y ago
No, in an arms race either you win or your laws don't matter.
xeckr 3y ago
Absolutely not. One has to wonder if the cultural climate was what it is today during the golden age of microchips whether there would be calls to impose a maximum transistor count per area, especially if they could foretell how much power this technology gives to those who master it.
decremental 3y ago
[dead]
barfbagginus 3y ago
[flagged]
anon373839 3y ago
God, no. The United States can’t possibly put that genie back the bottle, but it can cement the incumbents’ status as government backed monopolies in the next era of computing. That’s what all this doomer nonsense is really about.

lambdaba 3y ago

Even following all of this very closely it's hard to tell where the ceiling is, there are constant improvements, in 1 year or so we've seen at least 3 generations of significant performance upgrades. I have not read the report, what are the reasons they recommend such drastic measures?

atleastoptimal 3y ago
In summary, AI implemented in weapons systems present a tangible threat, along with AI trained past a certain order of magnitude potentially being adversarial to humans by default while not revealing its alignment.
- smoldesu 3y ago
  Lethal autonomy is more than 20 years old in the United States: https://en.wikipedia.org/wiki/MIM-104_Patriot https://en.wikipedia.org/wiki/MIM-104_Patriot At this point, I think we can safely say that making a system that gives AI enough control to fuck up is a "skill issue" in earnest. The overwhelmingly obvious solution to this problem is to just not design systems that trust AI whatsoever. Problem solved, alignment crisis averted.

shrimp_emoji 3y ago

I think we're probably heading toward another AI winter anyway, so this would be like the government covering it up until things thaw again. But anyway, there's no way to enforce not running certain programs without some kind of Orwellian surveillance baked in to hardware that I would abhor more than Roko's basilisk, and you now make the people at the forefront of the technology criminals and geopolitical adversaries. It would be an ultra-bonehead move. As always, technology and law don't mix well.

smoldesu 3y ago

> According to this government-commissioned report I feel like I can guess their stance on the topic without even knowing what it is.

rhelz 3y ago

The more we are getting ourselves all worked up about how dangerous AI is, the less we are thinking about the fact that LLMs are sucking up all the intellectual property we created in the information age, and then selling it back to us. Astounding. IMHO the biggest achievement wasn't technical at all--it was how they were all able to do this gigantic end-run around copywrite, trademark, or patent protections. Essentially, there is no such thing as intellectual property anymore.

atleastoptimal 3y ago
AI training doesn't constitute copyright infringement. It's not the same thing. Could I host a Hollywood movie on my personal website? Could I take the text of "Harry Potter" and sell it as my own novel? If not, then intellectual property obviously still exists. Inaccurate hyperbole doesn't help your side. It just makes claims about AI training overstepping IP boundaries seem illegitimate.
- rhelz 3y ago
  > AI training doesn't constitute copyright infringement. It's not the same thing. Why not? E.g., chagGpt 3.5, if you ask what the first paragraph of "Moby Dick" is, it prints it out verbatim. How is that not reselling somebody else's IP? If you ask it to print out the entire book of "Green Eggs and Ham" it dutifully says it can't because of copywrite restrictions. HOWEVER, you can ask it to print out the nth page, and it will happily print it out, for all 1 <= n<= #pages in book. It snorked up Dr. Suisses IP, and is reselling it to us. > Could I take the text of "Harry Potter" and sell it as my own novel? Well, I asked chatGtp to print out the first paragraph of Harry Potter, and it did, but said that it might violate copywright.... LLMs are quite "self aware" (for want of a better word) that they have appropriated copywrited IP. It will still sell it back to us. LLMs are a compressed representation of the training set. So...in answer to your question, no, you cannot take the text of "Harry Potter" and sell it as your own content. But chatGtp sure can!!!!! > Inaccurate hyperbole doesn't help your side. sigh yes, its important not to overstate your case. Let me try again: Intellectual Property Laws are for plebs like us. OpenAI and its cohorts, however, have found a way to rip off, repackage, and resell any IP they want to. And all this fuss over how dangerous AI, and how it should be regulated, etc, is just a false flag to distract us from the real issues.

romanhn 3y ago

The genie is out of the bottle, you can't just undo the last year of progress and knowledge. Even if the US took this drastic step, there will be other countries that will push forward. Or private interests. Or well-funded terrorist organizations. Etc, etc, you get the point. Much better, in my opinion, to stay ahead of the curve and tackle whatever issues head-on rather than burying your head in the sand, hoping for the best, and pulling out the surprised Pikachu face when someone else inevitably pushes state of the art forward.

artninja1988 3y ago

It's just a bunch of cranks who did the report. I hope no important politican takes this "report" seriously.

reducesuffering 3y ago
You're about 1 year behind the White House: https://twitter.com/kristjanmoore/status/1663860424100413440 https://twitter.com/kristjanmoore/status/1663860424100413440

lulznews 3y ago

That’s quite lulzy. But never estimate the stupidity of USG …

hollerith 3y ago

Yes. The US government should ban the training of new foundational models and should do whatever it can to slow down progress in GPUs because the way things are going now, we are very probably doomed. Even if China or some other country does not impose similar restrictions, a ban in the US and Great Britain would buy humanity a decade or maybe even 2 decades in which we might find a way out of the terrible situation we are in. (A decade or 2 is about how far the US and Great Britain are ahead in frontier AI research.) Also, buying an extra decade or 2 of life for everyone on Earth is a valuable goal in its own right even if after that Chinese AI researchers kill everyone. If humanity can survive somehow for a few more centuries, our prospects start improving because in a few centuries, the smartest among us might be smart enough to create powerful AI that doesn't quickly kill us all. It is not impossible that the current generation of humans can do it, but the probability is low enough (3.5 percent would be my current estimate) that I'm going to do anything I can to slow down AI research or preferably stop it altogether -- preferably for at least a few centuries.

MichaelMug 3y ago
How are we doomed? Or what is the doomsday scenario you have in mind? From what I read we have: massive unemployment, AI making military decisions. What else?
- hollerith 3y ago
  Massive unemployment is not the main problem. No one has a good idea on how to control an AI that is more capable than people are over a broad range of pursuits. (Actually we don't need to be able to control such an AI, it suffices for it to be aligned with our long-term interests. No one knows how to do that either.) The Machine Intelligence Research Institute has been paying researchers to think up ideas for over 20 years. The executive director of that institute and its most senior researcher (Nate Soares and Eliezer Yudkowsky) have been publicly saying now for a few years that it is hopeless and that there is no way any of their institute's research directions will bear fruit before AI kills us all. They say that people just aren't smart enough to figure out how to control or align the powerful AIs that will be created soon if things keep on going the way they are going; consequently some sort of pause or halt on AI research -- almost certainly for at least a few decades -- is going to be necessary for us to survive. The people in charge of the AI labs care enough about their own survival and the survival of their families that they could probably be persuaded to adopt an effective alignment method if someone or some group can come up with one, but it is highly unlikely that anyone will do so in time. Note that whether the effective alignment method is used to align the AI with the broad interests of the entire human population or whether it is used to align the AI to the particular interests of only the leaders of the AI lab is not a particularly important consideration at the current time: it only becomes important in the very unlikely event that someone somewhere finds an effective alignment method. AI is essentially a subfield of the broader field of optimization; an AI applies an impressive amount of optimization pressure. (The human brain can, too.) When you (or a team of researchers) apply tons of optimization pressure, or create an AI that does so, you steer reality towards some "optimization point" or goal. Most optimization points are incompatible with continued human survival when enough optimization pressure is applied towards that point. And no one knows how to aim a powerful optimization process at the few points that are consistent with human survival and human flourishing. No one has even provided a mathematically precise specification of an optimization point consistent with human survival (and mathematical precision would be quite useful and I suspect also necessary for the specification to be useful to a team creating an AI). The closest anyone has gotten is a 2004 publication by Yudkowsky called "coherent extrapolated volition". "The hidden complexity of wishes" is one way in which applying tremendous optimization pressure can go wrong, like in the story of the powerful genie who grants 3 wishes to the person (robber?) who found a magic lamp, but the hidden complexity of wishes is not the main snag. If an AI is better at discovering scientific laws and inventing technologies than people are, it doesn't have to gain control over a nuclear arsenal: it can create its own arsenal that is better than nukes out of materials that people did not even suspect could be used as weapons (just as before about 1930 no one suspected that uranium could be turned into a bomb).

hollerith 3y ago

>Would this still allow AI development to reach a state of usefulness that has been promised for years It is better that the ban start now because no one has a method for determining which frontier models will become dangerously capable just as no one at OpenAI had any way of determining at the start of training that GPT-4 would be able to get a 92 on the bar exam. That is a pity because the benefits of creating more capable models is considerable: they could've made us all wealthier and saved lives through medical advances.

thoughtstheseus 3y ago

Throwing large amounts of compute, power, and learning cycles is building fascinating outcomes. That alone means strategic competitors should be secret.

BriggyDwiggs42 3y ago

“Outlaw the publication of weights” is this regulatory capture or just the usual gov stupidity?

animal_spirits 3y ago

I genuinely can't understand the doom. Can someone provide me with a realistic step by step scenario where language models destroy human civilization? How can it counter each individual step we take to correct/secure our computer systems? I feel like I'm lost here. There is beautiful mental gymnastics we have to make to conclude that AI will cause the civilization destroying dangers that we all think(want?) to happen. The only thing that I can think of is somehow a language model accidentally launches all the nukes in the world. But, that assumes that literally every step of the process can be actioned upon by typing text into a terminal. There's many armed people and physical keys that are in between GPT4 and the launch codes...

p1esk 3y ago
Two likely scenarios: 1. Bad people will use powerful models to do bad things. 2. Many jobs will be automated, and many people will get laid off.
- animal_spirits 3y ago
  1. Bad people people have been doing bad things literally each and every day since the dawn of man. How does ChatGPT make this different? Do good guys not get the same tools to make the world better and safer? 2. This scenario I can understand causing _upheaval_ and significant societal change, but I can't see it causing the death of the human project like some predict. Will there be more food in this new society where it's incredibly cheap and efficient to make it? I presume so. Is it impossible for us to figure out how to distribute the food? No. Why will humanity come to it's end because people are not required to crunch numbers in excel day after day anymore? I'm not trying to be an ass, I just haven't seen an explanation that makes actual realistic conclusions.
  - p1esk 3y ago
    I thought we were talking about why would someone be concerned with recent progress in AI. That’s the question I tried to answer.
- 3y ago

__rito__ 3y ago

What that genuinely achieves are: - Power being consolidated to only some small number of companies that can lobby and spend well. This leads market capture and oligopoly. And bars smaller players from entering the market. This leads to poor conditions for consumers similar to aviation industry, cellular networks and broadband market. - Innovation stops, and growth stagnates. Consumers and developers are left with poorer and higher paid products. The development of the technology stagnates. - I guess government will also monitor development of weapons, etc. China, Russia won't have similar restrictions, and will get ahead.

aeternum 3y ago

No, in an arms race either you win or your laws don't matter.

xeckr 3y ago

Absolutely not. One has to wonder if the cultural climate was what it is today during the golden age of microchips whether there would be calls to impose a maximum transistor count per area, especially if they could foretell how much power this technology gives to those who master it.

decremental 3y ago

[dead]

barfbagginus 3y ago

[flagged]

anon373839 3y ago

God, no. The United States can’t possibly put that genie back the bottle, but it can cement the incumbents’ status as government backed monopolies in the next era of computing. That’s what all this doomer nonsense is really about.