4 ms·

We have plenty of technologies whose development has been stopped via collective agreement: nuclear proliferation, biological weapons, space weapons. AI won't b

We have plenty of technologies whose development has been stopped via collective agreement: nuclear proliferation, biological weapons, space weapons. AI won't be one of those technologies though, because it is relatively easy to develop and it gives advantages with externalized disadvantages.

didericis 4y ago
Project Orion is the most tragic (but probably wisest) stopped technology, was in both the space weapon and nuclear proliferation bucket. But it also could have enabled luxuriously spacious trips to Saturn in the 70s had it been realized. I hope we manage to trust each other enough one day to make it a reality.
radlad 4y ago
These are fantastic counterpoints, thanks. Still, I can't help but wonder if they are now out of the public view, but still being developed. (e.g. North Korea nuclear missile tests.) I think you make a fantastic point that the barrier to entry in developing an AI is much lower than building a nuclear bomb.
- gostsamo 4y ago
  In regard to chemical weapons - no, I think that they are abondonware because they are ineffective. Something similar with bio weapons, they can backfire to your own population rather easily. Nuclear are their special case, but still their proliferation is extremely limited.
oh_sigh 4y ago
You're right that it won't be - the difference is the tools for building those things were not distributed across millions of different people. It was distributed across only a small number of governments.
bloppe 4y ago
"Stopped" is a strong word. Nuclear proliferation and chemical weapons are ongoing and increasing threats. I'd bet a lot of money that we'll see the use of space weapons in our lifetime.
- gostsamo 4y ago
  In regard to chemical weapons, they are decreasing danger because they are not effective in modern warfare. I'd expect for their usage to decrease. Nuclear is is not something proliferating besides North Korea and even then it is deturance and not an appocaliptic scenario. Space weapons are forbidden by treaty and though some countries are playing close to the read line, I doubt that they will be deployed. They would be political liability while being rather vulnerable in space. Hence hypersonic weapons which work better for the same goals.
  - bloppe 4y ago
    Nuclear weapons are proliferating in many places besides North Korea: https://www.npr.org/2023/02/22/1158529106/nuclear-treaty-new-start-putin https://www.npr.org/2023/02/22/1158529106/nuclear-treaty-new... https://abcnews.go.com/Politics/iran-make-material-nuclear-device-12-days-us/story?id=97536371 https://abcnews.go.com/Politics/iran-make-material-nuclear-d... https://news.usni.org/2023/01/24/report-to-congress-on-chinese-nuclear-and-missile-proliferation https://news.usni.org/2023/01/24/report-to-congress-on-chine... Chemical weapons are very effective in modern warfare: https://en.m.wikipedia.org/wiki/Use_of_chemical_weapons_in_the_Syrian_civil_war https://en.m.wikipedia.org/wiki/Use_of_chemical_weapons_in_t... Though, it's more commonly used by non-state actors: https://en.m.wikipedia.org/wiki/Chemical_terrorism https://en.m.wikipedia.org/wiki/Chemical_terrorism
    - gostsamo 4y ago
      Iran is the exception that confirms the rule. They were so heavily sanctioned that it became feasible for them to say "screw it, we will do nukes anyway". I won't say that nuclear proliferation is completely stopped, but the system works for the most part. China and Russia are old nuclear powers and them having nukes is expected. Better counter examples are India and Pakistan, but I don't know the history of their nuclear programs well enough to comment with confidence. The chemical weapons in Syria were old stockpiles and I'd argue that the use there was not an increase of their use, but an opportunity attack. I don't think that there are many groups who currently have any kind of chemical program and can use it systematically.
deleted 4y ago
[deleted]
McDyver 4y ago
I would argue that they didn't proliferate due to high barriers to entry. I'm sure any "rogue" states that at one point would be ruled by extremists, would invest in those if it would give them a competitive advantage. AI can be, or will be at one point, developed at home using consumer devices.
- gostsamo 4y ago
  In regard to nuclear weapons the high barrier to entry is mostly because the international community made artificial difficulties in obtaining and purifying the materials. Do not forget that the first nuclear weapons were created with tech from the forties in just a few years. Since then the challenges and possible approaches have become better known and if given the chance and will, any modern country can create them if not for the international backlash.
- zirgs 4y ago
  North Korea is dirt poor, but still managed to create nuclear weapons. The barrier is not that high. If they could do it then countries like Poland or Finland could do that too. They don't, because they have other security guarantees at the moment. If NATO and EU collapsed - the number of nuclear weapons states would definitely increase.
  - akiselev 4y ago
    > The barrier is not that high. That ignores the sixty years it took North Korea to develop nuclear weapons. It took the US and USSR less time to develop the first nuclear weapons after the discovery of radioactivity in the late 19th century! The Soviet Union first began training DPRK nuclear scientists back in 1956 [1] and North Korea only managed to finally build nuclear weapons because of assistance from Pakistan with the uranium enrichment process in the late 90s and early 2000s, specifically the design for the centrifuges that fell into Libya's hands [2][3]. [1] https://en.wikipedia.org/wiki/Timeline_of_the_North_Korean_nuclear_program#Phase_I https://en.wikipedia.org/wiki/Timeline_of_the_North_Korean_n... [2] https://en.wikipedia.org/wiki/North_Korea%E2%80%93Pakistan_relations#Allegations_of_nuclear_assistance_by_Pakistan https://en.wikipedia.org/wiki/North_Korea%E2%80%93Pakistan_r... [3] https://en.wikipedia.org/wiki/Abdul_Qadeer_Khan#North_Korea,_Iran,_and_Libya https://en.wikipedia.org/wiki/Abdul_Qadeer_Khan#North_Korea,...
nradov 4y ago
Nuclear and space weapons proliferation have been slowed down but not stopped. The number of nuclear powers is larger now than ever before. While no one has launched kinetic weapons into orbit lately, the superpowers are currently engaged in an anti-satellite weapon arms race.
- gostsamo 4y ago
  The anti-satellite weapons are ground-based and are part of the playing close to the read line together with the spy satellites. The space weapons are when you have a nuclear head deployed in space with a few seconds to minutes ability to reach anywhere on Earth.
  - nradov 4y ago
    You seem to have some misunderstandings about orbital dynamics and nuclear weapons. Placing nuclear weapons in orbit wouldn't allow for striking anywhere on Earth within a "few seconds to minutes" unless you had an absolutely enormous constellation that would be unaffordable even for superpowers. You are forgetting to account for orbital parameter changes and then re-entry. In most realistic circumstances, a ground or sea based ICBM would be able to strike most points on Earth faster than a space weapon. Do the math on this, or you can try to simulate it in something like KSP. Nuclear warheads also can't be stored for more than a few years and still be expected to work reliably. They require periodic intensive inspection and maintenance. Even though space launch costs are coming down there is still no feasible way to conduct such maintenance in orbit.
    - gostsamo 4y ago
      Check some of the old cold war designs, they definitely thought in that direction. Space weapons were prohibited partly for the reasons you mentioned and partly due to the political liability of militarizing the space.
stcredzero 4y ago
Another way to put it: As technology progresses, the amount of stupidity to make an "oopsie" exponentially decreases while the size of the potential "oopsies" exponentially increases. Eventually, one gets to the point where even super geniuses are very likely to create a civilization ending "oopsie" in any given year. This seems to be where we are quickly heading with AI.
mdgrech23 4y ago
Stopped is an interesting word here. A lot of the countries that agreed not to further develop nukes and or hold nukes are either a) lying or b) we (the USA) not provide their defense to our detriment meanwhile the real bad guys who we really need to be concerned about still have plenty of nukes.
alach11 4y ago
The supplies to create biological weapons are actually surprisingly cheap! Not as cheap as graphics cards though. To stop AI development would require pretty extreme restrictions on computing resources. It's really hard to imagine that working without having massive negative knock-on effects on other fields/industries. The economic pressure alone seems to make an effective "anti-AI" policy a non-starter.
- gostsamo 4y ago
  In regard to biological weapons, they have the issue of being easy to backfire. I'm not mentioning any lab theories related to events from the recent years as an example of possible developments. There are other factors like not being very effective against people in the prime of their health like those serving in an army and the whole thing where nobody internationally likes the guys spreading illiterate viruses which do not respect boarders and kill outside of war zones.
midoridensha 4y ago
>We have plenty of technologies whose development has been stopped via collective agreement: nuclear proliferation, biological weapons, space weapons. Last time I checked, there was no agreement to stop nuclear proliferation at all. What do you think North Korea has been doing all this time? And Russia just decided to tear up one of the arms control treaties it was signatory to. At this point, putting society under the control of AI is probably safer than letting humans continue to run things.
- gostsamo 4y ago
  Oh, I hope that next time you will check this one: https://en.wikipedia.org/wiki/Treaty_on_the_Non-Proliferation_of_Nuclear_Weapons https://en.wikipedia.org/wiki/Treaty_on_the_Non-Proliferatio... Russia had nuclear weapons since 1949 and no matter the treaties, both it and the US have more than enough nukes to burn the planet surface a few times too many. Those treaties are for posturing and they will sign a new one when China decides to gain some international diplomatic points and join the next one.
  - midoridensha 4y ago
    Right, that's a bunch of countries agreeing not to develop nuclear weapons, but it doesn't include all countries. So what's the point? A few countries deciding to flout the treaty and develop their own WMDs means they can just threaten everyone else who followed the treaty. This is exactly what North Korea is doing. Now their neighbors are considering abandoning the treaty and joining the nuclear weapons club. A treaty like this is basically worthless if you can't enforce it.
    - gostsamo 4y ago
      The point is managing risk. The more nuclear powers there are, the more likely is that something goes wrong. And so far we don't have countries leaving NPT in the dozens so I'd say that it is working rather well. Not perfect, but well.

didericis 4y ago

Project Orion is the most tragic (but probably wisest) stopped technology, was in both the space weapon and nuclear proliferation bucket. But it also could have enabled luxuriously spacious trips to Saturn in the 70s had it been realized. I hope we manage to trust each other enough one day to make it a reality.

radlad 4y ago

These are fantastic counterpoints, thanks. Still, I can't help but wonder if they are now out of the public view, but still being developed. (e.g. North Korea nuclear missile tests.) I think you make a fantastic point that the barrier to entry in developing an AI is much lower than building a nuclear bomb.

gostsamo 4y ago
In regard to chemical weapons - no, I think that they are abondonware because they are ineffective. Something similar with bio weapons, they can backfire to your own population rather easily. Nuclear are their special case, but still their proliferation is extremely limited.

oh_sigh 4y ago

You're right that it won't be - the difference is the tools for building those things were not distributed across millions of different people. It was distributed across only a small number of governments.

bloppe 4y ago

"Stopped" is a strong word. Nuclear proliferation and chemical weapons are ongoing and increasing threats. I'd bet a lot of money that we'll see the use of space weapons in our lifetime.

gostsamo 4y ago
In regard to chemical weapons, they are decreasing danger because they are not effective in modern warfare. I'd expect for their usage to decrease. Nuclear is is not something proliferating besides North Korea and even then it is deturance and not an appocaliptic scenario. Space weapons are forbidden by treaty and though some countries are playing close to the read line, I doubt that they will be deployed. They would be political liability while being rather vulnerable in space. Hence hypersonic weapons which work better for the same goals.
- bloppe 4y ago
  Nuclear weapons are proliferating in many places besides North Korea: https://www.npr.org/2023/02/22/1158529106/nuclear-treaty-new-start-putin https://www.npr.org/2023/02/22/1158529106/nuclear-treaty-new... https://abcnews.go.com/Politics/iran-make-material-nuclear-device-12-days-us/story?id=97536371 https://abcnews.go.com/Politics/iran-make-material-nuclear-d... https://news.usni.org/2023/01/24/report-to-congress-on-chinese-nuclear-and-missile-proliferation https://news.usni.org/2023/01/24/report-to-congress-on-chine... Chemical weapons are very effective in modern warfare: https://en.m.wikipedia.org/wiki/Use_of_chemical_weapons_in_the_Syrian_civil_war https://en.m.wikipedia.org/wiki/Use_of_chemical_weapons_in_t... Though, it's more commonly used by non-state actors: https://en.m.wikipedia.org/wiki/Chemical_terrorism https://en.m.wikipedia.org/wiki/Chemical_terrorism
  - gostsamo 4y ago
    Iran is the exception that confirms the rule. They were so heavily sanctioned that it became feasible for them to say "screw it, we will do nukes anyway". I won't say that nuclear proliferation is completely stopped, but the system works for the most part. China and Russia are old nuclear powers and them having nukes is expected. Better counter examples are India and Pakistan, but I don't know the history of their nuclear programs well enough to comment with confidence. The chemical weapons in Syria were old stockpiles and I'd argue that the use there was not an increase of their use, but an opportunity attack. I don't think that there are many groups who currently have any kind of chemical program and can use it systematically.

deleted 4y ago

[deleted]

McDyver 4y ago

I would argue that they didn't proliferate due to high barriers to entry. I'm sure any "rogue" states that at one point would be ruled by extremists, would invest in those if it would give them a competitive advantage. AI can be, or will be at one point, developed at home using consumer devices.

gostsamo 4y ago
In regard to nuclear weapons the high barrier to entry is mostly because the international community made artificial difficulties in obtaining and purifying the materials. Do not forget that the first nuclear weapons were created with tech from the forties in just a few years. Since then the challenges and possible approaches have become better known and if given the chance and will, any modern country can create them if not for the international backlash.
zirgs 4y ago
North Korea is dirt poor, but still managed to create nuclear weapons. The barrier is not that high. If they could do it then countries like Poland or Finland could do that too. They don't, because they have other security guarantees at the moment. If NATO and EU collapsed - the number of nuclear weapons states would definitely increase.
- akiselev 4y ago
  > The barrier is not that high. That ignores the sixty years it took North Korea to develop nuclear weapons. It took the US and USSR less time to develop the first nuclear weapons after the discovery of radioactivity in the late 19th century! The Soviet Union first began training DPRK nuclear scientists back in 1956 [1] and North Korea only managed to finally build nuclear weapons because of assistance from Pakistan with the uranium enrichment process in the late 90s and early 2000s, specifically the design for the centrifuges that fell into Libya's hands [2][3]. [1] https://en.wikipedia.org/wiki/Timeline_of_the_North_Korean_nuclear_program#Phase_I https://en.wikipedia.org/wiki/Timeline_of_the_North_Korean_n... [2] https://en.wikipedia.org/wiki/North_Korea%E2%80%93Pakistan_relations#Allegations_of_nuclear_assistance_by_Pakistan https://en.wikipedia.org/wiki/North_Korea%E2%80%93Pakistan_r... [3] https://en.wikipedia.org/wiki/Abdul_Qadeer_Khan#North_Korea,_Iran,_and_Libya https://en.wikipedia.org/wiki/Abdul_Qadeer_Khan#North_Korea,...

nradov 4y ago

Nuclear and space weapons proliferation have been slowed down but not stopped. The number of nuclear powers is larger now than ever before. While no one has launched kinetic weapons into orbit lately, the superpowers are currently engaged in an anti-satellite weapon arms race.

gostsamo 4y ago
The anti-satellite weapons are ground-based and are part of the playing close to the read line together with the spy satellites. The space weapons are when you have a nuclear head deployed in space with a few seconds to minutes ability to reach anywhere on Earth.
- nradov 4y ago
  You seem to have some misunderstandings about orbital dynamics and nuclear weapons. Placing nuclear weapons in orbit wouldn't allow for striking anywhere on Earth within a "few seconds to minutes" unless you had an absolutely enormous constellation that would be unaffordable even for superpowers. You are forgetting to account for orbital parameter changes and then re-entry. In most realistic circumstances, a ground or sea based ICBM would be able to strike most points on Earth faster than a space weapon. Do the math on this, or you can try to simulate it in something like KSP. Nuclear warheads also can't be stored for more than a few years and still be expected to work reliably. They require periodic intensive inspection and maintenance. Even though space launch costs are coming down there is still no feasible way to conduct such maintenance in orbit.
  - gostsamo 4y ago
    Check some of the old cold war designs, they definitely thought in that direction. Space weapons were prohibited partly for the reasons you mentioned and partly due to the political liability of militarizing the space.

stcredzero 4y ago

Another way to put it: As technology progresses, the amount of stupidity to make an "oopsie" exponentially decreases while the size of the potential "oopsies" exponentially increases. Eventually, one gets to the point where even super geniuses are very likely to create a civilization ending "oopsie" in any given year. This seems to be where we are quickly heading with AI.

mdgrech23 4y ago

Stopped is an interesting word here. A lot of the countries that agreed not to further develop nukes and or hold nukes are either a) lying or b) we (the USA) not provide their defense to our detriment meanwhile the real bad guys who we really need to be concerned about still have plenty of nukes.

alach11 4y ago

The supplies to create biological weapons are actually surprisingly cheap! Not as cheap as graphics cards though. To stop AI development would require pretty extreme restrictions on computing resources. It's really hard to imagine that working without having massive negative knock-on effects on other fields/industries. The economic pressure alone seems to make an effective "anti-AI" policy a non-starter.

gostsamo 4y ago
In regard to biological weapons, they have the issue of being easy to backfire. I'm not mentioning any lab theories related to events from the recent years as an example of possible developments. There are other factors like not being very effective against people in the prime of their health like those serving in an army and the whole thing where nobody internationally likes the guys spreading illiterate viruses which do not respect boarders and kill outside of war zones.

midoridensha 4y ago

>We have plenty of technologies whose development has been stopped via collective agreement: nuclear proliferation, biological weapons, space weapons. Last time I checked, there was no agreement to stop nuclear proliferation at all. What do you think North Korea has been doing all this time? And Russia just decided to tear up one of the arms control treaties it was signatory to. At this point, putting society under the control of AI is probably safer than letting humans continue to run things.

gostsamo 4y ago
Oh, I hope that next time you will check this one: https://en.wikipedia.org/wiki/Treaty_on_the_Non-Proliferation_of_Nuclear_Weapons https://en.wikipedia.org/wiki/Treaty_on_the_Non-Proliferatio... Russia had nuclear weapons since 1949 and no matter the treaties, both it and the US have more than enough nukes to burn the planet surface a few times too many. Those treaties are for posturing and they will sign a new one when China decides to gain some international diplomatic points and join the next one.
- midoridensha 4y ago
  Right, that's a bunch of countries agreeing not to develop nuclear weapons, but it doesn't include all countries. So what's the point? A few countries deciding to flout the treaty and develop their own WMDs means they can just threaten everyone else who followed the treaty. This is exactly what North Korea is doing. Now their neighbors are considering abandoning the treaty and joining the nuclear weapons club. A treaty like this is basically worthless if you can't enforce it.
  - gostsamo 4y ago
    The point is managing risk. The more nuclear powers there are, the more likely is that something goes wrong. And so far we don't have countries leaving NPT in the dozens so I'd say that it is working rather well. Not perfect, but well.