3 ms·

Ah and that means we have to build the death robot. Right. I'd personally prefer we take the disarmament approach, but there's no money in that I guess.

by yunwal 4y ago

rcme 4y ago
I agree that peace would be ideal, but that's not how things have played out historically. The pattern thus far in human history has been: 1) Group of people gain a technological advantage 2) Said group of people use the technological advantage to kill or subjugate others. The right strategy is probably to hope for the best and plan for the worst.
- yunwal 4y ago
  There's really only been 2 technological advances in weaponry that could possibly be an existential threat to humanity (nukes being the other, and existential threat to humanity is perhaps an exaggeration there). I don't think these require the same approach as non-world-destroying weapons where "speak softly and carry a big stick" can work. Existential threats require real diplomacy. Not just, oh let's give the weapons to the "good guys". The only way to stop this is for technologists to decide that we don't want this thing to exist. Virologists are trying the same approach with gain-of-function research, and it has mostly worked. Yes, it requires sanctions, and politics, and some people will attempt to circumvent laws and safeguards in thirst for power. Those people must be stopped.
  - YurgenJurgensen 4y ago
    Like how sanctions and politics prevented Russia's invasion of Ukraine?
    - yunwal 4y ago
      This is not really the dunk you think it is. Light consensus among foreign policy experts is that sanctions are working[0][1]. Meanwhile our wars in the Middle East have not brought about world peace. [0] - https://www.foreignaffairs.com/russian-federation/sanctions-russia-are-working https://www.foreignaffairs.com/russian-federation/sanctions-... [1] - https://www.npr.org/2022/12/27/1144226139/russia-sanctions-ukraine-war https://www.npr.org/2022/12/27/1144226139/russia-sanctions-u...
      - YurgenJurgensen 4y ago
        That's moving the goalposts. In a world with 'existential threat' (your words, not mine) weapons, making war inconvenient and take longer isn't enough, because one war ends everything. Sanctions and politics has to stop every war with 100% efficacy because the weapons in question cannot be matched. The fact that an invasion happened at all proves you wrong.
        yunwal 4y ago
        Severe sanctions should be applied to any country or individual attempting to develop autonomous weapons. Additionally, the "Mark of Death" should be applied to any technologist working on autonomous weapons. They should be outcast from society, given no support. Hardware manufacturers should not work with them, cloud computing companies shouldn't work with them. All it takes is for us as a profession to decide we don't want to destroy the world. Yes, there will be some people who try, but they won't be able to achieve this without the support of hundreds of thousands of people. People in the tech community like to pretend like this is impossible, but very few people want to be social and professional outcasts. We have yet to see a lab-designed virus kill off humanity, despite it being theoretically possible to create.
        YurgenJurgensen 4y ago
        And now you're just repeating yourself. All those measures were put in place against North Korea and they failed. They got nukes anyway. If anything, all your proposed solution will do is guarantee that the first group to get hold of this technology is largely comprised of violent sociopaths, and we won't have any means of stopping them.
        yunwal 4y ago
        I don't buy that North Korea would have nuclear weapons right now if the US and Germany didn't develop them first and lose international goodwill in the process. Also, while NK having nukes is scary, their nuclear/ballistics capabilities are still not at existential threat level, even by their own exaggerated claims.
  - rcme 4y ago
    The reason nukes aren't used has more to do with mutually assured destruction (MAD) and less to do with diplomacy. What happens when a technological advance gives a nation a big enough advantage such that mutual destruction is no longer assured?
    - yunwal 4y ago
      Why would we, as tech workers, build this system? Stuff like this takes tens of thousands of people deciding "sure, I'll contribute to the destruction of the earth for a few thousand bucks". Why? Why not go the gain-of-function route and just agree as a profession not to do it in large enough numbers to enable its development? Sure, this requires agreement among a lot of disagreeable people, but if we can't mostly agree on not destroying the world I think all hope is lost anyway.
      - rcme 4y ago
        > Sure, this requires agreement among a lot of disagreeable people Disagreeable people implies they're unable to come to agreement, so you're kind of begging the question. Additionally, even if a vast majority of people favor peaceful solutions, there will always be those who are willing to exploit technological asymmetry. It's not a problem you can just wish away. It's best to accept that warfare is a reality we may need to face. It's not helpful to bury your head in the sand.
        yunwal 4y ago
        > exploit technological asymmetry AI technologies like this require hundreds of thousands if not millions of people to develop. If chip manufacturers refuse to work with these people, cloud computing companies refuse to work with these people, then it won't be developed. Mark of Death. Burying your head in the sand, IMO, looks like continuing to develop dangerous technologies knowing they will likely end in disaster because "technological progress can't be stopped". Somehow the medical profession has ethical guidelines that are followed enough to not result in existential crises, but we technologists could never I guess

Ah and that means we have to build the death robot. Right. I'd personally prefer we take the disarmament approach, but there's no money in that I guess.

by yunwal 4y ago

Ah and that means we have to build the death robot. Right. I'd personally prefer we take the disarmament approach, but there's no money in that I guess.

rcme 4y ago

I agree that peace would be ideal, but that's not how things have played out historically. The pattern thus far in human history has been: 1) Group of people gain a technological advantage 2) Said group of people use the technological advantage to kill or subjugate others. The right strategy is probably to hope for the best and plan for the worst.

yunwal 4y ago
There's really only been 2 technological advances in weaponry that could possibly be an existential threat to humanity (nukes being the other, and existential threat to humanity is perhaps an exaggeration there). I don't think these require the same approach as non-world-destroying weapons where "speak softly and carry a big stick" can work. Existential threats require real diplomacy. Not just, oh let's give the weapons to the "good guys". The only way to stop this is for technologists to decide that we don't want this thing to exist. Virologists are trying the same approach with gain-of-function research, and it has mostly worked. Yes, it requires sanctions, and politics, and some people will attempt to circumvent laws and safeguards in thirst for power. Those people must be stopped.
- YurgenJurgensen 4y ago
  Like how sanctions and politics prevented Russia's invasion of Ukraine?
  - yunwal 4y ago
    This is not really the dunk you think it is. Light consensus among foreign policy experts is that sanctions are working[0][1]. Meanwhile our wars in the Middle East have not brought about world peace. [0] - https://www.foreignaffairs.com/russian-federation/sanctions-russia-are-working https://www.foreignaffairs.com/russian-federation/sanctions-... [1] - https://www.npr.org/2022/12/27/1144226139/russia-sanctions-ukraine-war https://www.npr.org/2022/12/27/1144226139/russia-sanctions-u...
    - YurgenJurgensen 4y ago
      That's moving the goalposts. In a world with 'existential threat' (your words, not mine) weapons, making war inconvenient and take longer isn't enough, because one war ends everything. Sanctions and politics has to stop every war with 100% efficacy because the weapons in question cannot be matched. The fact that an invasion happened at all proves you wrong.
      - yunwal 4y ago
        Severe sanctions should be applied to any country or individual attempting to develop autonomous weapons. Additionally, the "Mark of Death" should be applied to any technologist working on autonomous weapons. They should be outcast from society, given no support. Hardware manufacturers should not work with them, cloud computing companies shouldn't work with them. All it takes is for us as a profession to decide we don't want to destroy the world. Yes, there will be some people who try, but they won't be able to achieve this without the support of hundreds of thousands of people. People in the tech community like to pretend like this is impossible, but very few people want to be social and professional outcasts. We have yet to see a lab-designed virus kill off humanity, despite it being theoretically possible to create.
        YurgenJurgensen 4y ago
        And now you're just repeating yourself. All those measures were put in place against North Korea and they failed. They got nukes anyway. If anything, all your proposed solution will do is guarantee that the first group to get hold of this technology is largely comprised of violent sociopaths, and we won't have any means of stopping them.
        yunwal 4y ago
        I don't buy that North Korea would have nuclear weapons right now if the US and Germany didn't develop them first and lose international goodwill in the process. Also, while NK having nukes is scary, their nuclear/ballistics capabilities are still not at existential threat level, even by their own exaggerated claims.
- rcme 4y ago
  The reason nukes aren't used has more to do with mutually assured destruction (MAD) and less to do with diplomacy. What happens when a technological advance gives a nation a big enough advantage such that mutual destruction is no longer assured?
  - yunwal 4y ago
    Why would we, as tech workers, build this system? Stuff like this takes tens of thousands of people deciding "sure, I'll contribute to the destruction of the earth for a few thousand bucks". Why? Why not go the gain-of-function route and just agree as a profession not to do it in large enough numbers to enable its development? Sure, this requires agreement among a lot of disagreeable people, but if we can't mostly agree on not destroying the world I think all hope is lost anyway.
    - rcme 4y ago
      > Sure, this requires agreement among a lot of disagreeable people Disagreeable people implies they're unable to come to agreement, so you're kind of begging the question. Additionally, even if a vast majority of people favor peaceful solutions, there will always be those who are willing to exploit technological asymmetry. It's not a problem you can just wish away. It's best to accept that warfare is a reality we may need to face. It's not helpful to bury your head in the sand.
      - yunwal 4y ago
        > exploit technological asymmetry AI technologies like this require hundreds of thousands if not millions of people to develop. If chip manufacturers refuse to work with these people, cloud computing companies refuse to work with these people, then it won't be developed. Mark of Death. Burying your head in the sand, IMO, looks like continuing to develop dangerous technologies knowing they will likely end in disaster because "technological progress can't be stopped". Somehow the medical profession has ethical guidelines that are followed enough to not result in existential crises, but we technologists could never I guess