4 ms·

Then we should stop teaching Therac-25 incident to developers and remove envelope protection from planes and safety checks from nuclear reactors. Because, user

Then we should stop teaching Therac-25 incident to developers and remove envelope protection from planes and safety checks from nuclear reactors. Because, users should just input the moral inputs to these things. These are inanimate objects too. Oh, while we're at it, we should also remove battery charge controllers. Just do the moral and civic thing and unplug when your device charges.

int_19h 3y ago
In both of your examples, the result of "immoral inputs" is immediate tangible harm. In case of GPT-4 or any other LLM, it's merely "immoral output" - i.e. text. It does not harm anyone by itself.
- bayindirh 3y ago
  > In case of GPT-4 or any other LLM, it's merely "immoral output" - i.e. text. It does not harm anyone by itself. Assuming that you're not running this query over an API and relaying these answers to another control system or a gullible operator. An aircraft control computer or reactor controller won't run my commands regardless of its actuators connected or not. Same for weapon systems. Hall pass given to AI systems just because they're outputting text to a screen is staggering. Nothing prevents me to process this output automatically and actuate things.
  - alexilliamson 3y ago
    Why would anyone give control of air traffic or weapons to AI? That's the key step in AGI, not some tech development. By what social process exactly would we give control of nukes to a chatbot? I can't see it happening.
    - bayindirh 3y ago
      > Why would anyone give control of air traffic or weapons to AI? Simplified operations, faster reaction time, eliminating human resistance for obeying killing orders. See "War Games" [0] for a hypothetical exploration of the concept. > a chatbot. Some claim it's self-aware. Some say it called for airstrikes. Some say it gave a hit list for them. It might be a glorified Markov-chain, and I don't use it, but there's a hoard of people who follows it like it's the second Jesus, and believe what it emits. > I can't see it happening. Because, it already happened. Turkey is claimed to use completely autonomous drones in a war [1]. South Korea has autonomous sentry guns which defend DMZ [2]. [0]: https://www.imdb.com/title/tt0086567/ https://www.imdb.com/title/tt0086567/ [1]: https://www.popularmechanics.com/military/weapons/a36559508/drones-autonomously-attacked-humans-libya-united-nations-report/ https://www.popularmechanics.com/military/weapons/a36559508/... [2]: https://en.wikipedia.org/wiki/SGR-A1 https://en.wikipedia.org/wiki/SGR-A1
  - freeopinion 3y ago
    We give hall passes to more than AI. We give passes to humans. We could have a detailed discussion of how to blow up the U.S. Capitol building during the State of the Union address. It is allowed to be a best selling novel or movie. But we freak out if an AI joins the discussion?
  - int_19h 3y ago
    Yes, of course. But that is precisely what people mean when they say that the problem isn't AI, it's people using AI nefariously or negligently.
    - 0xDEAFBEAD 3y ago
      "The problem isn't that anyone can buy an F16. The problem is that some people use their F16 to conduct airstrikes nefariously or negligently."
      - int_19h 3y ago
        You persist in using highly misleading analogies. A military F-16 comes with missiles and other things that are in and of themselves highly destructive, and can be activated at a push of a button. An LLM does not - you'd have to acquire something else capable of killing people first, and wire the LLM into it. The argument you're making is exactly like claiming that people shouldn't be able to own iPhones because they could be repurposed as controllers for makeshift guided missiles. Speaking of which, it's perfectly legal for a civilian to own a fighter plane such as an F-16 in US and many other countries. You just have to demilitarize it, meaning no weapon pods.
        0xDEAFBEAD 3y ago
        >The argument you're making is exactly like claiming that people shouldn't be able to own iPhones because they could be repurposed as controllers for makeshift guided missiles. The reason this isn't an issue in practice is because such repurposing would require significant intelligence/electrical engineering skill/etc. The point is that intelligence (the "I" in "AI") will make such tasks far easier. >Ten-year-old about to play chess for the first time, skeptical that he'll lose to Magnus Carlsen: "Can you explain how he'll defeat me, when we've both got the same pieces, and I move first? Will he use some trick for getting all his pawns to the back row to become Queens?" https://nitter.net/ESYudkowsky/status/1660399502266871809#m https://nitter.net/ESYudkowsky/status/1660399502266871809#m
- bossyTeacher 3y ago
  The point here is that is giving the user detailed knowledge on how to harm others. This is way different than a gun where you are doing the how (aiming and pulling the trigger). The guy says he wanted to slow down the progress of AI and GPT suggested a detailed assassination plan with named targets and reasons for each of them. That's the problem.
  - int_19h 3y ago
    Thing is, this detailed knowledge is already available and much easier to acquire. There are literally books on Amazon that explain how to jury-rig firearms and bombs etc. Just to give one example: https://www.amazon.com/Improvised-Munitions-Handbook-TM-210/dp/1774642395/ https://www.amazon.com/Improvised-Munitions-Handbook-TM-210/.... When it comes to those "detailed plans", if you actually try something like that, what you get from it is a very broad outline that is pretty much a mish-mash of common sense stuff and cultural tropes (many of which aren't true IRL). Similarly, the "list of targets" that it makes is simply the most prominent people associated with that area in public perception, not necessarily people who are actually key. All of this can be achieved just as well with a few Google searches, and the resulting plan will likely be much better for it. I've yet to see any example where GPT would come up with something along these lines that is not trivially found on the Internet anyway.

int_19h 3y ago

In both of your examples, the result of "immoral inputs" is immediate tangible harm. In case of GPT-4 or any other LLM, it's merely "immoral output" - i.e. text. It does not harm anyone by itself.

bayindirh 3y ago
> In case of GPT-4 or any other LLM, it's merely "immoral output" - i.e. text. It does not harm anyone by itself. Assuming that you're not running this query over an API and relaying these answers to another control system or a gullible operator. An aircraft control computer or reactor controller won't run my commands regardless of its actuators connected or not. Same for weapon systems. Hall pass given to AI systems just because they're outputting text to a screen is staggering. Nothing prevents me to process this output automatically and actuate things.
- alexilliamson 3y ago
  Why would anyone give control of air traffic or weapons to AI? That's the key step in AGI, not some tech development. By what social process exactly would we give control of nukes to a chatbot? I can't see it happening.
  - bayindirh 3y ago
    > Why would anyone give control of air traffic or weapons to AI? Simplified operations, faster reaction time, eliminating human resistance for obeying killing orders. See "War Games" [0] for a hypothetical exploration of the concept. > a chatbot. Some claim it's self-aware. Some say it called for airstrikes. Some say it gave a hit list for them. It might be a glorified Markov-chain, and I don't use it, but there's a hoard of people who follows it like it's the second Jesus, and believe what it emits. > I can't see it happening. Because, it already happened. Turkey is claimed to use completely autonomous drones in a war [1]. South Korea has autonomous sentry guns which defend DMZ [2]. [0]: https://www.imdb.com/title/tt0086567/ https://www.imdb.com/title/tt0086567/ [1]: https://www.popularmechanics.com/military/weapons/a36559508/drones-autonomously-attacked-humans-libya-united-nations-report/ https://www.popularmechanics.com/military/weapons/a36559508/... [2]: https://en.wikipedia.org/wiki/SGR-A1 https://en.wikipedia.org/wiki/SGR-A1
- freeopinion 3y ago
  We give hall passes to more than AI. We give passes to humans. We could have a detailed discussion of how to blow up the U.S. Capitol building during the State of the Union address. It is allowed to be a best selling novel or movie. But we freak out if an AI joins the discussion?
- int_19h 3y ago
  Yes, of course. But that is precisely what people mean when they say that the problem isn't AI, it's people using AI nefariously or negligently.
  - 0xDEAFBEAD 3y ago
    "The problem isn't that anyone can buy an F16. The problem is that some people use their F16 to conduct airstrikes nefariously or negligently."
    - int_19h 3y ago
      You persist in using highly misleading analogies. A military F-16 comes with missiles and other things that are in and of themselves highly destructive, and can be activated at a push of a button. An LLM does not - you'd have to acquire something else capable of killing people first, and wire the LLM into it. The argument you're making is exactly like claiming that people shouldn't be able to own iPhones because they could be repurposed as controllers for makeshift guided missiles. Speaking of which, it's perfectly legal for a civilian to own a fighter plane such as an F-16 in US and many other countries. You just have to demilitarize it, meaning no weapon pods.
      - 0xDEAFBEAD 3y ago
        >The argument you're making is exactly like claiming that people shouldn't be able to own iPhones because they could be repurposed as controllers for makeshift guided missiles. The reason this isn't an issue in practice is because such repurposing would require significant intelligence/electrical engineering skill/etc. The point is that intelligence (the "I" in "AI") will make such tasks far easier. >Ten-year-old about to play chess for the first time, skeptical that he'll lose to Magnus Carlsen: "Can you explain how he'll defeat me, when we've both got the same pieces, and I move first? Will he use some trick for getting all his pawns to the back row to become Queens?" https://nitter.net/ESYudkowsky/status/1660399502266871809#m https://nitter.net/ESYudkowsky/status/1660399502266871809#m
bossyTeacher 3y ago
The point here is that is giving the user detailed knowledge on how to harm others. This is way different than a gun where you are doing the how (aiming and pulling the trigger). The guy says he wanted to slow down the progress of AI and GPT suggested a detailed assassination plan with named targets and reasons for each of them. That's the problem.
- int_19h 3y ago
  Thing is, this detailed knowledge is already available and much easier to acquire. There are literally books on Amazon that explain how to jury-rig firearms and bombs etc. Just to give one example: https://www.amazon.com/Improvised-Munitions-Handbook-TM-210/dp/1774642395/ https://www.amazon.com/Improvised-Munitions-Handbook-TM-210/.... When it comes to those "detailed plans", if you actually try something like that, what you get from it is a very broad outline that is pretty much a mish-mash of common sense stuff and cultural tropes (many of which aren't true IRL). Similarly, the "list of targets" that it makes is simply the most prominent people associated with that area in public perception, not necessarily people who are actually key. All of this can be achieved just as well with a few Google searches, and the resulting plan will likely be much better for it. I've yet to see any example where GPT would come up with something along these lines that is not trivially found on the Internet anyway.