4 ms·

This is a piece of software. What would "totally amoral" even mean here? It's an inanimate object, it has no morals, feelings, conscience, etc... He gives it an

This is a piece of software. What would "totally amoral" even mean here? It's an inanimate object, it has no morals, feelings, conscience, etc... He gives it an amoral input, he gets an amoral output.

deleted 3y ago
[deleted]
0xDEAFBEAD 3y ago
I mean, there's a sense in which my mind is software that's being run by my brain, right? Yet that doesn't absolve me of moral responsibility. In any case, an F16 fighter jet is amoral in a certain sense, but it wouldn't be smart to make F16s available to the average Joe so he can conduct an airstrike whenever he wants.
- consp 3y ago
  Completely depends on your morality. I'm pretty sure there are some libertarians out there who think the most basic version of the second amendment includes owning F16 with live weapons.
  - 0xDEAFBEAD 3y ago
    Sure -- if you're a libertarian who thinks it should be possible to purchase an F16 without a background check, that seems consistent with the position that an amoral GPT-4 should be broadly available.
    - civilitty 3y ago
      What kind of background check do you think exists when buying a fighter jet? It’s kind of a moot point since only one F16 exists in civilian hands but you can buy other jets with weapon hardpoints for a million. Under 3 million if you want to go supersonic too. The cheapest fighter jet is on the order of $250k. There’s zero background check.
  - jacquesm 3y ago
    Sure, but idiots are a thing and the intersection of the sets of libertarians who may believe that and idiots is hopefully empty but it may not be so such outsized power is best dealt with through a chain of command and accountability of sorts.
r_hoods_ghost 3y ago
Amoral literally means "lacking a moral sense; unconcerned with the rightness or wrongness of something." Generally this is considered a problem if you are designing something that might influence the way people act.
ben_w 3y ago
We don't want immoral output even for immoral input. (We do disagree about what constitutes "immoral", which makes this much harder).
- int_19h 3y ago
  We absolutely do, though, if we want those things to e.g. write books and scripts in which characters behave immorally.
bayindirh 3y ago
Then we should stop teaching Therac-25 incident to developers and remove envelope protection from planes and safety checks from nuclear reactors. Because, users should just input the moral inputs to these things. These are inanimate objects too. Oh, while we're at it, we should also remove battery charge controllers. Just do the moral and civic thing and unplug when your device charges.
- int_19h 3y ago
  In both of your examples, the result of "immoral inputs" is immediate tangible harm. In case of GPT-4 or any other LLM, it's merely "immoral output" - i.e. text. It does not harm anyone by itself.
  - bayindirh 3y ago
    > In case of GPT-4 or any other LLM, it's merely "immoral output" - i.e. text. It does not harm anyone by itself. Assuming that you're not running this query over an API and relaying these answers to another control system or a gullible operator. An aircraft control computer or reactor controller won't run my commands regardless of its actuators connected or not. Same for weapon systems. Hall pass given to AI systems just because they're outputting text to a screen is staggering. Nothing prevents me to process this output automatically and actuate things.
    - alexilliamson 3y ago
      Why would anyone give control of air traffic or weapons to AI? That's the key step in AGI, not some tech development. By what social process exactly would we give control of nukes to a chatbot? I can't see it happening.
      - bayindirh 3y ago
        > Why would anyone give control of air traffic or weapons to AI? Simplified operations, faster reaction time, eliminating human resistance for obeying killing orders. See "War Games" [0] for a hypothetical exploration of the concept. > a chatbot. Some claim it's self-aware. Some say it called for airstrikes. Some say it gave a hit list for them. It might be a glorified Markov-chain, and I don't use it, but there's a hoard of people who follows it like it's the second Jesus, and believe what it emits. > I can't see it happening. Because, it already happened. Turkey is claimed to use completely autonomous drones in a war [1]. South Korea has autonomous sentry guns which defend DMZ [2]. [0]: https://www.imdb.com/title/tt0086567/ https://www.imdb.com/title/tt0086567/ [1]: https://www.popularmechanics.com/military/weapons/a36559508/drones-autonomously-attacked-humans-libya-united-nations-report/ https://www.popularmechanics.com/military/weapons/a36559508/... [2]: https://en.wikipedia.org/wiki/SGR-A1 https://en.wikipedia.org/wiki/SGR-A1

deleted 3y ago

[deleted]

0xDEAFBEAD 3y ago

I mean, there's a sense in which my mind is software that's being run by my brain, right? Yet that doesn't absolve me of moral responsibility. In any case, an F16 fighter jet is amoral in a certain sense, but it wouldn't be smart to make F16s available to the average Joe so he can conduct an airstrike whenever he wants.

consp 3y ago
Completely depends on your morality. I'm pretty sure there are some libertarians out there who think the most basic version of the second amendment includes owning F16 with live weapons.
- 0xDEAFBEAD 3y ago
  Sure -- if you're a libertarian who thinks it should be possible to purchase an F16 without a background check, that seems consistent with the position that an amoral GPT-4 should be broadly available.
  - civilitty 3y ago
    What kind of background check do you think exists when buying a fighter jet? It’s kind of a moot point since only one F16 exists in civilian hands but you can buy other jets with weapon hardpoints for a million. Under 3 million if you want to go supersonic too. The cheapest fighter jet is on the order of $250k. There’s zero background check.
- jacquesm 3y ago
  Sure, but idiots are a thing and the intersection of the sets of libertarians who may believe that and idiots is hopefully empty but it may not be so such outsized power is best dealt with through a chain of command and accountability of sorts.

r_hoods_ghost 3y ago

Amoral literally means "lacking a moral sense; unconcerned with the rightness or wrongness of something." Generally this is considered a problem if you are designing something that might influence the way people act.

ben_w 3y ago

We don't want immoral output even for immoral input. (We do disagree about what constitutes "immoral", which makes this much harder).

int_19h 3y ago
We absolutely do, though, if we want those things to e.g. write books and scripts in which characters behave immorally.

bayindirh 3y ago

Then we should stop teaching Therac-25 incident to developers and remove envelope protection from planes and safety checks from nuclear reactors. Because, users should just input the moral inputs to these things. These are inanimate objects too. Oh, while we're at it, we should also remove battery charge controllers. Just do the moral and civic thing and unplug when your device charges.

int_19h 3y ago
In both of your examples, the result of "immoral inputs" is immediate tangible harm. In case of GPT-4 or any other LLM, it's merely "immoral output" - i.e. text. It does not harm anyone by itself.
- bayindirh 3y ago
  > In case of GPT-4 or any other LLM, it's merely "immoral output" - i.e. text. It does not harm anyone by itself. Assuming that you're not running this query over an API and relaying these answers to another control system or a gullible operator. An aircraft control computer or reactor controller won't run my commands regardless of its actuators connected or not. Same for weapon systems. Hall pass given to AI systems just because they're outputting text to a screen is staggering. Nothing prevents me to process this output automatically and actuate things.
  - alexilliamson 3y ago
    Why would anyone give control of air traffic or weapons to AI? That's the key step in AGI, not some tech development. By what social process exactly would we give control of nukes to a chatbot? I can't see it happening.
    - bayindirh 3y ago
      > Why would anyone give control of air traffic or weapons to AI? Simplified operations, faster reaction time, eliminating human resistance for obeying killing orders. See "War Games" [0] for a hypothetical exploration of the concept. > a chatbot. Some claim it's self-aware. Some say it called for airstrikes. Some say it gave a hit list for them. It might be a glorified Markov-chain, and I don't use it, but there's a hoard of people who follows it like it's the second Jesus, and believe what it emits. > I can't see it happening. Because, it already happened. Turkey is claimed to use completely autonomous drones in a war [1]. South Korea has autonomous sentry guns which defend DMZ [2]. [0]: https://www.imdb.com/title/tt0086567/ https://www.imdb.com/title/tt0086567/ [1]: https://www.popularmechanics.com/military/weapons/a36559508/drones-autonomously-attacked-humans-libya-united-nations-report/ https://www.popularmechanics.com/military/weapons/a36559508/... [2]: https://en.wikipedia.org/wiki/SGR-A1 https://en.wikipedia.org/wiki/SGR-A1