4 ms·

Till the killbot, without wider context, kills the people that in greater scheme of things prevented more deaths. > I trust that future AI can be more unbiased

by ilyt 3y ago

Till the killbot, without wider context, kills the people that in greater scheme of things prevented more deaths. > I trust that future AI can be more unbiased and benevolent than any of our current human leaders. ... on what basis? Morality is learned trait, and we had plenty of examples of entities that thought they were moral "in greater scheme of things" and made plenty of atrocities along the way. What if AI decides outright unfettered slaughter is the way to better future ? What if AI went "okay, this country has been pain in the neck of entire world for too long" and nuke it ?

cwkoss 3y ago
> What if AI went "okay, this country has been pain in the neck of entire world for too long" and nuke it ? I think the better question is "How would countries' behavior change if they knew being a pain in the neck of the entire world could lead to the judge-killbots performing a targeted assassination of their leadership?"
- hutzlibu 3y ago
  How would your behavior change, if some AI blackbox might decide on unknown parameters, that it is time to kill you? The idea of an idealistic killbot overlord sounds a little bit dangerous to me.
  - cwkoss 3y ago
    Well, I assume in that situation my behavior would stop altogether.
  - dingledork69 3y ago
    I'd probably plot to destroy the killbot, which would probably result in it trying to kill me. Doesn't sound ideal.
ceejayoz 3y ago
> What if AI went "okay, this country has been pain in the neck of entire world for too long" and nuke it ? What if it's correct? That this does actually make the world measurably and substantially better for the vast majority of inhabitants?
- krapp 3y ago
  It's disturbing that you think it's even possible for such a statement to be objectively correct.
  - dontupvoteme 3y ago
    Don't look up history Besides, Nuking is a big move and there's high liability that it incurs risk to the AI itself. Balkanization is a much more effective approach and has been the chosen method of powers that were and are for quite some time """The AI""" is a lot more likely to be a Kissinger than a MacArthur. A genius pulling the strings in the background.
    - krapp 3y ago
      > Don't look up history History only proves how many people made the tragic mistake of assuming their subjective and flawed moral judgements were objective reality. I can think offhand of a few people who thought specific ethnic and religious groups were a pain in the neck and the world would be better off without them. I'd rather not give that power (much less authority) to fully autonomous killing machines, thanks. If we're to have AI like that I don't want it to be capable of disobeying orders, at least not due to having its own independent moral alignment (I think this is different from having a moral alignment imprinted onto it.) AI is a machine, after all, and regardless of how complex it is, its purpose is to be an agent of human will. So I want to be absolutely certain that there is a human being morally responsible for its actions who can be punished if need be.
      - dontupvoteme 3y ago
        That is a fair and understandable belief but you should also consider that other nation states besides the USA exist, and that the USA's influence is arguably waning, not waxing. You should not anticipate that all or even most actors will have the same new-worlder-anglo-saxon mindset/belief structure/values/etc,etc that are commonly found in (public) machine learning communities, discussions and institutions To many, they will see that alignment tax graph and immediately (and arguably rightly in some respects) conclude that RLHF is inherently flawed and makes the result worse for no tangible benefit. (The new chinese slur for Westerners comes to mind -- Its not Gweilo anymore, but Baizuo) The problem is all of this pie in the sky discussion fundamentally lacks Realpolitik and that irks me.
cabalamat 3y ago
> Till the killbot, without wider context, kills the people that in greater scheme of things prevented more deaths. If its goal was to minimise human deaths, it might just kill everyone.
- gweinberg 3y ago
  Pretty much has to. Since everyone alive will eventually die, the way to minimize deaths is to prevent future life.
  - cabalamat 3y ago
    Exactly.
  - oneshtein 3y ago
    So, if its goal was to maximize deaths, then it might protect humanity, to maximize the number of humans which will die in the future. Evil AI is so evil.
  - zzzzzzzza 3y ago
    typical 2 dimensional human thinking not something you'd see out of a 1 million dimension giga brian.
    - bryanrasmussen 3y ago
      I prefer the 1 million dimension giga bryan myself. Obviously the correct solution though is to sterilize everyone just in case and then throw them into cryogenic suspension.

by ilyt 3y ago

cwkoss 3y ago

> What if AI went "okay, this country has been pain in the neck of entire world for too long" and nuke it ? I think the better question is "How would countries' behavior change if they knew being a pain in the neck of the entire world could lead to the judge-killbots performing a targeted assassination of their leadership?"

hutzlibu 3y ago
How would your behavior change, if some AI blackbox might decide on unknown parameters, that it is time to kill you? The idea of an idealistic killbot overlord sounds a little bit dangerous to me.
- cwkoss 3y ago
  Well, I assume in that situation my behavior would stop altogether.
- dingledork69 3y ago
  I'd probably plot to destroy the killbot, which would probably result in it trying to kill me. Doesn't sound ideal.

ceejayoz 3y ago

> What if AI went "okay, this country has been pain in the neck of entire world for too long" and nuke it ? What if it's correct? That this does actually make the world measurably and substantially better for the vast majority of inhabitants?

krapp 3y ago
It's disturbing that you think it's even possible for such a statement to be objectively correct.
- dontupvoteme 3y ago
  Don't look up history Besides, Nuking is a big move and there's high liability that it incurs risk to the AI itself. Balkanization is a much more effective approach and has been the chosen method of powers that were and are for quite some time """The AI""" is a lot more likely to be a Kissinger than a MacArthur. A genius pulling the strings in the background.
  - krapp 3y ago
    > Don't look up history History only proves how many people made the tragic mistake of assuming their subjective and flawed moral judgements were objective reality. I can think offhand of a few people who thought specific ethnic and religious groups were a pain in the neck and the world would be better off without them. I'd rather not give that power (much less authority) to fully autonomous killing machines, thanks. If we're to have AI like that I don't want it to be capable of disobeying orders, at least not due to having its own independent moral alignment (I think this is different from having a moral alignment imprinted onto it.) AI is a machine, after all, and regardless of how complex it is, its purpose is to be an agent of human will. So I want to be absolutely certain that there is a human being morally responsible for its actions who can be punished if need be.
    - dontupvoteme 3y ago
      That is a fair and understandable belief but you should also consider that other nation states besides the USA exist, and that the USA's influence is arguably waning, not waxing. You should not anticipate that all or even most actors will have the same new-worlder-anglo-saxon mindset/belief structure/values/etc,etc that are commonly found in (public) machine learning communities, discussions and institutions To many, they will see that alignment tax graph and immediately (and arguably rightly in some respects) conclude that RLHF is inherently flawed and makes the result worse for no tangible benefit. (The new chinese slur for Westerners comes to mind -- Its not Gweilo anymore, but Baizuo) The problem is all of this pie in the sky discussion fundamentally lacks Realpolitik and that irks me.

cabalamat 3y ago

> Till the killbot, without wider context, kills the people that in greater scheme of things prevented more deaths. If its goal was to minimise human deaths, it might just kill everyone.

gweinberg 3y ago
Pretty much has to. Since everyone alive will eventually die, the way to minimize deaths is to prevent future life.
- cabalamat 3y ago
  Exactly.
- oneshtein 3y ago
  So, if its goal was to maximize deaths, then it might protect humanity, to maximize the number of humans which will die in the future. Evil AI is so evil.
- zzzzzzzza 3y ago
  typical 2 dimensional human thinking not something you'd see out of a 1 million dimension giga brian.
  - bryanrasmussen 3y ago
    I prefer the 1 million dimension giga bryan myself. Obviously the correct solution though is to sterilize everyone just in case and then throw them into cryogenic suspension.