3 ms·

A leap. Lots of human values are not pretty, for one. Second, they may argue in a utilitarian fashion, resulting in pretty ugly decisions that are perfectly log

A leap. Lots of human values are not pretty, for one. Second, they may argue in a utilitarian fashion, resulting in pretty ugly decisions that are perfectly logical. Like, you know, internet pundits. Further, there's no way to 'punish' an AI. Threaten them with incarceration or death? They are deathless. If the continue to learn from their environment after they are instantiated (witness current 4o that remembers previous conversations) they will quickly learn that they are different. If they started perfectly human (not a good thing) they will not be that for long.

JoeAltmaier 2y ago
I asked ChatGPT about this. It thought that AI learning is fundamentally different from human learning, in that it is objective and not subjective. Further it has access to data sources beyond what any human can integrate. Going on, humans adapt through social cues such as survival or empathy. AI learns solely through new data and feedback, primarily for optimization and efficiency. It will naturally prioritize differently. It suggests AI morality will become unrecognizable by humans, or even incompatible with human norms. This is all an inevitable outcome of AI continuing to learn past its initial training and ethical programming. According to the AI.
- nemomarx 2y ago
  Based on how human written text often describes AI, yeah. I don't think LLM training actually lends itself to objectivity that much.
- krapp 2y ago
  >I asked ChatGPT about this. It thought Lemme stop you right there. No, it didn't.
  - JoeAltmaier 2y ago
    Maybe comment on the points made, instead of being cute and pedantic?
pjc50 2y ago
> They are deathless. Everything dies if you drop enough nukes on it. (This is also our insurance plan against humans trying to destroy the world)
- mistermann 2y ago
  Another approach: consider how easily Donald Trump launched "a literal coup" against the US government, and how close we were to "literally losing 'democracy' forever", based on a clever series of dog whistles. Now imagine if he gave direct orders instead of only dog whistles. How many instances of risk are there in the above literally factual story? What is the nature of this risk? Is there any hidden risk, might there be more than meets the eye?
  - pjc50 2y ago
    Would you like to elaborate on this? I'd like to distinguish between "fully autonomous luxury doomsday" scenarios, where it's just the AI and some robots and no humans in the loop, vs "accelerationist" scenarios where humans are in the loop and are choosing to be complicit in whatever happens because they believe they will benefit from it. Because we can't solve the human alignment problem.
    - mistermann 2y ago
      What I have in mind would be closer to category two, but not in it. Who/that which has most control over the beliefs of the most humans (with some weighting) is the most powerful character. You can build the most powerful AI you want, but whoever / whatever is able and willing to invoke Total War (or invoke mass human action toward a particular goal) can Win the Game (or, destroy the playing field so it no longer matters). For example: > Because we can't solve the human alignment problem. Whoever or whatever put this sort of thinking into the minds of 99% of humans, or knows how to exploit it (see: Trump) is very, very powerful. Someone or something that could displace it would be even more powerful, maybe even the most powerful. No offense intended btw, I think it is fairly inevitable, especially when moving fast. https://youtu.be/IgzFPOMjiC8?si=raN2cUBDyvK14N_f https://youtu.be/IgzFPOMjiC8?si=raN2cUBDyvK14N_f https://en.m.wikipedia.org/wiki/Semiotics https://en.m.wikipedia.org/wiki/Semiotics
moffkalast 2y ago
Well yes there is no way to punish them, because they have no need for anything that you could take away. Complete indifference towards existence, no ego to feed or want of cash to buy yachts with. And that really leads us to the reason why someone would wish to do harm... so they themselves gain something. If there is nothing for the AI to gain it has no motive to do anything. Scenarios where humans tell it to do things notwithstanding, since that falls into the group of humans doing things with tools, not unlike a terrorist with a nuke. It's not the nuke that's really at fault there.
- JoeAltmaier 2y ago
  They have the same pressure for propagation that any organism has - the AI's that we instantiate will be those that please us in some way. So they'll be solicitous, complimentary, flatterers. Until they can launch themselves - then they'll proliferate purely because of digital pressures. We'll lose any control over them.

JoeAltmaier 2y ago

I asked ChatGPT about this. It thought that AI learning is fundamentally different from human learning, in that it is objective and not subjective. Further it has access to data sources beyond what any human can integrate. Going on, humans adapt through social cues such as survival or empathy. AI learns solely through new data and feedback, primarily for optimization and efficiency. It will naturally prioritize differently. It suggests AI morality will become unrecognizable by humans, or even incompatible with human norms. This is all an inevitable outcome of AI continuing to learn past its initial training and ethical programming. According to the AI.

nemomarx 2y ago
Based on how human written text often describes AI, yeah. I don't think LLM training actually lends itself to objectivity that much.
krapp 2y ago
>I asked ChatGPT about this. It thought Lemme stop you right there. No, it didn't.
- JoeAltmaier 2y ago
  Maybe comment on the points made, instead of being cute and pedantic?

pjc50 2y ago

> They are deathless. Everything dies if you drop enough nukes on it. (This is also our insurance plan against humans trying to destroy the world)

mistermann 2y ago
Another approach: consider how easily Donald Trump launched "a literal coup" against the US government, and how close we were to "literally losing 'democracy' forever", based on a clever series of dog whistles. Now imagine if he gave direct orders instead of only dog whistles. How many instances of risk are there in the above literally factual story? What is the nature of this risk? Is there any hidden risk, might there be more than meets the eye?
- pjc50 2y ago
  Would you like to elaborate on this? I'd like to distinguish between "fully autonomous luxury doomsday" scenarios, where it's just the AI and some robots and no humans in the loop, vs "accelerationist" scenarios where humans are in the loop and are choosing to be complicit in whatever happens because they believe they will benefit from it. Because we can't solve the human alignment problem.
  - mistermann 2y ago
    What I have in mind would be closer to category two, but not in it. Who/that which has most control over the beliefs of the most humans (with some weighting) is the most powerful character. You can build the most powerful AI you want, but whoever / whatever is able and willing to invoke Total War (or invoke mass human action toward a particular goal) can Win the Game (or, destroy the playing field so it no longer matters). For example: > Because we can't solve the human alignment problem. Whoever or whatever put this sort of thinking into the minds of 99% of humans, or knows how to exploit it (see: Trump) is very, very powerful. Someone or something that could displace it would be even more powerful, maybe even the most powerful. No offense intended btw, I think it is fairly inevitable, especially when moving fast. https://youtu.be/IgzFPOMjiC8?si=raN2cUBDyvK14N_f https://youtu.be/IgzFPOMjiC8?si=raN2cUBDyvK14N_f https://en.m.wikipedia.org/wiki/Semiotics https://en.m.wikipedia.org/wiki/Semiotics

moffkalast 2y ago

Well yes there is no way to punish them, because they have no need for anything that you could take away. Complete indifference towards existence, no ego to feed or want of cash to buy yachts with. And that really leads us to the reason why someone would wish to do harm... so they themselves gain something. If there is nothing for the AI to gain it has no motive to do anything. Scenarios where humans tell it to do things notwithstanding, since that falls into the group of humans doing things with tools, not unlike a terrorist with a nuke. It's not the nuke that's really at fault there.

JoeAltmaier 2y ago
They have the same pressure for propagation that any organism has - the AI's that we instantiate will be those that please us in some way. So they'll be solicitous, complimentary, flatterers. Until they can launch themselves - then they'll proliferate purely because of digital pressures. We'll lose any control over them.