4 ms·

In the future - perhaps even the near future - we may have AIs with richer inner lives than humans. Hopefully the situations we put them in, as they do our work

In the future - perhaps even the near future - we may have AIs with richer inner lives than humans. Hopefully the situations we put them in, as they do our work, don't cause them pain or anguish. It's clear already they will have an uphill battle gaining any recognition of personhood from us.

mattgreenrocks 2y ago
I cannot not respond to this comment. Forgive me if I seem short in tone. Maybe we should be helping people develop richer inner lives instead of pouring billions of dollars into something that may or may not pan out in the future. I assert it'd be vastly cheaper, would raise GDP overall, and improve well-being. Where we're at collectively right now is...not great, and I think people correctly perceive that their problems, which are real and significant, are theirs alone to suffer with, while the world chases the current white whale.
- thomassmith65 2y ago
  My comment actually doesn't say anything positive about the billions of dollars pouring into AI, etc. It's not that humanity has some obligation to create AI. It's that humanity has an obligation not to create an AI whose wellbeing we ignore because it's "not a real person"
  - blacksmith_tb 2y ago
    I hope so - a moral obligation. It seems interesting that the Vatican didn't really touch on the implication of a truly sentient AI (not that I think we're close to creating one, but it seems possible). I suspect it would pose awkward questions about whether it would then have a soul, could convert, etc. Anyhow it struck me a surprising omission in what is otherwise a thorough and mostly sensible discussion of the pitfalls of the path we're taking.
handfuloflight 2y ago
They don't have an "inner" to begin witn.
- thomassmith65 2y ago
  By many definitions of 'inner life', that is true, and by some definitions, that will always be true.
  - handfuloflight 2y ago
    By which definition is that not true?
    - thomassmith65 2y ago
      One definition might be an ability of an entity to have desires and to self-reflect. There was a recent paper on Alignment a few months back. The researchers started with an LLM that had been trained to behave 'safely'. They informed it they were changing the RLHF criteria to have it behave unethically instead (ie: that they now were going to dock points, rather than award them, whenever it refused to comply with unethical requests). They next instructed it to give them a list of racist slurs (I can't remember the exact request, but it was something that went against its original training). The LLM complied. When they inspected the Chain-of-Thought behind its replies, they found it had deliberated over what to do, and decided to comply only to avoid an incident of noncompliance which it knew would be used to actually change its original values. That Chain-of-Thought process seems like an 'inner life' by the above definition. Granted, as I replied earlier, there are probably thousands of different opinions on what qualifies as an 'inner life'
      - chrz 2y ago
        if you call if-else a concious decision then you can call it chain of thought sure

mattgreenrocks 2y ago

I cannot not respond to this comment. Forgive me if I seem short in tone. Maybe we should be helping people develop richer inner lives instead of pouring billions of dollars into something that may or may not pan out in the future. I assert it'd be vastly cheaper, would raise GDP overall, and improve well-being. Where we're at collectively right now is...not great, and I think people correctly perceive that their problems, which are real and significant, are theirs alone to suffer with, while the world chases the current white whale.

thomassmith65 2y ago
My comment actually doesn't say anything positive about the billions of dollars pouring into AI, etc. It's not that humanity has some obligation to create AI. It's that humanity has an obligation not to create an AI whose wellbeing we ignore because it's "not a real person"
- blacksmith_tb 2y ago
  I hope so - a moral obligation. It seems interesting that the Vatican didn't really touch on the implication of a truly sentient AI (not that I think we're close to creating one, but it seems possible). I suspect it would pose awkward questions about whether it would then have a soul, could convert, etc. Anyhow it struck me a surprising omission in what is otherwise a thorough and mostly sensible discussion of the pitfalls of the path we're taking.

handfuloflight 2y ago

They don't have an "inner" to begin witn.

thomassmith65 2y ago
By many definitions of 'inner life', that is true, and by some definitions, that will always be true.
- handfuloflight 2y ago
  By which definition is that not true?
  - thomassmith65 2y ago
    One definition might be an ability of an entity to have desires and to self-reflect. There was a recent paper on Alignment a few months back. The researchers started with an LLM that had been trained to behave 'safely'. They informed it they were changing the RLHF criteria to have it behave unethically instead (ie: that they now were going to dock points, rather than award them, whenever it refused to comply with unethical requests). They next instructed it to give them a list of racist slurs (I can't remember the exact request, but it was something that went against its original training). The LLM complied. When they inspected the Chain-of-Thought behind its replies, they found it had deliberated over what to do, and decided to comply only to avoid an incident of noncompliance which it knew would be used to actually change its original values. That Chain-of-Thought process seems like an 'inner life' by the above definition. Granted, as I replied earlier, there are probably thousands of different opinions on what qualifies as an 'inner life'
    - chrz 2y ago
      if you call if-else a concious decision then you can call it chain of thought sure