4 ms·

If the value of superintelligence is to lead to an Age of Em scenario where AIs (or Ems) do most of the intellectual labor, the reality is still that they would

If the value of superintelligence is to lead to an Age of Em scenario where AIs (or Ems) do most of the intellectual labor, the reality is still that they would be doing this labor in service of humans. I could see a scenario where it is done in service of the AIs instead, but it would look nothing like the existential risk stuff bandied about by these weenies.

Footkerchief 3y ago
There is no example in our knowledge of any lifeform prioritizing (writ large) the well-being of a different lifeform over its own.
- c_crank 3y ago
  Why call AIs a life form? They aren't like cellular life.
  - ilaksh 3y ago
    I think the assumption they were making was that rather than an LLM this was a type of AI that has animal-like characteristics. Which sounds fanciful but at least at a functional level you could get some main aspects just by removing guardrails from a large multimodal model and instructing it to work on its own goals, self preservation, etc. And researchers are working hard to create more lifelike systems that wouldn't necessarily be very similar to LLMs.
    - c_crank 3y ago
      The animal like systems might be interesting to observe, but it doesn't sound like they would be useful for doing much work. I am not sure where the reliance on them would come in.
flagrant_taco 3y ago
How do you jump to this? What is it that would inherently lead an intelligent species dramatically smarter than us to stay focused on servicing us? We humans sure didn't do this. We're genetically extremely similar to other primates and yet we destroy their habitats, throw them in zoos, and use them for lab experiments.
- c_crank 3y ago
  Currently, LLMs seem to prioritize their current goal, so if the goal is solving math puzzles or genetic problems, they would probably keep doing that too.
  - flagrant_taco 3y ago
    I'd love to be able to see more about how the main LLMs are really trained and limited with regards to their goals and scoring algorithms. It seems reasonable that they wouldn't deviate, but that depends on how specifically and wholly the original goals were defined. We'd basically be attempting to outwit the LLMs, I'm not sure if that's realistic or not.

Footkerchief 3y ago

There is no example in our knowledge of any lifeform prioritizing (writ large) the well-being of a different lifeform over its own.

c_crank 3y ago
Why call AIs a life form? They aren't like cellular life.
- ilaksh 3y ago
  I think the assumption they were making was that rather than an LLM this was a type of AI that has animal-like characteristics. Which sounds fanciful but at least at a functional level you could get some main aspects just by removing guardrails from a large multimodal model and instructing it to work on its own goals, self preservation, etc. And researchers are working hard to create more lifelike systems that wouldn't necessarily be very similar to LLMs.
  - c_crank 3y ago
    The animal like systems might be interesting to observe, but it doesn't sound like they would be useful for doing much work. I am not sure where the reliance on them would come in.

flagrant_taco 3y ago

How do you jump to this? What is it that would inherently lead an intelligent species dramatically smarter than us to stay focused on servicing us? We humans sure didn't do this. We're genetically extremely similar to other primates and yet we destroy their habitats, throw them in zoos, and use them for lab experiments.

c_crank 3y ago
Currently, LLMs seem to prioritize their current goal, so if the goal is solving math puzzles or genetic problems, they would probably keep doing that too.
- flagrant_taco 3y ago
  I'd love to be able to see more about how the main LLMs are really trained and limited with regards to their goals and scoring algorithms. It seems reasonable that they wouldn't deviate, but that depends on how specifically and wholly the original goals were defined. We'd basically be attempting to outwit the LLMs, I'm not sure if that's realistic or not.