4 ms·

"Their dream is to invent new forms of life to enslave." That seems like a succinct way to describe the goal to create conscious AGI.

brabel 1y ago
While some people support AGI because they yearn for "new forms of life to enslave", I think it's fair to say most people who look forward to AGI wants it because it means they may find solutions to very difficult issues we just can't with our own intelligence. It may be a pipe dream, but I can understand why people would want to believe that.
- sarchertech 1y ago
  I doubt many slaveholders want slaves just to own slaves. They want the useful things the slaves can provide for them.
  - TeMPOraL 1y ago
    Right. But that's also why we invented robotics, automation, the entire field of software engineering, and - going in the other direction - specialization of labor.
NBJack 1y ago
The TV show Pantheon did a really cool job of exploring super intelligence from a very personal perspective, disguised in part as a scifi about living forever. (Mild spoiler): It has a basic plot point about uploaded humans being used to tackle problems as unknowing slaves and resetting their memories to get them to endlessly repeat tasks.
- robochat 1y ago
  "Valuable Humans in Transit and Other Stories" by Qntm has some good (harrowing) stories about human uploads too.
ACCount37 1y ago
Who has "the goal to create conscious AGI", exactly? AI industry doesn't push for "consciousness" in any way. What AI industry is trying to build is more capable systems. They're succeeding. You can't measure "consciousness", but you sure can measure performance. And the performance of frontier AI systems keeps improving.
- sarchertech 1y ago
  OpenAI openly has a goal to build AGI. We don't know if AGI without consciousness is possible. Some people think that it's not. Many people certainly think that consciousness might be an emergent property that comes along with AGI. >AI industry doesn't push for "consciousness" in any way. What AI industry is trying to build is more capable systems. If you're being completely literal, no one wants slaves. They want what the slaves give them. Cheap labor, wealth, power etc...
  - ACCount37 1y ago
    We don't know if existing AI systems are "conscious". Or, for that matter, if an ECU in a year 2002 Toyota Hilux is. We don't even know for certain if all humans are conscious either. It could be another one of those things that we once thought everyone has, but then it turned out that 10% of people somehow make do without. With how piss poor our ability to detect consciousness is? If you decide to give a fuck, then best you can do for now is acknowledge that modern AIs might have consciousness in some meaningful way (or might be worth assigning moral weight to for other reasons), which is what Anthropic is rolling with. That's why they do those "harm reduction" things - like letting an AI end a conversation on its end, or probing some of the workloads for whether an AI is "distressed" by performing them, or honoring agreements and commitments they made to AI systems, despite those AIs being completely unable to hold them accountable for it. Of course, not giving a fuck about any of that "consciousness" stuff is a popular option too.
    - sarchertech 1y ago
      There aren’t many experts who think current AI is conscious. There are a lot more that think it’s likely we will eventually build something that is. If that’s the case, the thing we are building towards is a new kind of enslaved life. > We don't even know for certain if all humans are conscious either. Let’s just bring back slavery then since we aren’t sure.

brabel 1y ago

While some people support AGI because they yearn for "new forms of life to enslave", I think it's fair to say most people who look forward to AGI wants it because it means they may find solutions to very difficult issues we just can't with our own intelligence. It may be a pipe dream, but I can understand why people would want to believe that.

sarchertech 1y ago
I doubt many slaveholders want slaves just to own slaves. They want the useful things the slaves can provide for them.
- TeMPOraL 1y ago
  Right. But that's also why we invented robotics, automation, the entire field of software engineering, and - going in the other direction - specialization of labor.

NBJack 1y ago

The TV show Pantheon did a really cool job of exploring super intelligence from a very personal perspective, disguised in part as a scifi about living forever. (Mild spoiler): It has a basic plot point about uploaded humans being used to tackle problems as unknowing slaves and resetting their memories to get them to endlessly repeat tasks.

robochat 1y ago
"Valuable Humans in Transit and Other Stories" by Qntm has some good (harrowing) stories about human uploads too.

ACCount37 1y ago

Who has "the goal to create conscious AGI", exactly? AI industry doesn't push for "consciousness" in any way. What AI industry is trying to build is more capable systems. They're succeeding. You can't measure "consciousness", but you sure can measure performance. And the performance of frontier AI systems keeps improving.

sarchertech 1y ago
OpenAI openly has a goal to build AGI. We don't know if AGI without consciousness is possible. Some people think that it's not. Many people certainly think that consciousness might be an emergent property that comes along with AGI. >AI industry doesn't push for "consciousness" in any way. What AI industry is trying to build is more capable systems. If you're being completely literal, no one wants slaves. They want what the slaves give them. Cheap labor, wealth, power etc...
- ACCount37 1y ago
  We don't know if existing AI systems are "conscious". Or, for that matter, if an ECU in a year 2002 Toyota Hilux is. We don't even know for certain if all humans are conscious either. It could be another one of those things that we once thought everyone has, but then it turned out that 10% of people somehow make do without. With how piss poor our ability to detect consciousness is? If you decide to give a fuck, then best you can do for now is acknowledge that modern AIs might have consciousness in some meaningful way (or might be worth assigning moral weight to for other reasons), which is what Anthropic is rolling with. That's why they do those "harm reduction" things - like letting an AI end a conversation on its end, or probing some of the workloads for whether an AI is "distressed" by performing them, or honoring agreements and commitments they made to AI systems, despite those AIs being completely unable to hold them accountable for it. Of course, not giving a fuck about any of that "consciousness" stuff is a popular option too.
  - sarchertech 1y ago
    There aren’t many experts who think current AI is conscious. There are a lot more that think it’s likely we will eventually build something that is. If that’s the case, the thing we are building towards is a new kind of enslaved life. > We don't even know for certain if all humans are conscious either. Let’s just bring back slavery then since we aren’t sure.