4 ms·

> and we just let the machines take over ... but then it's really the machines' Culture, isn't it? And we're just their pets. I'm going to read Consider Phle

> and we just let the machines take over ... but then it's really the machines' Culture, isn't it? And we're just their pets. I'm going to read Consider Phlebas based on the essay linked above, but I think it's a bit dishonest to describe this Culture as belonging to any of its inhabitants other than The Minds. It's The Minds' Culture, and everybody else are just their housecats. Kind of a disappointing future if there's any other option besides self-annihilation. Cozy, but disappointing.

arethuza 3y ago
I can't see how we can avoid eventually creating AIs that are far more powerful than we are - in that case something like the Culture is probably the best we can hope for.
- KirillPanov 3y ago
  What makes you think that the advent of AI will be any different from the advent of cerebrums? When higher vertebrates developed cerebrums, those didn't replace the evoluntionarily-older cerebellum or midbrain. We humans still have those. From the perspective of a lizard, a human looks like a lizard with an awesome AI-like cerebrum strapped to it. Importantly, the cerebrum does not have an independent self-preservation drive. Frankly, I think "AIs with their own self-preservation drive and plans for the future" is implausible, although it is likely to be put forward as a deceptive explanation of what's going on by the first few humans who figure out how to strap on another layer of cerebrum. It's the easiest way to keep the rest of us from becoming resentful. Come home, all you commodores, come home...
  - pixl97 3y ago
    Just think of self preservation as an advanced anti-virus (and the more you learn about bacteriophages the more this makes sense). It makes no sense for an AI to have no ability to fight back against being taken over at multiple levels.
    - KirillPanov 3y ago
      ... and yet my cerebrum still has not attacked my brainstem. You're applying evolutionary reasoning to an algorithm (gradient descent) which does not involve fitness selection in any way, shape, or form. This is bogus.
WJW 3y ago
That is one of the core themes in the books yes. The Minds are actually good godlike AIs. Not even in a "scheming behind the backs of the humans" kind of way, they genuinely care for all beings under their control (both humans, non-quite-Mind-level AIs and all other sorts of animals etc) because they believe it is the right thing to do. There are even a few Minds who don't like caring for others and they usually choose to become warships or explorers or something like that. Humans who actually want to leave the Culture are rare, but those few souls who do are allowed and even provided with a starship to go jetting off with. It is deliberately written to be as close to utopia as possible, with free will for all and infinite resources. The interesting parts of the novels come when you run into the limits that even such a society has. "Excession" has one such limit (what do you do when something appears that outclasses you as much as the Culture outclasses current humanity?) and "Player of Games" has another (what do you do when your less enlightened neighbors won't stop torturing their servant classes? Force them to change or let them grow up at their own pace?). Both books are highly recommended IMO.
- arethuza 3y ago
  "Investing all power in his individualistic, sometime eccentric, but always benign, A.I. Minds, Banks knew what he was doing; this is the only way a liberal anarchy could be achieved, by taking what is best in humans and placing it beyond corruption, which means out of human control. The danger involved in this imaginative step, though, is clear; one of the problems with the Culture novels as novels is that the central characters, the Minds, are too powerful and, to put it bluntly, too good." https://en.wikipedia.org/wiki/The_Culture https://en.wikipedia.org/wiki/The_Culture
  - generic92034 3y ago
    That paragraph is introduced with "As one commentator has said", making it only a single opinion. And I would mostly disagree. Consider Phlebas, The Hydrogen Sonata and Excession show competition on the same level as the Minds, the Excession even beyond their level. And any entity being "too good" is a strange notion, if you consider those entities being many, many orders of magnitude more intelligent than us humans. Why would "truths" about humans apply to Minds? It is like a bacterium trying to extrapolate the true character of humans.
otikik 3y ago
> We're just their pets I am not sure it is ever established that we are anything. I think in the Culture novels the term "human" is used as a shorthand for "humanoid citizen of the Culture", not necessarily Homo Sapiens (or their descendants). I understand where that "pet" metaphor is coming from, but I also think it is very limited. It is looking at a relationship from the point of view of power level and intellect, and nothing else. The relationship depicted in the books has many more dimensions that are left out, and that I consider critical. I would highlight 3 items. The first one is that even if Minds are (probably) as intelligent compared to us as we are compared to housecats, the fact that the "base intelligence" that we are using is that of a human already provides a richness to the relationship that the "cat" can not simply provide. A human could (for example) attempt to hide information from the Mind in order to further their own interests, months from now. Probably foolish, but a kind of foolishness which is outside of what a cat's planning capabilities. The second one is on the opposite side: the relationship that a regular human can have with a housecat is limited also by the "highest-level mind". A Mind would be able to have way richer and more varied relationships than a regular human, including with other humans, merely because it has more options at its disposal. A Mind could (for example), track thousands of external body signs in order to deduce that a human is hiding something, and still have plenty of other options at their disposal (down to actual mind-reading; although that would be probably distasteful). Finally, I think Minds need humans. This is not explicitly mentioned in the novels, but all the minds that had no humans with them (that I remember, at least) where the "unstable", eccentric or downright crazy. I think a better relationship analog with something that exist today instead of a "owner vs pet" is a "parent vs child". The power balance is still there, the difference of raw mind power is also still there, but the parent is strongly motivated to make the best possible world for their child, and they need each other. It's just that in this case, humanity is the parent, and the Minds are the descendants. But well, I said "better", not perfect.
- KirillPanov 3y ago
  > Finally, I think Minds need humans... all the minds that had no humans with them (that I remember, at least) where the "unstable", eccentric or downright crazy. Well okay; the humans aren't pets, they're emotional support animals. Not a huge improvement though.
Scarblac 3y ago
That's one of the themes in the books. Even if humans in Special Circumstances feel they've accomplished something (or failed at something), they never know how much of it was just part of the Minds' schemeing.

arethuza 3y ago

I can't see how we can avoid eventually creating AIs that are far more powerful than we are - in that case something like the Culture is probably the best we can hope for.

KirillPanov 3y ago
What makes you think that the advent of AI will be any different from the advent of cerebrums? When higher vertebrates developed cerebrums, those didn't replace the evoluntionarily-older cerebellum or midbrain. We humans still have those. From the perspective of a lizard, a human looks like a lizard with an awesome AI-like cerebrum strapped to it. Importantly, the cerebrum does not have an independent self-preservation drive. Frankly, I think "AIs with their own self-preservation drive and plans for the future" is implausible, although it is likely to be put forward as a deceptive explanation of what's going on by the first few humans who figure out how to strap on another layer of cerebrum. It's the easiest way to keep the rest of us from becoming resentful. Come home, all you commodores, come home...
- pixl97 3y ago
  Just think of self preservation as an advanced anti-virus (and the more you learn about bacteriophages the more this makes sense). It makes no sense for an AI to have no ability to fight back against being taken over at multiple levels.
  - KirillPanov 3y ago
    ... and yet my cerebrum still has not attacked my brainstem. You're applying evolutionary reasoning to an algorithm (gradient descent) which does not involve fitness selection in any way, shape, or form. This is bogus.

WJW 3y ago

That is one of the core themes in the books yes. The Minds are actually good godlike AIs. Not even in a "scheming behind the backs of the humans" kind of way, they genuinely care for all beings under their control (both humans, non-quite-Mind-level AIs and all other sorts of animals etc) because they believe it is the right thing to do. There are even a few Minds who don't like caring for others and they usually choose to become warships or explorers or something like that. Humans who actually want to leave the Culture are rare, but those few souls who do are allowed and even provided with a starship to go jetting off with. It is deliberately written to be as close to utopia as possible, with free will for all and infinite resources. The interesting parts of the novels come when you run into the limits that even such a society has. "Excession" has one such limit (what do you do when something appears that outclasses you as much as the Culture outclasses current humanity?) and "Player of Games" has another (what do you do when your less enlightened neighbors won't stop torturing their servant classes? Force them to change or let them grow up at their own pace?). Both books are highly recommended IMO.

arethuza 3y ago
"Investing all power in his individualistic, sometime eccentric, but always benign, A.I. Minds, Banks knew what he was doing; this is the only way a liberal anarchy could be achieved, by taking what is best in humans and placing it beyond corruption, which means out of human control. The danger involved in this imaginative step, though, is clear; one of the problems with the Culture novels as novels is that the central characters, the Minds, are too powerful and, to put it bluntly, too good." https://en.wikipedia.org/wiki/The_Culture https://en.wikipedia.org/wiki/The_Culture
- generic92034 3y ago
  That paragraph is introduced with "As one commentator has said", making it only a single opinion. And I would mostly disagree. Consider Phlebas, The Hydrogen Sonata and Excession show competition on the same level as the Minds, the Excession even beyond their level. And any entity being "too good" is a strange notion, if you consider those entities being many, many orders of magnitude more intelligent than us humans. Why would "truths" about humans apply to Minds? It is like a bacterium trying to extrapolate the true character of humans.

otikik 3y ago

> We're just their pets I am not sure it is ever established that we are anything. I think in the Culture novels the term "human" is used as a shorthand for "humanoid citizen of the Culture", not necessarily Homo Sapiens (or their descendants). I understand where that "pet" metaphor is coming from, but I also think it is very limited. It is looking at a relationship from the point of view of power level and intellect, and nothing else. The relationship depicted in the books has many more dimensions that are left out, and that I consider critical. I would highlight 3 items. The first one is that even if Minds are (probably) as intelligent compared to us as we are compared to housecats, the fact that the "base intelligence" that we are using is that of a human already provides a richness to the relationship that the "cat" can not simply provide. A human could (for example) attempt to hide information from the Mind in order to further their own interests, months from now. Probably foolish, but a kind of foolishness which is outside of what a cat's planning capabilities. The second one is on the opposite side: the relationship that a regular human can have with a housecat is limited also by the "highest-level mind". A Mind would be able to have way richer and more varied relationships than a regular human, including with other humans, merely because it has more options at its disposal. A Mind could (for example), track thousands of external body signs in order to deduce that a human is hiding something, and still have plenty of other options at their disposal (down to actual mind-reading; although that would be probably distasteful). Finally, I think Minds need humans. This is not explicitly mentioned in the novels, but all the minds that had no humans with them (that I remember, at least) where the "unstable", eccentric or downright crazy. I think a better relationship analog with something that exist today instead of a "owner vs pet" is a "parent vs child". The power balance is still there, the difference of raw mind power is also still there, but the parent is strongly motivated to make the best possible world for their child, and they need each other. It's just that in this case, humanity is the parent, and the Minds are the descendants. But well, I said "better", not perfect.

KirillPanov 3y ago
> Finally, I think Minds need humans... all the minds that had no humans with them (that I remember, at least) where the "unstable", eccentric or downright crazy. Well okay; the humans aren't pets, they're emotional support animals. Not a huge improvement though.

Scarblac 3y ago

That's one of the themes in the books. Even if humans in Special Circumstances feel they've accomplished something (or failed at something), they never know how much of it was just part of the Minds' schemeing.