5 ms·

I read this back when I was a neuroscientist studying consciousness, and it was a very intriguing premise. The title was super-relevant, since I was actually an

I read this back when I was a neuroscientist studying consciousness, and it was a very intriguing premise. The title was super-relevant, since I was actually analyzing Patient GY’s data at the time! However, I think the key idea that consciousness is a local maximum in the universe is fundamentally mistaken, if a cool thought experiment. The brain is an expensive organ, and unless consciousness is truly pointless, it would be subject to evolutionary pressure like anything else. I doubt it could escape those pressures, even just within earth. The vampires were cool tho.

sho 3y ago
> I think the key idea that consciousness is a local maximum in the universe is fundamentally mistaken, if a cool thought experiment. Very much agree, on both counts. Even "just" on earth, it's not like the more mechanistic approaches didn't have plenty of chances to compete. The near-instant and very complete domination by the first species to reach a certain level - and its subsequent orders-of-magnitude raising as it then starts to compete mainly with itself - was totally driven by evolution and there's no reason to think it wouldn't happen anywhere else. The only variable I can really see is in where it (evolution-driven intelligence increase) stops. It's extremely hard to imagine even the basics such as group coordination and complex communication without some sense of self, and even the animals we see on earth with some level of these abilities demonstrate at least a proto-consciousness, which might well have won out in the end had we not been first past the post. There's no reason to think this phenomenon would be an earth-only anomaly. It reminds of the "million monkeys with a million keyboards" thought experiment. Yes, they will, given infinite time, produce the entire works of shakespeare. But in the real universe they don't have infinite time, the actual shakespeare evolved, wrote it all first, then took over the monkey planet and enslaved or at least repurposed them before they even got the first paragraph out. I don't mean to suggest that homo sapiens is some kind of universal peak of capability - we are almost certainly not - but I find it pretty much impossible to imagine sentient creatures being outcompeted by non-sentient ones under any circumstances. It's expensive, sure, but consciousness appears to be a - the - superweapon, and whoeever grasps it first, wins.
- MeImCounting 3y ago
  I think maybe you are misrepresenting the thesis of the book. My impression was, that in the book consciousness was a consequence of the evolution of intelligence on earth and that the scramblers had a totally different evolutionary pathway. Now that intelligence had evolved, its side effects namely consciousness was taking up processing power that could be used for more intelligence. Maybe I am misrepresenting the thesis of the book also, but at the very least I think it presents something more than an interesting thought experiment. I think the idea that intelligence, creativity etc can be totally separate from consciousness is a perfectly valid proposition and certainly relevant to current and future developments in ML, collective intelligence (companies, countries, mobs etc) I think in reality, separate from the story of the book, that consciousness being a key component in evolving truly intelligent systems from scratch makes a lot of sense. (and probably isnt as "expensive" as the book makes it out to be) I think it also makes sense that you can have non-sentient/unconscious intelligent systems just as intelligent as conscious ones and perhaps more. If the chinese room gives convincing answers why does it matter one way or the other whether it truly "understands" chinese?
  - sho 3y ago
    > I think maybe you are misrepresenting the thesis of the book I certainly could be! It has been a while. But are you saying your understand was that sentience was achieved in the scrambler's evolution, and then later discarded in favour of "more teraflops", if you'll excuse my analogy? That wasn't my understanding - but I might well be wrong. Even if so, the idea remains unconvincing, in fact more so. After a species achieves a certain domination in its environment - something you would very much expect from a civilisation capable of building spacecraft - evolution simply doesn't apply any more, not in any darwinian sense. Even with humans, evolution has basically stopped, or if it continues, does so under circumstances entirely under our control. I find it impossible to imagine any naturally-ocurring scenario in which such any sentient species is forced by evolutionary pressure to optimise away consciousness in favour of other mental tasks. They would instead, as you mention, simply augment their powers with technology - or deliberate genetic intervention. It's still an interesting premise and definitely got me thinking. I also highly recommend the book. It's actually quite exciting, all this philosophy does not take up that much space in the text ;) The problem with the chinese room is that someone had to build it... and someone had to invent chinese.
jdewerd 3y ago
Right, and while I'm no expert in this field I vaguely recall some proposals that consciousness / self-awareness fell out of planning and the need for a representation of your future / hypothetical self in those plans. If so, that would make it very important or a harmless side effect of something very important. I was mildly upset to see characters in Blindsight repeatedly bring up the "consciousness is pointless" thesis without any allusion to the common counterarguments, if only to dismiss them in the context of the story. But maybe that makes for better horror? 3 Body Problem did the same thing. It repeatedly presented the strengths of the Dark Forest Fermi paradox solution, but I don't think it ever presented the weaknesses, not even to lampshade. I usually try appreciate what's in a story rather than whine about what it "should have done" (which is a recipe for never enjoying anything) but in this case I've seen it negatively affect people who didn't already have the "antidote" background knowledge. I've seen this in online communities (you'd think there would always be at least one smartass in each sci-fi thread but I guess not) and I've seen it in person with a coworker, so I wonder if it isn't a bit of an ethics issue. Ah well, on the scale of ethics issues this one is pretty minor. Most people are stable enough to handle spooky alien stories, and maybe the double take is part of the fun.
- polotics 3y ago
  Could you point us to what these weaknesses are? Thank You!
  - jdewerd 3y ago
    Sure, a few brief points. 1: Remember that 3BP had to break the speed of light (among other things) to make the story work. A first strike is a huge gamble, you have to make sure your shot in the dark kills the monster rather than makes it angry. If you put information lag back into the equation (not to mention other intelligence gathering and military imperfections), the confidence that you need to make the game theory work starts to look incredibly steep. 2: Nearby grabby aliens are highly unlikely in any event because otherwise they would have grabbed already. The time to colonize the galaxy (millions of years) is tiny compared to likely variation in the timescale on which interstellar-capable life emerged and evolved (billions of years). If you turn the colonization of the galaxy into a 2 hour movie and separate it into periods of "no aliens," "sparse aliens," and "dense aliens," the "sparse aliens" section is one small scene lasting seconds. It would take an enormous feat of synchronization for us to be "born" into those few seconds rather than before or after (and "after" probably doesn't work because colonization stops development). "Sparse aliens" makes for great sci-fi, so we are predisposed to imagine that it is more likely than it actually is. It's like Jurassic Park: the dinosaurs don't always escape because the mathematics of chaos theory dictate that it would be impossible to put resurrected dinosaurs in a zoo, they escape because otherwise I wouldn't buy a movie ticket (and neither would you). 3: Suppose that the situation has defied the odds and there are spooky aliens out there just waiting to kill us. What's our move? Hide? Or grow aggressively? The "hide" option is already a bust, so we should grow aggressively. Everyone else will do the same. Just as the desirability of a first-strike hinges on unlikely confidence in the ability to effectively carry out complete retaliation-eliminating destruction on a rapidly advancing adversary far in the future, the desirability of hiding hinges on unlikely confidence in the ability to hide from adversaries with truly staggering surveillance options. Even if the forest is full of monsters, we do not expect them to hide. I love the 3BP trilogy and the atmosphere of horror that it builds, I just don't think its thesis works very well IRL. Which is a good thing :)
Log_out_ 3y ago
One could argue it is.. And it is reduced and replaced in some adaption to stressful environs? A traumatized system consists mostly of flashbacks and instant reactions to the stimulus. A paranoid adaptation to a hostile world shirks conscious perception of complexity for a "all things are enemy out to get me" minimalistic consciousness as it would be found in a hunted mouse. The more dangerous the environment becomes the more reduced cognition seems to be. Assuming now that most civilizations destroy the environment they are in, longterm evolution should award subconscious processing.. That is focused on longterm instinct behavior. Such a adaptation would present itself as worship of one's own gut feelings as divine being whos instructions and recommendations point towards less conscious processing and more optimization towards the environment, which does not reward consciousness and it's products:music, art, literature but rewards more offspring by continued existence. Watts seemed pretty correct to me and the billions in subsistence or warzones arguing in religious mobs for less conscious processing seem to agree. You are correct on the brain needed for conscious controlled cognition being expensive, thus only being able to be kept around by a scientific calorie surplus producing society. Without that the universal Fermi trap closes shut, the individual forever adapting towards a barely conscious tool using tribal entity.

sho 3y ago

> I think the key idea that consciousness is a local maximum in the universe is fundamentally mistaken, if a cool thought experiment. Very much agree, on both counts. Even "just" on earth, it's not like the more mechanistic approaches didn't have plenty of chances to compete. The near-instant and very complete domination by the first species to reach a certain level - and its subsequent orders-of-magnitude raising as it then starts to compete mainly with itself - was totally driven by evolution and there's no reason to think it wouldn't happen anywhere else. The only variable I can really see is in where it (evolution-driven intelligence increase) stops. It's extremely hard to imagine even the basics such as group coordination and complex communication without some sense of self, and even the animals we see on earth with some level of these abilities demonstrate at least a proto-consciousness, which might well have won out in the end had we not been first past the post. There's no reason to think this phenomenon would be an earth-only anomaly. It reminds of the "million monkeys with a million keyboards" thought experiment. Yes, they will, given infinite time, produce the entire works of shakespeare. But in the real universe they don't have infinite time, the actual shakespeare evolved, wrote it all first, then took over the monkey planet and enslaved or at least repurposed them before they even got the first paragraph out. I don't mean to suggest that homo sapiens is some kind of universal peak of capability - we are almost certainly not - but I find it pretty much impossible to imagine sentient creatures being outcompeted by non-sentient ones under any circumstances. It's expensive, sure, but consciousness appears to be a - the - superweapon, and whoeever grasps it first, wins.

MeImCounting 3y ago
I think maybe you are misrepresenting the thesis of the book. My impression was, that in the book consciousness was a consequence of the evolution of intelligence on earth and that the scramblers had a totally different evolutionary pathway. Now that intelligence had evolved, its side effects namely consciousness was taking up processing power that could be used for more intelligence. Maybe I am misrepresenting the thesis of the book also, but at the very least I think it presents something more than an interesting thought experiment. I think the idea that intelligence, creativity etc can be totally separate from consciousness is a perfectly valid proposition and certainly relevant to current and future developments in ML, collective intelligence (companies, countries, mobs etc) I think in reality, separate from the story of the book, that consciousness being a key component in evolving truly intelligent systems from scratch makes a lot of sense. (and probably isnt as "expensive" as the book makes it out to be) I think it also makes sense that you can have non-sentient/unconscious intelligent systems just as intelligent as conscious ones and perhaps more. If the chinese room gives convincing answers why does it matter one way or the other whether it truly "understands" chinese?
- sho 3y ago
  > I think maybe you are misrepresenting the thesis of the book I certainly could be! It has been a while. But are you saying your understand was that sentience was achieved in the scrambler's evolution, and then later discarded in favour of "more teraflops", if you'll excuse my analogy? That wasn't my understanding - but I might well be wrong. Even if so, the idea remains unconvincing, in fact more so. After a species achieves a certain domination in its environment - something you would very much expect from a civilisation capable of building spacecraft - evolution simply doesn't apply any more, not in any darwinian sense. Even with humans, evolution has basically stopped, or if it continues, does so under circumstances entirely under our control. I find it impossible to imagine any naturally-ocurring scenario in which such any sentient species is forced by evolutionary pressure to optimise away consciousness in favour of other mental tasks. They would instead, as you mention, simply augment their powers with technology - or deliberate genetic intervention. It's still an interesting premise and definitely got me thinking. I also highly recommend the book. It's actually quite exciting, all this philosophy does not take up that much space in the text ;) The problem with the chinese room is that someone had to build it... and someone had to invent chinese.

jdewerd 3y ago

Right, and while I'm no expert in this field I vaguely recall some proposals that consciousness / self-awareness fell out of planning and the need for a representation of your future / hypothetical self in those plans. If so, that would make it very important or a harmless side effect of something very important. I was mildly upset to see characters in Blindsight repeatedly bring up the "consciousness is pointless" thesis without any allusion to the common counterarguments, if only to dismiss them in the context of the story. But maybe that makes for better horror? 3 Body Problem did the same thing. It repeatedly presented the strengths of the Dark Forest Fermi paradox solution, but I don't think it ever presented the weaknesses, not even to lampshade. I usually try appreciate what's in a story rather than whine about what it "should have done" (which is a recipe for never enjoying anything) but in this case I've seen it negatively affect people who didn't already have the "antidote" background knowledge. I've seen this in online communities (you'd think there would always be at least one smartass in each sci-fi thread but I guess not) and I've seen it in person with a coworker, so I wonder if it isn't a bit of an ethics issue. Ah well, on the scale of ethics issues this one is pretty minor. Most people are stable enough to handle spooky alien stories, and maybe the double take is part of the fun.

polotics 3y ago
Could you point us to what these weaknesses are? Thank You!
- jdewerd 3y ago
  Sure, a few brief points. 1: Remember that 3BP had to break the speed of light (among other things) to make the story work. A first strike is a huge gamble, you have to make sure your shot in the dark kills the monster rather than makes it angry. If you put information lag back into the equation (not to mention other intelligence gathering and military imperfections), the confidence that you need to make the game theory work starts to look incredibly steep. 2: Nearby grabby aliens are highly unlikely in any event because otherwise they would have grabbed already. The time to colonize the galaxy (millions of years) is tiny compared to likely variation in the timescale on which interstellar-capable life emerged and evolved (billions of years). If you turn the colonization of the galaxy into a 2 hour movie and separate it into periods of "no aliens," "sparse aliens," and "dense aliens," the "sparse aliens" section is one small scene lasting seconds. It would take an enormous feat of synchronization for us to be "born" into those few seconds rather than before or after (and "after" probably doesn't work because colonization stops development). "Sparse aliens" makes for great sci-fi, so we are predisposed to imagine that it is more likely than it actually is. It's like Jurassic Park: the dinosaurs don't always escape because the mathematics of chaos theory dictate that it would be impossible to put resurrected dinosaurs in a zoo, they escape because otherwise I wouldn't buy a movie ticket (and neither would you). 3: Suppose that the situation has defied the odds and there are spooky aliens out there just waiting to kill us. What's our move? Hide? Or grow aggressively? The "hide" option is already a bust, so we should grow aggressively. Everyone else will do the same. Just as the desirability of a first-strike hinges on unlikely confidence in the ability to effectively carry out complete retaliation-eliminating destruction on a rapidly advancing adversary far in the future, the desirability of hiding hinges on unlikely confidence in the ability to hide from adversaries with truly staggering surveillance options. Even if the forest is full of monsters, we do not expect them to hide. I love the 3BP trilogy and the atmosphere of horror that it builds, I just don't think its thesis works very well IRL. Which is a good thing :)

Log_out_ 3y ago

One could argue it is.. And it is reduced and replaced in some adaption to stressful environs? A traumatized system consists mostly of flashbacks and instant reactions to the stimulus. A paranoid adaptation to a hostile world shirks conscious perception of complexity for a "all things are enemy out to get me" minimalistic consciousness as it would be found in a hunted mouse. The more dangerous the environment becomes the more reduced cognition seems to be. Assuming now that most civilizations destroy the environment they are in, longterm evolution should award subconscious processing.. That is focused on longterm instinct behavior. Such a adaptation would present itself as worship of one's own gut feelings as divine being whos instructions and recommendations point towards less conscious processing and more optimization towards the environment, which does not reward consciousness and it's products:music, art, literature but rewards more offspring by continued existence. Watts seemed pretty correct to me and the billions in subsistence or warzones arguing in religious mobs for less conscious processing seem to agree. You are correct on the brain needed for conscious controlled cognition being expensive, thus only being able to be kept around by a scientific calorie surplus producing society. Without that the universal Fermi trap closes shut, the individual forever adapting towards a barely conscious tool using tribal entity.