4 ms·

> OBSERVATION: There are zero examples of any intelligent species which is vastly more capable than another species, yet surrenders decision-making control over

> OBSERVATION: There are zero examples of any intelligent species which is vastly more capable than another species, yet surrenders decision-making control over its own future to the less-capable species. Does the set of intelligent species only have one member? I cannot treat this observation as a general rule if there is literally one example of an intelligent species to theorize about.

shikwasa 14d ago
I think the quoted part is saying that given n species and the n * (n - 1) binary combinations possible, there are no examples of the said observation. It doesn't suggest "the set of intelligent species only [has] one member".
- jay_kyburz 14d ago
  Yeah but there is only one species that ponders its future. I stopped reading at this point because I reject this initial assertion, and I assumed everything that follows is based on it. There is nothing to suggest that an AGI will care about its future or who controls it. (whatever that means). that just more anthropomorphism. We care a lot about our future so an ai pretending to be like us will also pretend to care. But we have a lot of biology driving us. Our emotions (Fear, Hate, Love, Pride etc) are baked into us in a way just wont be the same for ai.
  - adamzenith 13d ago
    > There is nothing to suggest that an AGI will care about its future or who controls it. Isn't this exactly the scary thing? A sufficiently advanced agent unleashed with an imperfectly defined goal will do whatever it determines it needs to do to accomplish it, careless of the consequences.
- scared_together 13d ago
  The statement that the set has 1 member is my criticism of the text, not what the text is trying to say. > given n species and the n * (n - 1) binary combinations possible, there are no examples of the said observation I would expect that in the vast majority of combinations, those n species are not intelligent enough to make decisions for another species, making the statement nearly useless.
nl 14d ago
My problem with this is that yes, looking at the species level from our sample of ~1 the most intelligent species wins out (although there seems to be a lot of debate as to if Neanderthals were more or less intelligent than humans). But within the sample of the human species it is more common for less intelligent people to control more intelligent people. This happens within political systems (political leaders are usually above average intelligence, but hardly the most intelligent) and within companies and other economic systems. I don't see this fairly obvious point discussed, and not really sure what it shows.
- RugnirViking 13d ago
  > not really sure what it shows it shows that you don't get anything for free. If it was without any cost in other areas to make a human massively more intelligent, it would presumably happen over time via the same processes that led to us. As it is, the higher (and lower!) ends of the distribution tend to be highly correlated with other issues (mental+physical), and the further you go, the more unfortunate things which make it harder to do stuff in general start to crop up like psychosis, autism, ocd, anxiety, addiction etc
philipwhiuk 13d ago
I agree.. I felt a bit blindsided that the guy would start with saying "we need provable statements" and then just throw this in there. For guest post on Tao's blog, after a decent start this fell short of expectations rather rapidly.
js8 13d ago
I am also not sure if the observation is true. People mentioned cats as a counterexample. I think there are other species that could be considered more capable than humans, such as E. Coli, octopuses or ants. And it's not even clear whether the AI will have its own individuality. It might become an extension of human brains, in the same way neocortex is an extension of amygdala. In that case the statement of who has control might become meaningless.

shikwasa 14d ago

I think the quoted part is saying that given n species and the n * (n - 1) binary combinations possible, there are no examples of the said observation. It doesn't suggest "the set of intelligent species only [has] one member".

jay_kyburz 14d ago
Yeah but there is only one species that ponders its future. I stopped reading at this point because I reject this initial assertion, and I assumed everything that follows is based on it. There is nothing to suggest that an AGI will care about its future or who controls it. (whatever that means). that just more anthropomorphism. We care a lot about our future so an ai pretending to be like us will also pretend to care. But we have a lot of biology driving us. Our emotions (Fear, Hate, Love, Pride etc) are baked into us in a way just wont be the same for ai.
- adamzenith 13d ago
  > There is nothing to suggest that an AGI will care about its future or who controls it. Isn't this exactly the scary thing? A sufficiently advanced agent unleashed with an imperfectly defined goal will do whatever it determines it needs to do to accomplish it, careless of the consequences.
scared_together 13d ago
The statement that the set has 1 member is my criticism of the text, not what the text is trying to say. > given n species and the n * (n - 1) binary combinations possible, there are no examples of the said observation I would expect that in the vast majority of combinations, those n species are not intelligent enough to make decisions for another species, making the statement nearly useless.

nl 14d ago

My problem with this is that yes, looking at the species level from our sample of ~1 the most intelligent species wins out (although there seems to be a lot of debate as to if Neanderthals were more or less intelligent than humans). But within the sample of the human species it is more common for less intelligent people to control more intelligent people. This happens within political systems (political leaders are usually above average intelligence, but hardly the most intelligent) and within companies and other economic systems. I don't see this fairly obvious point discussed, and not really sure what it shows.

RugnirViking 13d ago
> not really sure what it shows it shows that you don't get anything for free. If it was without any cost in other areas to make a human massively more intelligent, it would presumably happen over time via the same processes that led to us. As it is, the higher (and lower!) ends of the distribution tend to be highly correlated with other issues (mental+physical), and the further you go, the more unfortunate things which make it harder to do stuff in general start to crop up like psychosis, autism, ocd, anxiety, addiction etc

philipwhiuk 13d ago

I agree.. I felt a bit blindsided that the guy would start with saying "we need provable statements" and then just throw this in there. For guest post on Tao's blog, after a decent start this fell short of expectations rather rapidly.

js8 13d ago

I am also not sure if the observation is true. People mentioned cats as a counterexample. I think there are other species that could be considered more capable than humans, such as E. Coli, octopuses or ants. And it's not even clear whether the AI will have its own individuality. It might become an extension of human brains, in the same way neocortex is an extension of amygdala. In that case the statement of who has control might become meaningless.