3 ms·

Imagine an idiot-savant. He can tell you with complete accuracy the answer to complex mathematical problems, but has no understanding of how he arrives at the r

Imagine an idiot-savant. He can tell you with complete accuracy the answer to complex mathematical problems, but has no understanding of how he arrives at the result. Now imagine a hyper intelligent alien who is like the savant. It can construct interstellar vessels, but has no self-awareness of how it came up with the designs. Though, it is clear that they do work. The alien can observe a human and know exactly how to best injure him or her, but it cannot understand how it came to know human anatomy, or even why the attacks work. For example, if it performed a Vulcan neck pinch, and you asked it how it figured out how to do that then it might tell you that "it just felt like the right thing to do." It has no conscious notion of the human mind or neurology. The alien in Blindsight also apparently does not care to understand such things, and may lack the capacity for this kind of thought. It runs entirely on instinct, but these instincts bake in extremely complex behaviors that are beyond human capabilities. I'm also reminded of two other aliens from other Sci Fi series: 1) the Orkz from Warhammer 40k. Dumb as a rock, but can build devastatingly effective vehicles and weapons from salvage and garbage. God knows how. They certainly have no idea. 2) The Motie engineers from "The Mote in God's Eye." They can instinctually build and improve on machines but cannot understand, or come close to explaining, How they did it.

aeorgnoieang 9y ago
This confuses 'levels', or what exactly is the intelligent 'agent'. The idiot-savant is not generally intelligent. Depending on what you mean by "complex mathematical problems" even the neural components actually responsible for solving the problems might not be intelligent. In the case of actual idiot-savants, I'd suspect those components aren't 'intelligent'; they're too simple. The alien in Blindsight, if it really "runs entirely on instinct", then I'd expect it to do badly in novel situations, unless it's version of "instincts" basically "bake in extremely complex behaviors" that make it functionally equivalent or similar to a human – at the level we're used to dealing with as humans. Also note that most of your examples are fictional, i.e. not actual evidence beyond maybe extremely weak plausibility. AFAIK, every "high intelligence" of which actual people have actual strong evidence is self aware. That seems like it should be an important fact! Also, you're conflating self-awareness with the ability to (verbally) explain, i.e. communicate, thoughts to others. By "self-awareness", I was thinking of an 'intelligent system' including some sub-model of itself in whatever the relevant domain it is we're considering. For a 'general-purpose reproductive-fitness-maximizing-strategy executor' that's intelligent, e.g. us, it seems pretty unlikely that they don't, generally, model themselves. We should probably taboo "self-awareness" as it's pretty obviously too slippery and ambiguous.

aeorgnoieang 9y ago

This confuses 'levels', or what exactly is the intelligent 'agent'. The idiot-savant is not generally intelligent. Depending on what you mean by "complex mathematical problems" even the neural components actually responsible for solving the problems might not be intelligent. In the case of actual idiot-savants, I'd suspect those components aren't 'intelligent'; they're too simple. The alien in Blindsight, if it really "runs entirely on instinct", then I'd expect it to do badly in novel situations, unless it's version of "instincts" basically "bake in extremely complex behaviors" that make it functionally equivalent or similar to a human – at the level we're used to dealing with as humans. Also note that most of your examples are fictional, i.e. not actual evidence beyond maybe extremely weak plausibility. AFAIK, every "high intelligence" of which actual people have actual strong evidence is self aware. That seems like it should be an important fact! Also, you're conflating self-awareness with the ability to (verbally) explain, i.e. communicate, thoughts to others. By "self-awareness", I was thinking of an 'intelligent system' including some sub-model of itself in whatever the relevant domain it is we're considering. For a 'general-purpose reproductive-fitness-maximizing-strategy executor' that's intelligent, e.g. us, it seems pretty unlikely that they don't, generally, model themselves. We should probably taboo "self-awareness" as it's pretty obviously too slippery and ambiguous.