3 ms·

> I will continue dismissing AGI until someone can tell me what GI is Here's a rather famous recommendation for a definition of general intelligence: https://a

> I will continue dismissing AGI until someone can tell me what GI is Here's a rather famous recommendation for a definition of general intelligence: https://arxiv.org/pdf/0712.3329.pdf https://arxiv.org/pdf/0712.3329.pdf Very loosely paraphrasing their paper, general intelligence is the ability to adapt to novel situations and predict the future better than any other algorithm could do. Personally, I think of perfect AGI as this: given that our universe is capable of approximation to arbitrary precision using a universal Turing machine (and infinite time and memory), AGI is the best* computable approximation to Solomonoff induction. *"best" being some mathematical notion of optimality.

idontwantthis 5y ago
So are humans not GI then? We are terrible at predicting the future, in general. "Frustration" is the feeling of things not going how you thought they would or should.
eru 5y ago
> Very loosely paraphrasing their paper, general intelligence is the ability to adapt to novel situations and predict the future better than any other algorithm could do. The 'better than any other algorithm' part seems to make this into a pretty weird definition? By that standard only John von Neumann [0] was intelligent, because every other human wasn't as smart as him? I suspect any reasonable definition of intelligence has to work on a 'good enough' basis, not on optimality. The paper you linked is quite interesting, and I'll be reading it now. I suspect your summary was perhaps a bit too brisk, and their more careful longer definition survives this trivial objection? [0] Replace von Neumann with your favourite genius.
- idontwantthis 5y ago
  Humans are terrible at pretty much everything except what they are specifically highly trained in. von Neuman was a genius, but he also actively promoted nuclear war. Does that mean he is or is not intelligent? Do you think in a world where von Neuman got to nuke the USSR in 1950, we would be happy we gave him the keys?
  - eru 5y ago
    Ignore von Neumann, he was just an example. Basically, with the definition as given in the comment, only whatever matches up with the smartest thing around would count as intelligent. (Whether that's the smartest thing currently around, or the smartest thing possible even in theory isn't quite clear.) In any case, von Neumann was definitely extremely intelligent for a human. Don't conflate intelligence with wisdom, or with having goals that align with yours.
novok 5y ago
I didn't read the paper, but going along with the sun metaphor, your description kind of sounds like "hot ball in sky that radiates light". It still doesn't describe fusion.
aqsalose 5y ago
Quoting the key definition: "Bringing all these pieces together, we can now define our formal measure of intelligence for arbitrary systems. Let E be the space of all computable reward summable environmental measures with respect to the reference machine U, and let K be the Kolmogorov complexity function. The expected performance of agent π with respect to the universal distribution 2−K(μ) over the space of all environments E is given by Υ(π) := [unquotable latex expression]. We call this the universal intelligence of agent π." It is an useless kind of mathematical object from any practical perspective. Sure, some formal object is defined, but all the actual important things to use it are left as difficult, possibly devilishly or impossibly difficult implementation details. How do you actually build a map from the physical reality to environment E? How the agent π is mapped to any real physical being? All the real work that would make the definition useful is left out. It is like defining complexity classes without providing any algorithms. edit. Further complaint: Here is how they present "examples": >A very specialised agent. From the equation for Υ, we see that an agent could have very low universal intelligence but still perform extremely well at a few very specific and complex tasks. Consider, for example, IBM’s Deep Blue chess supercomputer, which we will represent by π_dblue. When μ_chess describes the game of chess, V π_dblue μ_chess is very high. However 2−K(μ_chess ) is small, and for μ != μ_chess the value function will be low as π_dblue only plays chess. Therefore, the value of Υ(π_dblue) will be very low. Intuitively, this is because Deep Blue is too inflexible and narrow to have general intelligence; a characteristic weakness of specialised artificial intelligence systems. Why they need any formal definition of Y to express this if they don't bother fleshing out some features of μ_chess so that you could provide any bounds for K(μ_chess)? Without such work, all of the actual claims in the paragraph is all appeal to intuition. Nothing is proved, so any formal definitions are unused. The contents of the quoted paragraph could be expressed without referring to any equations or "definitions" at all, they are totally superfluous.

idontwantthis 5y ago

So are humans not GI then? We are terrible at predicting the future, in general. "Frustration" is the feeling of things not going how you thought they would or should.

eru 5y ago

> Very loosely paraphrasing their paper, general intelligence is the ability to adapt to novel situations and predict the future better than any other algorithm could do. The 'better than any other algorithm' part seems to make this into a pretty weird definition? By that standard only John von Neumann [0] was intelligent, because every other human wasn't as smart as him? I suspect any reasonable definition of intelligence has to work on a 'good enough' basis, not on optimality. The paper you linked is quite interesting, and I'll be reading it now. I suspect your summary was perhaps a bit too brisk, and their more careful longer definition survives this trivial objection? [0] Replace von Neumann with your favourite genius.

idontwantthis 5y ago
Humans are terrible at pretty much everything except what they are specifically highly trained in. von Neuman was a genius, but he also actively promoted nuclear war. Does that mean he is or is not intelligent? Do you think in a world where von Neuman got to nuke the USSR in 1950, we would be happy we gave him the keys?
- eru 5y ago
  Ignore von Neumann, he was just an example. Basically, with the definition as given in the comment, only whatever matches up with the smartest thing around would count as intelligent. (Whether that's the smartest thing currently around, or the smartest thing possible even in theory isn't quite clear.) In any case, von Neumann was definitely extremely intelligent for a human. Don't conflate intelligence with wisdom, or with having goals that align with yours.

novok 5y ago

I didn't read the paper, but going along with the sun metaphor, your description kind of sounds like "hot ball in sky that radiates light". It still doesn't describe fusion.

aqsalose 5y ago

Quoting the key definition: "Bringing all these pieces together, we can now define our formal measure of intelligence for arbitrary systems. Let E be the space of all computable reward summable environmental measures with respect to the reference machine U, and let K be the Kolmogorov complexity function. The expected performance of agent π with respect to the universal distribution 2−K(μ) over the space of all environments E is given by Υ(π) := [unquotable latex expression]. We call this the universal intelligence of agent π." It is an useless kind of mathematical object from any practical perspective. Sure, some formal object is defined, but all the actual important things to use it are left as difficult, possibly devilishly or impossibly difficult implementation details. How do you actually build a map from the physical reality to environment E? How the agent π is mapped to any real physical being? All the real work that would make the definition useful is left out. It is like defining complexity classes without providing any algorithms. edit. Further complaint: Here is how they present "examples": >A very specialised agent. From the equation for Υ, we see that an agent could have very low universal intelligence but still perform extremely well at a few very specific and complex tasks. Consider, for example, IBM’s Deep Blue chess supercomputer, which we will represent by π_dblue. When μ_chess describes the game of chess, V π_dblue μ_chess is very high. However 2−K(μ_chess ) is small, and for μ != μ_chess the value function will be low as π_dblue only plays chess. Therefore, the value of Υ(π_dblue) will be very low. Intuitively, this is because Deep Blue is too inflexible and narrow to have general intelligence; a characteristic weakness of specialised artificial intelligence systems. Why they need any formal definition of Y to express this if they don't bother fleshing out some features of μ_chess so that you could provide any bounds for K(μ_chess)? Without such work, all of the actual claims in the paragraph is all appeal to intuition. Nothing is proved, so any formal definitions are unused. The contents of the quoted paragraph could be expressed without referring to any equations or "definitions" at all, they are totally superfluous.