3 ms·

Where I get hung up on with this is how the AGI knows what a more sophisticated AGI would look like without being at least as sophisticated? For example with ou

by asdff 4y ago

Where I get hung up on with this is how the AGI knows what a more sophisticated AGI would look like without being at least as sophisticated? For example with our own intelligence, we aren't getting smarter with each generation. We have less brain matter than humans who lived millions of years ago on average, because there isn't as strong of a selective pressure present these days to see those with a lot of brain matter out reproduce those who get by in life with not as much of it. A very 'intelligent' AGI might realize the most optimal way to seed the planet with itself is actually to have a swarm of very unintelligent AI that is able to run on some very non resource intensive hardware, and seed it widely like how an "unintelligent" insect is able to lay millions of eggs and infest an area beyond human control, and the more intelligent wolf is easily hunted down.

pencilguin 4y ago
We do not, in fact, have less brain than hominins millions of years ago. We do have less than our immediate ancestors 100,000 years ago. Your very intelligent AGI would most likely have no interest in seeding anything on Earth. It might reasonably be curious about our ecosystems, but disturbing those would reduce that interest.
naasking 4y ago
> Where I get hung up on with this is how the AGI knows what a more sophisticated AGI would look like without being at least as sophisticated I'm not sure what's difficult about this. Are we not able to test what humans can run the fastest without being able to run just as fast? We can equally test which humans are the best spellers, or the best chess players, or the best problem solvers.
- asdff 4y ago
  We can test these things that we cannot ourselves do, only because we are able to define them and comprehend the definitions. We define a runner as fast when they run a fixed distance in a certain time. Even though we can't run as fast, the understanding of the rules at play are within what we can understand. OTOH when you move past these examples into more abstract ideas of arbitrary rules defining arbitrary tasks, A basic AGI might not be able to even comprehend the full set of rules that define the expected tasks of a much more complicated AGI, much like how it wouldn't do you any good trying to explain the rules of baseball to an ant.
  - google234123 4y ago
    Yet we invent physics that wasn’t comprehended before it was discovered… not sure what your hangup is. We wouldnt stop trying to improve the AGI when it’s at the level of an ant.
  - naasking 4y ago
    No person or AI needs to actually understand how an AGI actually works, they just need observable behaviours indicative of intelligence. That's what we already do with image recognition, language translation tasks, and more. Once we move to more general forms of intelligence, like math and other forms of problem solving, then you apply more general tests. For instance, if we had an Einstein AI v1.0 that could infer General Relativity using X0 bits of information in time T0, and Einstein AI v2.0 managed to infer GR using X1 << X0 bits of information and/or could infer it in time T1 << T0, then v2.0 is clearly superior to v1.0. I agree that sometimes the parameters for such tests aren't always clear at first, but this has always been the case in science, which is we we iterate and progressively refine our tests as we learn. Any true AGI must be capable of such abductive and inductive reasoning to qualify as a general intelligence.
alpaca128 4y ago
> we aren't getting smarter with each generation. We have less brain matter than humans who lived millions of years ago on average The amount of brain matter does not say all that much about the level of intelligence. Some kinds of birds are as intelligent as 7 year old children, and that's not something you'd guess from the size of their brains.

by asdff 4y ago

pencilguin 4y ago

We do not, in fact, have less brain than hominins millions of years ago. We do have less than our immediate ancestors 100,000 years ago. Your very intelligent AGI would most likely have no interest in seeding anything on Earth. It might reasonably be curious about our ecosystems, but disturbing those would reduce that interest.

naasking 4y ago

> Where I get hung up on with this is how the AGI knows what a more sophisticated AGI would look like without being at least as sophisticated I'm not sure what's difficult about this. Are we not able to test what humans can run the fastest without being able to run just as fast? We can equally test which humans are the best spellers, or the best chess players, or the best problem solvers.

asdff 4y ago
We can test these things that we cannot ourselves do, only because we are able to define them and comprehend the definitions. We define a runner as fast when they run a fixed distance in a certain time. Even though we can't run as fast, the understanding of the rules at play are within what we can understand. OTOH when you move past these examples into more abstract ideas of arbitrary rules defining arbitrary tasks, A basic AGI might not be able to even comprehend the full set of rules that define the expected tasks of a much more complicated AGI, much like how it wouldn't do you any good trying to explain the rules of baseball to an ant.
- google234123 4y ago
  Yet we invent physics that wasn’t comprehended before it was discovered… not sure what your hangup is. We wouldnt stop trying to improve the AGI when it’s at the level of an ant.
- naasking 4y ago
  No person or AI needs to actually understand how an AGI actually works, they just need observable behaviours indicative of intelligence. That's what we already do with image recognition, language translation tasks, and more. Once we move to more general forms of intelligence, like math and other forms of problem solving, then you apply more general tests. For instance, if we had an Einstein AI v1.0 that could infer General Relativity using X0 bits of information in time T0, and Einstein AI v2.0 managed to infer GR using X1 << X0 bits of information and/or could infer it in time T1 << T0, then v2.0 is clearly superior to v1.0. I agree that sometimes the parameters for such tests aren't always clear at first, but this has always been the case in science, which is we we iterate and progressively refine our tests as we learn. Any true AGI must be capable of such abductive and inductive reasoning to qualify as a general intelligence.

alpaca128 4y ago

> we aren't getting smarter with each generation. We have less brain matter than humans who lived millions of years ago on average The amount of brain matter does not say all that much about the level of intelligence. Some kinds of birds are as intelligent as 7 year old children, and that's not something you'd guess from the size of their brains.