3 ms·

What never was so clear to me in that vision is, how the version n actually makes sure the version n+1 is actually faster and better. Initially there might be

by generic92034 3y ago

What never was so clear to me in that vision is, how the version n actually makes sure the version n+1 is actually faster and better. Initially there might be the easy option of using more tokens/memory/<whatever easily measurable and beneficial factor you can imagine>. But when that becomes impractical, how will the "dumber AI" select between x generated "smarter AIs"? How will it ensure that the newly generated versions are better at all (if there are no easily measurable parameters to be increased)?

travisjungroth 3y ago
You have it optimize towards reasoning tests, not tokens or memory. It’s like if you were optimizing a race car. You make the explicit goal time around the track, not horsepower or weight.
- generic92034 3y ago
  That could end up in getting better and better special-purpose expert systems. How do you create better and better general-purpose AIs this way? What is more, when the AIs are more advanced, it might be challenging to create meaningful tests for them (outside of very specialized domains).
  - travisjungroth 3y ago
    My point with the race analogy was you score it on the outcome (time) not a factor (horsepower). For your concern, just make the outcome broader. Someone will still say it’s not AGI as it does 100 different remote jobs, but just plan on ignoring them. Something that scares me is people don’t want to believe this is where we’re headed or it’s even possible. I say this because I think your concerns are easy enough to address, it makes me think you didn’t try answering your own questions. The idea in this scenario is it’s self optimizing. No reason it can’t make it’s own more specific tests to the general test of “become as smart as possible”. And people can make tests that they’re not smart enough to pass. You just make them as a team and take more time. It could also discover new things and then verify if they’re true, which is easier.
    - generic92034 3y ago
      The "easy" optimization targets I have mentioned (tokens/memory/etc.) seem to confuse you. Ignore them for the main point - how do you create good tests for a more advanced AI than you yourself are (or have control of)? For so many human fields of knowledge and ability it is already extremely hard to find meaningful tests to rank humans. No amount of letting an AI optimize on games with fixed and limited rules will automatically improve them on all the other abilities, which are not easily testable or even measurable. And that difficulty pertains for human intelligence trying to measure or test human intelligence. How much more difficult would it be for humans to measure or test super-human intelligence?
      - travisjungroth 3y ago
        > how do you create good tests for a more advanced AI than you yourself are (or have control of)? I think I answered that. You use more resources to create it than answer it. When creating it, you can have a team take lots of time, use all sorts of tools, etc. When taking it, you constrain time and tools. That’s the general pattern of how to make a question harder than you can answer. There are also lots of trapdoor questions, especially in math. An overly simple example is factoring numbers.
        generic92034 3y ago
        I think that can only work with very small, incremental steps in intelligence, though. Will a ten year old human be able to create a meaningful test for an adult with high intelligence? No matter the resources you give the young one, they usually will not be able to. There also might be thresholds, barriers, were small increments in intelligence are not possible. But that is speculative, I will admit.
        travisjungroth 3y ago
        You don’t need big jumps. You have the previous version write the test for the next. I’m not saying I’ve solved the path to AGI or anything. It’s just that this specific issue of “making a test harder than you can solve” has an answer.
deleted 3y ago
[deleted]

travisjungroth 3y ago

You have it optimize towards reasoning tests, not tokens or memory. It’s like if you were optimizing a race car. You make the explicit goal time around the track, not horsepower or weight.

generic92034 3y ago
That could end up in getting better and better special-purpose expert systems. How do you create better and better general-purpose AIs this way? What is more, when the AIs are more advanced, it might be challenging to create meaningful tests for them (outside of very specialized domains).
- travisjungroth 3y ago
  My point with the race analogy was you score it on the outcome (time) not a factor (horsepower). For your concern, just make the outcome broader. Someone will still say it’s not AGI as it does 100 different remote jobs, but just plan on ignoring them. Something that scares me is people don’t want to believe this is where we’re headed or it’s even possible. I say this because I think your concerns are easy enough to address, it makes me think you didn’t try answering your own questions. The idea in this scenario is it’s self optimizing. No reason it can’t make it’s own more specific tests to the general test of “become as smart as possible”. And people can make tests that they’re not smart enough to pass. You just make them as a team and take more time. It could also discover new things and then verify if they’re true, which is easier.
  - generic92034 3y ago
    The "easy" optimization targets I have mentioned (tokens/memory/etc.) seem to confuse you. Ignore them for the main point - how do you create good tests for a more advanced AI than you yourself are (or have control of)? For so many human fields of knowledge and ability it is already extremely hard to find meaningful tests to rank humans. No amount of letting an AI optimize on games with fixed and limited rules will automatically improve them on all the other abilities, which are not easily testable or even measurable. And that difficulty pertains for human intelligence trying to measure or test human intelligence. How much more difficult would it be for humans to measure or test super-human intelligence?
    - travisjungroth 3y ago
      > how do you create good tests for a more advanced AI than you yourself are (or have control of)? I think I answered that. You use more resources to create it than answer it. When creating it, you can have a team take lots of time, use all sorts of tools, etc. When taking it, you constrain time and tools. That’s the general pattern of how to make a question harder than you can answer. There are also lots of trapdoor questions, especially in math. An overly simple example is factoring numbers.
      - generic92034 3y ago
        I think that can only work with very small, incremental steps in intelligence, though. Will a ten year old human be able to create a meaningful test for an adult with high intelligence? No matter the resources you give the young one, they usually will not be able to. There also might be thresholds, barriers, were small increments in intelligence are not possible. But that is speculative, I will admit.
        travisjungroth 3y ago
        You don’t need big jumps. You have the previous version write the test for the next. I’m not saying I’ve solved the path to AGI or anything. It’s just that this specific issue of “making a test harder than you can solve” has an answer.

deleted 3y ago

[deleted]