5 ms·

Could you explain in detail how that causes the singularity? I’m lost. I saw a cool tool used to make another cool tool. You saw the singularity. Where is this

Could you explain in detail how that causes the singularity? I’m lost. I saw a cool tool used to make another cool tool. You saw the singularity. Where is this logic coming from? Why would terminal access which I’m fairly certain I’ve seen in autogpt, change much.

jimbokun 3y ago
The cool tool makes another cool tool, which in turn makes another cool tool, faster and faster, until we really don't understand at all what the latest cool tool is doing. But it just keeps getting smarter/faster/more effective/whatever it's optimizing for. That's the basic definition of "the singularity".
- gwoolhurme 3y ago
  I don’t see that in this specific case for one thing or how we are closer from before. Code generation and auto correction getting closer is very cool, the singularity needs to be spelled out for me how a tool making a tool will get us there. How does that solve all of the hurdles that go into AGI or the singularity. How does it change a large language model.
- monkpit 3y ago
  Would there be some limit that it approaches over a large number of iterations though?
- generic92034 3y ago
  What never was so clear to me in that vision is, how the version n actually makes sure the version n+1 is actually faster and better. Initially there might be the easy option of using more tokens/memory/<whatever easily measurable and beneficial factor you can imagine>. But when that becomes impractical, how will the "dumber AI" select between x generated "smarter AIs"? How will it ensure that the newly generated versions are better at all (if there are no easily measurable parameters to be increased)?
  - travisjungroth 3y ago
    You have it optimize towards reasoning tests, not tokens or memory. It’s like if you were optimizing a race car. You make the explicit goal time around the track, not horsepower or weight.
    - generic92034 3y ago
      That could end up in getting better and better special-purpose expert systems. How do you create better and better general-purpose AIs this way? What is more, when the AIs are more advanced, it might be challenging to create meaningful tests for them (outside of very specialized domains).
      - travisjungroth 3y ago
        My point with the race analogy was you score it on the outcome (time) not a factor (horsepower). For your concern, just make the outcome broader. Someone will still say it’s not AGI as it does 100 different remote jobs, but just plan on ignoring them. Something that scares me is people don’t want to believe this is where we’re headed or it’s even possible. I say this because I think your concerns are easy enough to address, it makes me think you didn’t try answering your own questions. The idea in this scenario is it’s self optimizing. No reason it can’t make it’s own more specific tests to the general test of “become as smart as possible”. And people can make tests that they’re not smart enough to pass. You just make them as a team and take more time. It could also discover new things and then verify if they’re true, which is easier.
        generic92034 3y ago
        The "easy" optimization targets I have mentioned (tokens/memory/etc.) seem to confuse you. Ignore them for the main point - how do you create good tests for a more advanced AI than you yourself are (or have control of)? For so many human fields of knowledge and ability it is already extremely hard to find meaningful tests to rank humans. No amount of letting an AI optimize on games with fixed and limited rules will automatically improve them on all the other abilities, which are not easily testable or even measurable. And that difficulty pertains for human intelligence trying to measure or test human intelligence. How much more difficult would it be for humans to measure or test super-human intelligence?
c7b 3y ago
The cycle could look something like this: LLMv1 > prompt "get me the code for a better LLM" > paste output to terminal, execute > LLMv2 > repeat. There are lots of different singularity definitions on wikipedia, most focus on 'intelligence' (eg "an upgradable intelligent agent will eventually enter a 'runaway reaction' of self-improvement cycles, each new and more intelligent generation appearing more and more rapidly"). I think focusing on what 'intelligence' really means, or whether any model is 'truly' intelligent can be be a bit of a distraction. So I just emphasized capabilities in a rather generic sense instead. It's clear that LLMs can compete with humans on many tasks (coding, creative writing, medical diagnosis, psychotherapy,...), and it is conceivable that they may surpass most humans on those tasks some day. If we have models that can outperform most or all humans on important professional and everyday tasks, and also produce new models that perform even better, maybe at an accelerating rate, through means that are ultimately not interpretable for us, I'd say that's pretty close to a singularity, regardless of whether they're 'truly' intelligent. Even more so if they pass a duck test (walks like a duck, quacks like a duck,...).
- ZephyrBlu 3y ago
  You can already do this in theory with tools like AutoGPT and BabyAGI. Why hasn't the singularity happened already?
  - gwoolhurme 3y ago
    Exactly AutoGPT is exactly what he is describing but I’m still lost as to what the verbiage is then. Is that the singularity?
    - KaoruAoiShiho 3y ago
      Yes AutoGPT would be the singularity if it had more brain-like subsystems like long and short term memory and is hooked up to real world instruments.
      - ZephyrBlu 3y ago

jimbokun 3y ago

The cool tool makes another cool tool, which in turn makes another cool tool, faster and faster, until we really don't understand at all what the latest cool tool is doing. But it just keeps getting smarter/faster/more effective/whatever it's optimizing for. That's the basic definition of "the singularity".

gwoolhurme 3y ago
I don’t see that in this specific case for one thing or how we are closer from before. Code generation and auto correction getting closer is very cool, the singularity needs to be spelled out for me how a tool making a tool will get us there. How does that solve all of the hurdles that go into AGI or the singularity. How does it change a large language model.
monkpit 3y ago
Would there be some limit that it approaches over a large number of iterations though?
generic92034 3y ago
What never was so clear to me in that vision is, how the version n actually makes sure the version n+1 is actually faster and better. Initially there might be the easy option of using more tokens/memory/<whatever easily measurable and beneficial factor you can imagine>. But when that becomes impractical, how will the "dumber AI" select between x generated "smarter AIs"? How will it ensure that the newly generated versions are better at all (if there are no easily measurable parameters to be increased)?
- travisjungroth 3y ago
  You have it optimize towards reasoning tests, not tokens or memory. It’s like if you were optimizing a race car. You make the explicit goal time around the track, not horsepower or weight.
  - generic92034 3y ago
    That could end up in getting better and better special-purpose expert systems. How do you create better and better general-purpose AIs this way? What is more, when the AIs are more advanced, it might be challenging to create meaningful tests for them (outside of very specialized domains).
    - travisjungroth 3y ago
      My point with the race analogy was you score it on the outcome (time) not a factor (horsepower). For your concern, just make the outcome broader. Someone will still say it’s not AGI as it does 100 different remote jobs, but just plan on ignoring them. Something that scares me is people don’t want to believe this is where we’re headed or it’s even possible. I say this because I think your concerns are easy enough to address, it makes me think you didn’t try answering your own questions. The idea in this scenario is it’s self optimizing. No reason it can’t make it’s own more specific tests to the general test of “become as smart as possible”. And people can make tests that they’re not smart enough to pass. You just make them as a team and take more time. It could also discover new things and then verify if they’re true, which is easier.
      - generic92034 3y ago
        The "easy" optimization targets I have mentioned (tokens/memory/etc.) seem to confuse you. Ignore them for the main point - how do you create good tests for a more advanced AI than you yourself are (or have control of)? For so many human fields of knowledge and ability it is already extremely hard to find meaningful tests to rank humans. No amount of letting an AI optimize on games with fixed and limited rules will automatically improve them on all the other abilities, which are not easily testable or even measurable. And that difficulty pertains for human intelligence trying to measure or test human intelligence. How much more difficult would it be for humans to measure or test super-human intelligence?

c7b 3y ago

The cycle could look something like this: LLMv1 > prompt "get me the code for a better LLM" > paste output to terminal, execute > LLMv2 > repeat. There are lots of different singularity definitions on wikipedia, most focus on 'intelligence' (eg "an upgradable intelligent agent will eventually enter a 'runaway reaction' of self-improvement cycles, each new and more intelligent generation appearing more and more rapidly"). I think focusing on what 'intelligence' really means, or whether any model is 'truly' intelligent can be be a bit of a distraction. So I just emphasized capabilities in a rather generic sense instead. It's clear that LLMs can compete with humans on many tasks (coding, creative writing, medical diagnosis, psychotherapy,...), and it is conceivable that they may surpass most humans on those tasks some day. If we have models that can outperform most or all humans on important professional and everyday tasks, and also produce new models that perform even better, maybe at an accelerating rate, through means that are ultimately not interpretable for us, I'd say that's pretty close to a singularity, regardless of whether they're 'truly' intelligent. Even more so if they pass a duck test (walks like a duck, quacks like a duck,...).

ZephyrBlu 3y ago
You can already do this in theory with tools like AutoGPT and BabyAGI. Why hasn't the singularity happened already?
- gwoolhurme 3y ago
  Exactly AutoGPT is exactly what he is describing but I’m still lost as to what the verbiage is then. Is that the singularity?
  - KaoruAoiShiho 3y ago
    Yes AutoGPT would be the singularity if it had more brain-like subsystems like long and short term memory and is hooked up to real world instruments.
    - ZephyrBlu 3y ago