6 ms·

Seems like we're getting very close to recursive self-improvement [0]. [0] https://www.lesswrong.com/tag/recursive-self-improvement https://www.lesswrong.com/t

Seems like we're getting very close to recursive self-improvement [0]. [0] https://www.lesswrong.com/tag/recursive-self-improvement https://www.lesswrong.com/tag/recursive-self-improvement

mattnewton 3y ago
I upvoted because this was my first thought too, but reading the abstract and skimming the paper makes me think it’s not really an advance for general recursive improvement. I think the title makes people think this is a text -> model model, when it is really a bunch of model weights -> new model weights optimizer for a specific architecture and problem. Still a potentially very useful idea for learning from a bunch of training runs and very interesting work!
- fnordpiglet 3y ago
  I suspect this is useful for porting one vector space to another which is an open problem when you’ve trained one model with one architecture and need to port it to another architecture without paying the full retraining cost.
deleted 3y ago
[deleted]
GuB-42 3y ago
Doesn't look that different from what we are already doing. For example AlphaGo/AlphaZero/MuZero learn to play board games by playing repeatedly against itself, it is a self improvement loop leading to superhuman play. It was a major breakthrough for the game of Go, and it lead to advances in the field of machine learning, but we are still far from something resembling technological singularity. GANs are another example of self-improvement. It was famous for creating "deep fakes". It works by pitting a fake generator and a fake detector against each other, resulting in a cycle of improvement. It didn't get much further than that, in fact, it is all about attention and transformers now. This is just a way of optimizing parameters, it will not invent new techniques. It can say "put 1000 neurons there, 2000 there, etc...", but it still has to pick from what designers tell it to pick from. It may adjust these parameters better than a human can, leading to more efficient systems, I expect some improvement to existing systems, but not a breaking change.
- pests 3y ago
  Go and Chess still has rules that are hard coded which at least gives a framework to optimize in. What rules do you give an LLM?
  - spangry 3y ago
    Physics.
    - faceplanted 3y ago
      I'm skeptical of the idea that anything is going to derive intelligence from the bottom up, but I'll be super impressed if that's how it goes.
      - spangry 3y ago
        Why not? We started off as single celled organisms and look at where we are now.
  - drdeca 3y ago
    Some sort of "generate descriptions of novel tasks including ways to evaluate performance at those tasks, evaluate quality of the generated tasks+evaluation-metrics, split tasks into subtasks, estimate difficulty of tasks in a way that is is judged on how it compares to a combined estimated difficulty of generated subtasks and to actual success rate and quality" sort of deal?
bamboozled 3y ago
The ai is ready to take off to perfection land
astrange 3y ago
No, this is an example of an existing technique called hypernetworks. It's not "recursive self improvement", which is just a belief that magic is real and you can wish an AI into existence. In particular, this one needs too much training data, and you can't define "improvement" without knowing what to improve to.
- deleted 3y ago
  [deleted]
- killerstorm 3y ago
  > which is just a belief that magic is real Is there a law of thermodynamics which prevents AI from writing code which would train a better AI? Never learned that one in school. And FYI here's OpenAI plan to align superintelligence: "Our goal is to build a roughly human-level automated alignment researcher. We can then use vast amounts of compute to scale our efforts, and iteratively align superintelligence." I guess people working there believe in magic. > and you can wish an AI into existence. Eh? People believe that self-improvement might happen when AI is around human-level.
  - advael 3y ago
    To be honest, I think a lot of smart people are willing to believe in magic when they've demonstrated some strong capability and the people funding their company want magic to happen.
    - killerstorm 3y ago
      It's not magic, though. If AI can do work of a human, it can do work of a human. It's a trivial statement, and inability to see it is a hard cope. Are you gonna to take a bet "AI won't be able to do X in 10 years" for some X which people can learn to do now? If you're unwilling to bet then you believe that AI would plausibly be able to perform any human job, including job of AI researcher.
      - advael 3y ago
philsnow 3y ago
A rare opportunity for the other four-letter comic to be applicable: http://smbc-comics.com/comic/2011-12-13 http://smbc-comics.com/comic/2011-12-13 (Though I suppose this skips Neuralink / step 3 and jumps right to step 4.)
AgentME 3y ago
The real magic of recursive self improvement happens only after you have human-level AI that is able to match and surpass human ability in designing AI architectures. Escape-velocity-breaking recursive self improvement doesn't look like a human-made architecture being trained further, it looks like an AI understanding why transformers/etc were successful and coming up with an advancement over transformers.

Seems like we're getting very close to recursive self-improvement [0]. [0] https://www.lesswrong.com/tag/recursive-self-improvement https://www.lesswrong.com/tag/recursive-self-improvement

mattnewton 3y ago

I upvoted because this was my first thought too, but reading the abstract and skimming the paper makes me think it’s not really an advance for general recursive improvement. I think the title makes people think this is a text -> model model, when it is really a bunch of model weights -> new model weights optimizer for a specific architecture and problem. Still a potentially very useful idea for learning from a bunch of training runs and very interesting work!

fnordpiglet 3y ago
I suspect this is useful for porting one vector space to another which is an open problem when you’ve trained one model with one architecture and need to port it to another architecture without paying the full retraining cost.

deleted 3y ago

[deleted]

GuB-42 3y ago

Doesn't look that different from what we are already doing. For example AlphaGo/AlphaZero/MuZero learn to play board games by playing repeatedly against itself, it is a self improvement loop leading to superhuman play. It was a major breakthrough for the game of Go, and it lead to advances in the field of machine learning, but we are still far from something resembling technological singularity. GANs are another example of self-improvement. It was famous for creating "deep fakes". It works by pitting a fake generator and a fake detector against each other, resulting in a cycle of improvement. It didn't get much further than that, in fact, it is all about attention and transformers now. This is just a way of optimizing parameters, it will not invent new techniques. It can say "put 1000 neurons there, 2000 there, etc...", but it still has to pick from what designers tell it to pick from. It may adjust these parameters better than a human can, leading to more efficient systems, I expect some improvement to existing systems, but not a breaking change.

pests 3y ago
Go and Chess still has rules that are hard coded which at least gives a framework to optimize in. What rules do you give an LLM?
- spangry 3y ago
  Physics.
  - faceplanted 3y ago
    I'm skeptical of the idea that anything is going to derive intelligence from the bottom up, but I'll be super impressed if that's how it goes.
    - spangry 3y ago
      Why not? We started off as single celled organisms and look at where we are now.
- drdeca 3y ago
  Some sort of "generate descriptions of novel tasks including ways to evaluate performance at those tasks, evaluate quality of the generated tasks+evaluation-metrics, split tasks into subtasks, estimate difficulty of tasks in a way that is is judged on how it compares to a combined estimated difficulty of generated subtasks and to actual success rate and quality" sort of deal?

bamboozled 3y ago

The ai is ready to take off to perfection land

astrange 3y ago

No, this is an example of an existing technique called hypernetworks. It's not "recursive self improvement", which is just a belief that magic is real and you can wish an AI into existence. In particular, this one needs too much training data, and you can't define "improvement" without knowing what to improve to.

deleted 3y ago
[deleted]
killerstorm 3y ago
> which is just a belief that magic is real Is there a law of thermodynamics which prevents AI from writing code which would train a better AI? Never learned that one in school. And FYI here's OpenAI plan to align superintelligence: "Our goal is to build a roughly human-level automated alignment researcher. We can then use vast amounts of compute to scale our efforts, and iteratively align superintelligence." I guess people working there believe in magic. > and you can wish an AI into existence. Eh? People believe that self-improvement might happen when AI is around human-level.
- advael 3y ago
  To be honest, I think a lot of smart people are willing to believe in magic when they've demonstrated some strong capability and the people funding their company want magic to happen.
  - killerstorm 3y ago
    It's not magic, though. If AI can do work of a human, it can do work of a human. It's a trivial statement, and inability to see it is a hard cope. Are you gonna to take a bet "AI won't be able to do X in 10 years" for some X which people can learn to do now? If you're unwilling to bet then you believe that AI would plausibly be able to perform any human job, including job of AI researcher.
    - advael 3y ago

philsnow 3y ago

A rare opportunity for the other four-letter comic to be applicable: http://smbc-comics.com/comic/2011-12-13 http://smbc-comics.com/comic/2011-12-13 (Though I suppose this skips Neuralink / step 3 and jumps right to step 4.)

AgentME 3y ago

The real magic of recursive self improvement happens only after you have human-level AI that is able to match and surpass human ability in designing AI architectures. Escape-velocity-breaking recursive self improvement doesn't look like a human-made architecture being trained further, it looks like an AI understanding why transformers/etc were successful and coming up with an advancement over transformers.