4 ms·

Unless I'm misunderstanding, calling this RSI seems misleading? This looks like an optimization of current training methods, and a good one, but not "RSI" in t

Unless I'm misunderstanding, calling this RSI seems misleading? This looks like an optimization of current training methods, and a good one, but not "RSI" in the sense of a system that can perpetually improve itself forever.

EthanHeilman 19d ago
Does RSI actually mean anything specific anymore? RSI, AGI, at this point seem like buzzwords. Sure AGI has definition that are measurable, say "better than 95% of humans on 95% of intellectual tasks" but if we used that definition we already have AGI and almost no one thinks we have achieved AGI. We use AIs to train AIs which we use to train AIs, why is that not RSI? How much human intervention means that is not RSI?
- ctolsen 19d ago
  So few terms in AI are well defined. We will get ASI via AGI because of RSI but neither of those three things have any definition except pure vibes. I struggle with the argument that RSI doesn't already exist like you say, it's existed since before the term LLM (hey, one that can be defined!) was common parlance. Though the biggest use for those is not superintelligence, it's to serve you ads and get your kids addicted to TikTok.
  - marcosdumay 19d ago
    RSI has always being a well defined name, and you can only have RSI if you have an intelligence capable or creating itself. It has technically existed for a long time (for longer than the name), but only on academical applications for extremely limited intelligences that could only create something like themselves. And that is still the only form that exists today. It was never powerful enough to optimize ads distribution, and all the claims people are pushing around today are plain bullshit.
  - MASNeo 19d ago
    Clearly well-defined must live in a probability bracket. Only humans demand exactitude.
  - computably 19d ago
    Narrow pre-LLM models that spit out content and ad recommendations have never been capable of also suggesting, let alone implementing, self-improvements.
    - ctolsen 19d ago
      The systems that train them do.
- computably 19d ago
  > say "better than 95% of humans on 95% of intellectual tasks" but if we used that definition we already have AGI and almost no one thinks we have achieved AGI What matters isn't 95% of humans, it's 95% of actual professionals. Benchmarking an AI accountant against people with zero accounting experience is worse than worthless.
  - EthanHeilman 17d ago
    That's what 95% is intended to capture. That some level of expertise in an area is should be captured by 95% sample of the population, you could push it to 99% of 99.9%, but you want it to be quantified by a number to avoid arguments that something is not AGI because obscure field or expert exists that AI can not do. 95% better at 95% of the population is already approaching ASI. One could even argue that AGI is 50% better than 50% of the population. All that said, this over rotation on benchmarking misses something critical. What is general intelligence? We assume that humans have it and we assume it is captured by benchmarks on "intellectual tasks", but it is probably the case that general intelligence is based displayed by judgement on uncertain outcomes. Benchmarks by their very nature have certain outcomes, they have a wrong and right answer. Test AIs on questions we don't have the answers to and there is no clear right answer, but there will be at some point in the future. What will the economy do? Which US senators will be be re-elected that polling correctly suggests will not be re-elected. Which US senators, currently not in office, will actually pass bills representing the wishes of their voting base? What published papers will be seen are groundbreaking in 5, 10 15 years? What approach to unifying physics should be taken?
jephs 19d ago
Yeah, this is absolutely not what anyone reasonable is thinking about when they say recursive self-improvement. I'd say it's much closer to the concept of continual learning, but I'm only a few pages deep and haven't groqued it fully yet.
addag 19d ago
Agreed, what I understand from RSI would be models creating new models, or at least upgrading their own weights/architecture. It does not seem to be the case here.
IAmGraydon 19d ago
According to industry leaders, we currently have AGI and RSI in the last month or so. Of course, we've seen zero evidence of any of this and have to take their word for it.
xidong_wu 19d ago
This paper optimizes a controller/policy which will be used to agent itself in the next round
- owenshen24 19d ago
  the use of the phrase here feels very clickbaity tbqh

EthanHeilman 19d ago

Does RSI actually mean anything specific anymore? RSI, AGI, at this point seem like buzzwords. Sure AGI has definition that are measurable, say "better than 95% of humans on 95% of intellectual tasks" but if we used that definition we already have AGI and almost no one thinks we have achieved AGI. We use AIs to train AIs which we use to train AIs, why is that not RSI? How much human intervention means that is not RSI?

ctolsen 19d ago
So few terms in AI are well defined. We will get ASI via AGI because of RSI but neither of those three things have any definition except pure vibes. I struggle with the argument that RSI doesn't already exist like you say, it's existed since before the term LLM (hey, one that can be defined!) was common parlance. Though the biggest use for those is not superintelligence, it's to serve you ads and get your kids addicted to TikTok.
- marcosdumay 19d ago
  RSI has always being a well defined name, and you can only have RSI if you have an intelligence capable or creating itself. It has technically existed for a long time (for longer than the name), but only on academical applications for extremely limited intelligences that could only create something like themselves. And that is still the only form that exists today. It was never powerful enough to optimize ads distribution, and all the claims people are pushing around today are plain bullshit.
- MASNeo 19d ago
  Clearly well-defined must live in a probability bracket. Only humans demand exactitude.
- computably 19d ago
  Narrow pre-LLM models that spit out content and ad recommendations have never been capable of also suggesting, let alone implementing, self-improvements.
  - ctolsen 19d ago
    The systems that train them do.
computably 19d ago
> say "better than 95% of humans on 95% of intellectual tasks" but if we used that definition we already have AGI and almost no one thinks we have achieved AGI What matters isn't 95% of humans, it's 95% of actual professionals. Benchmarking an AI accountant against people with zero accounting experience is worse than worthless.
- EthanHeilman 17d ago
  That's what 95% is intended to capture. That some level of expertise in an area is should be captured by 95% sample of the population, you could push it to 99% of 99.9%, but you want it to be quantified by a number to avoid arguments that something is not AGI because obscure field or expert exists that AI can not do. 95% better at 95% of the population is already approaching ASI. One could even argue that AGI is 50% better than 50% of the population. All that said, this over rotation on benchmarking misses something critical. What is general intelligence? We assume that humans have it and we assume it is captured by benchmarks on "intellectual tasks", but it is probably the case that general intelligence is based displayed by judgement on uncertain outcomes. Benchmarks by their very nature have certain outcomes, they have a wrong and right answer. Test AIs on questions we don't have the answers to and there is no clear right answer, but there will be at some point in the future. What will the economy do? Which US senators will be be re-elected that polling correctly suggests will not be re-elected. Which US senators, currently not in office, will actually pass bills representing the wishes of their voting base? What published papers will be seen are groundbreaking in 5, 10 15 years? What approach to unifying physics should be taken?

jephs 19d ago

Yeah, this is absolutely not what anyone reasonable is thinking about when they say recursive self-improvement. I'd say it's much closer to the concept of continual learning, but I'm only a few pages deep and haven't groqued it fully yet.

addag 19d ago

Agreed, what I understand from RSI would be models creating new models, or at least upgrading their own weights/architecture. It does not seem to be the case here.

IAmGraydon 19d ago

According to industry leaders, we currently have AGI and RSI in the last month or so. Of course, we've seen zero evidence of any of this and have to take their word for it.

xidong_wu 19d ago

This paper optimizes a controller/policy which will be used to agent itself in the next round

owenshen24 19d ago
the use of the phrase here feels very clickbaity tbqh