3 ms·

Is it really likely that a "recursive self-improvement" capability would lead to a great acceleration of AIs capabilities? Isn't the preponderant bottleneck in

by g-b-r 4mo ago

Is it really likely that a "recursive self-improvement" capability would lead to a great acceleration of AIs capabilities? Isn't the preponderant bottleneck in improving the models the need to train them at scale to verify the hypotheses, and the time and cost that it takes? Or does someone think that they could get magically able to predict big improvements without training?

dataviz1000 4mo ago
I built a self-learning recursive agent that finds academic research about using options data to trade, re-creates the research, and then probes and tests for gaps and potential strategies testing against over one year of out-of-sample trading data with one of several strategies that beat SPY by 10x. [0] One rule is that if a position is opened using the historical data, it can't close the position until the next morning so it isn't a day trading strategy. I'm curious how this self-learning recursive agent would have preformed in the past 4 months? I don't feel like shelling out $200 to access the data. Do you think that trading strategy will collapse? Whatever the case, if this agent really can perform like that and there isn't a look ahead bias leak in the backtesting (which is definitely a possibility or more likely what happened even though I spent days trying to harden against that), it is game over! [0] https://github.com/adam-s/alphadidactic https://github.com/adam-s/alphadidactic
- thin_carapace 4mo ago
  the amount of strategies that perform good in back testing dwarfs the amount of strategies that perform good in reality
- user43928 4mo ago
  I assume that Claude has access to historical market data. Could this not influence the strategies it implements?
bloaf 4mo ago
I don't think anyone really knows. I consider these scenarios: 1) We stumble onto an algorithmic improvement in intelligence. This isn't just "what humans do but faster", its "better than what humans do". I've got no idea what that might mean (it could be fundamentally different heuristics, it could be that we've got some intellectual blind spot that they cast off). It doesn't matter, the instant this happens AI is smarter than us and we won't be able to keep up. We're intelligencing at O(n^2) and they're doing O(n log(n)). 2) AI gets good enough at physics and engineering that they can really quickly use up all "the room at the bottom" as Feyman put it. They design and build a factory that produces a mystery metal amalgam that computes at some small percentage of the minimum predicted by the Landauer principle, within a few percent of Bremermann's limit. It's not "smarter" its just suddenly tens-of-orders of magnitude faster. But those orders of magnitude matter: there's only 8 billion of us, and there's plenty more than a factor of 10 billion "at the bottom". 3) It turns out that this is a "sum is greater than the parts" situation. No human can be an expert in all subjects, but we eventually build a big enough AI that it is. Turns out, you don't need extreme speed or different algorithms, just knowing everything all at once is enough to catapult AI dramatically beyond our grasp. Always knowing the best statistical test to apply, the best mathematical techniques, and relevant physics means that AI never makes a mistake, and can learn with maximum efficiency.
- Haunt1000 4mo ago
  > 2) AI gets good enough at physics and engineering that they can really quickly use up all "the room at the bottom" as Feyman put it. They design and build a factory that produces a mystery metal amalgam that computes at some small percentage of the minimum predicted by the Landauer principle, within a few percent of Bremermann's limit. It's not "smarter" its just suddenly tens-of-orders of magnitude faster. But those orders of magnitude matter: there's only 8 billion of us, and there's plenty more than a factor of 10 billion "at the bottom". Actually your comment made me sign up for an account just so I could say this is the real reason why AI won't take over in the way you say. This kind of stuff requires an enormous amount of experimentation. You can ask any theoretical physicist or chemist versus an experimental one and the conclusion is the experimental people actually find out what happens and how the great puzzle of the universe is solved. And humans could just refuse to collaborate. But that's the big weakness with AI I think it has no real world knowledge or empirical experience.
  - everyday7732 4mo ago
    What makes you think it would need humans to collaborate with it or that they would refuse? Robotics exist and are improving and humans are actively trying to collaborate with AI to improve it.
- w4der 4mo ago
  > just knowing everything all at once is enough to catapult AI dramatically beyond our grasp But that would still be limited to "knowing everything all at once" _at the time_ this event happened, and as we've been shown time and time again by the fundamental sciences, the boundary between what is known and what we know we don't know is ever expanding. Plus there's everything we don't know we don't know, and an LLM can't know that either. Discussions like this can always end up with a fitting reference to some of Borges' novels and that how at least I tend to think we've hit a wall for now.
g-b-r 4mo ago
Weird, there was a good comment about this but it vanished
themafia 4mo ago
Humans gain knowledge through experiments. Without a physical body it has no chance of performing the same. That it can update it's training weights does not seem particularly significant.
- user43928 4mo ago
  It would not need robotics to experiment with software for machine learning. Would there not be a lot of potential already there?
cat_plus_plus 4mo ago
The preponderant bottleneck is inventing new architectures to make AI actually good at human and superhuman tasks. For example, AI agent harnesses add tool calls and long term state management, allowing AI to autonomously complete complex tasks. Once these are in place, finetuning models with examples of good tool calls helps, but somebody first needed to invent the fundamental capability. Now try to implement humanlike long term memory for AI to be your coworker or life assistant working on tasks that last month. Even if necessary low level technologies are already there, structuring them to be practically useful is non trivial.
redox99 4mo ago
RSI most likely does not exist. At least not in the sci fi sense that AI becomes super intelligent over night. It will be like any other technology. Do computers make it faster and easier to design better computers? Yes. But that doesn't mean a step change overnight.

by g-b-r 4mo ago

dataviz1000 4mo ago

I built a self-learning recursive agent that finds academic research about using options data to trade, re-creates the research, and then probes and tests for gaps and potential strategies testing against over one year of out-of-sample trading data with one of several strategies that beat SPY by 10x. [0] One rule is that if a position is opened using the historical data, it can't close the position until the next morning so it isn't a day trading strategy. I'm curious how this self-learning recursive agent would have preformed in the past 4 months? I don't feel like shelling out $200 to access the data. Do you think that trading strategy will collapse? Whatever the case, if this agent really can perform like that and there isn't a look ahead bias leak in the backtesting (which is definitely a possibility or more likely what happened even though I spent days trying to harden against that), it is game over! [0] https://github.com/adam-s/alphadidactic https://github.com/adam-s/alphadidactic

thin_carapace 4mo ago
the amount of strategies that perform good in back testing dwarfs the amount of strategies that perform good in reality
user43928 4mo ago
I assume that Claude has access to historical market data. Could this not influence the strategies it implements?

bloaf 4mo ago

I don't think anyone really knows. I consider these scenarios: 1) We stumble onto an algorithmic improvement in intelligence. This isn't just "what humans do but faster", its "better than what humans do". I've got no idea what that might mean (it could be fundamentally different heuristics, it could be that we've got some intellectual blind spot that they cast off). It doesn't matter, the instant this happens AI is smarter than us and we won't be able to keep up. We're intelligencing at O(n^2) and they're doing O(n log(n)). 2) AI gets good enough at physics and engineering that they can really quickly use up all "the room at the bottom" as Feyman put it. They design and build a factory that produces a mystery metal amalgam that computes at some small percentage of the minimum predicted by the Landauer principle, within a few percent of Bremermann's limit. It's not "smarter" its just suddenly tens-of-orders of magnitude faster. But those orders of magnitude matter: there's only 8 billion of us, and there's plenty more than a factor of 10 billion "at the bottom". 3) It turns out that this is a "sum is greater than the parts" situation. No human can be an expert in all subjects, but we eventually build a big enough AI that it is. Turns out, you don't need extreme speed or different algorithms, just knowing everything all at once is enough to catapult AI dramatically beyond our grasp. Always knowing the best statistical test to apply, the best mathematical techniques, and relevant physics means that AI never makes a mistake, and can learn with maximum efficiency.

Haunt1000 4mo ago
> 2) AI gets good enough at physics and engineering that they can really quickly use up all "the room at the bottom" as Feyman put it. They design and build a factory that produces a mystery metal amalgam that computes at some small percentage of the minimum predicted by the Landauer principle, within a few percent of Bremermann's limit. It's not "smarter" its just suddenly tens-of-orders of magnitude faster. But those orders of magnitude matter: there's only 8 billion of us, and there's plenty more than a factor of 10 billion "at the bottom". Actually your comment made me sign up for an account just so I could say this is the real reason why AI won't take over in the way you say. This kind of stuff requires an enormous amount of experimentation. You can ask any theoretical physicist or chemist versus an experimental one and the conclusion is the experimental people actually find out what happens and how the great puzzle of the universe is solved. And humans could just refuse to collaborate. But that's the big weakness with AI I think it has no real world knowledge or empirical experience.
- everyday7732 4mo ago
  What makes you think it would need humans to collaborate with it or that they would refuse? Robotics exist and are improving and humans are actively trying to collaborate with AI to improve it.
w4der 4mo ago
> just knowing everything all at once is enough to catapult AI dramatically beyond our grasp But that would still be limited to "knowing everything all at once" _at the time_ this event happened, and as we've been shown time and time again by the fundamental sciences, the boundary between what is known and what we know we don't know is ever expanding. Plus there's everything we don't know we don't know, and an LLM can't know that either. Discussions like this can always end up with a fitting reference to some of Borges' novels and that how at least I tend to think we've hit a wall for now.

g-b-r 4mo ago

Weird, there was a good comment about this but it vanished

themafia 4mo ago

Humans gain knowledge through experiments. Without a physical body it has no chance of performing the same. That it can update it's training weights does not seem particularly significant.

user43928 4mo ago
It would not need robotics to experiment with software for machine learning. Would there not be a lot of potential already there?

cat_plus_plus 4mo ago

The preponderant bottleneck is inventing new architectures to make AI actually good at human and superhuman tasks. For example, AI agent harnesses add tool calls and long term state management, allowing AI to autonomously complete complex tasks. Once these are in place, finetuning models with examples of good tool calls helps, but somebody first needed to invent the fundamental capability. Now try to implement humanlike long term memory for AI to be your coworker or life assistant working on tasks that last month. Even if necessary low level technologies are already there, structuring them to be practically useful is non trivial.

redox99 4mo ago

RSI most likely does not exist. At least not in the sci fi sense that AI becomes super intelligent over night. It will be like any other technology. Do computers make it faster and easier to design better computers? Yes. But that doesn't mean a step change overnight.