4 ms·

Self-Taught Optimizer (Stop): Recursively Self-Improving Code Generation

49 points by birriel 3y ago

norir 3y ago
My crude reaction to this piece is that it feels like they are describing a technique for finding a local maximum for any coding problem. It is an unsurprising result but it seems like it would be hard to avoid getting stuck in a low valley without human intervention. I also believe that for any deterministic program, the best human written program will converge on the optimal solution given the real world matetial and temporal constraints with or without LLMs. I am not sure if LLMs will be in wide use in 10 years, but they will certainly help us humans write better code if only to avoid getting stuck in the traps they introduce.
- j_maffe 3y ago
  For the local maximum issue, there's plenty of optimization algorithms that deal with this issue which I think are perfectly viable for this case scenario.
- seeknotfind 3y ago
  Hmmm, I mean there is a more general question: in the space of algorithms, how continuous are the paths between different solutions? For instance, if you start with bubble sort, I don't think it's at all clear this algorithm can take small finite steps to turn into improved to merge sort. This is the low valley you are talking about. This also aligns nicely with the results in the paper. GPT-4 improves, so it's perhaps good enough to make larger steps or transformations of an algorithm, whereas weaker models don't. The real interesting result would be if they improved an algorithm beyond what people have been able to do. Otherwise it's just interesting and a peek into the future..
ganzuul 3y ago
1. This search space is too large to hit anything by firing shots in the dark. 2. Results will either be completely redundant or ethically questionable. 3. The resources used for optimization should not exceed what performance may be gained. There is no route around full comprehension of this problem before you can solve it. It is a philosophical event horizon.
- KRAKRISMOTT 3y ago
  > 1. This search space is too large to hit anything by firing shots in the dark. 2. Results will either be completely redundant or ethically questionable. 3. The resources used for optimization should not exceed what performance may be gained. There is no route around full comprehension of this problem before you can solve it. It is a philosophical event horizon. My sweet summer child http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html
  - ganzuul 3y ago
    That links to a spiritual quest described in the language of information theory. It is looking for what makes us different from machines instead of what makes us similar. It is an ego talking about predictions that did not pan out. Rookie mistake.
    - KRAKRISMOTT 3y ago
      That's not a spiritual quest, that's an empirical observation of what worked and what failed. Expert systems without the guidance of statistical learning has gone nowhere. The 90s Japan is calling and they want you back to wage war against the AI winter and make their fifth generation computing project great again.
- inasio 3y ago
  People routinely solve quadratic assignment problems (perhaps not provably optimal, but finding good solutions within a small bound of optimality) where the search space is greater than 10^3000 [0], using relatively simple heuristics like simulated annealing. [0] e.g. any of the QAPLIB instances in Goh et al. Proceedings of the 2022 Genetic and Evolutionary Computation Conference
Havoc 3y ago
Vaguely unrelated - the tech youtubers in LLM space are becoming really fast. Heard about this paper there before hn. Normally they're a good day or two behind
- dankhat 3y ago
  could you please provide us with some recommendations?

49 points by birriel 3y ago

My crude reaction to this piece is that it feels like they are describing a technique for finding a local maximum for any coding problem. It is an unsurprising result but it seems like it would be hard to avoid getting stuck in a low valley without human intervention. I also believe that for any deterministic program, the best human written program will converge on the optimal solution given the real world matetial and temporal constraints with or without LLMs. I am not sure if LLMs will be in wide use in 10 years, but they will certainly help us humans write better code if only to avoid getting stuck in the traps they introduce.

j_maffe 3y ago
For the local maximum issue, there's plenty of optimization algorithms that deal with this issue which I think are perfectly viable for this case scenario.
seeknotfind 3y ago
Hmmm, I mean there is a more general question: in the space of algorithms, how continuous are the paths between different solutions? For instance, if you start with bubble sort, I don't think it's at all clear this algorithm can take small finite steps to turn into improved to merge sort. This is the low valley you are talking about. This also aligns nicely with the results in the paper. GPT-4 improves, so it's perhaps good enough to make larger steps or transformations of an algorithm, whereas weaker models don't. The real interesting result would be if they improved an algorithm beyond what people have been able to do. Otherwise it's just interesting and a peek into the future..

ganzuul 3y ago

1. This search space is too large to hit anything by firing shots in the dark. 2. Results will either be completely redundant or ethically questionable. 3. The resources used for optimization should not exceed what performance may be gained. There is no route around full comprehension of this problem before you can solve it. It is a philosophical event horizon.

KRAKRISMOTT 3y ago
> 1. This search space is too large to hit anything by firing shots in the dark. 2. Results will either be completely redundant or ethically questionable. 3. The resources used for optimization should not exceed what performance may be gained. There is no route around full comprehension of this problem before you can solve it. It is a philosophical event horizon. My sweet summer child http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html
- ganzuul 3y ago
  That links to a spiritual quest described in the language of information theory. It is looking for what makes us different from machines instead of what makes us similar. It is an ego talking about predictions that did not pan out. Rookie mistake.
  - KRAKRISMOTT 3y ago
    That's not a spiritual quest, that's an empirical observation of what worked and what failed. Expert systems without the guidance of statistical learning has gone nowhere. The 90s Japan is calling and they want you back to wage war against the AI winter and make their fifth generation computing project great again.
inasio 3y ago
People routinely solve quadratic assignment problems (perhaps not provably optimal, but finding good solutions within a small bound of optimality) where the search space is greater than 10^3000 [0], using relatively simple heuristics like simulated annealing. [0] e.g. any of the QAPLIB instances in Goh et al. Proceedings of the 2022 Genetic and Evolutionary Computation Conference

Havoc 3y ago

Vaguely unrelated - the tech youtubers in LLM space are becoming really fast. Heard about this paper there before hn. Normally they're a good day or two behind

dankhat 3y ago
could you please provide us with some recommendations?