5 ms·

Please read section 2 of the paper[1] cited in the blog post. LLMs are used as a mutation function in an evolutionary loop. LLMs are certainly an enabler, but I

Please read section 2 of the paper[1] cited in the blog post. LLMs are used as a mutation function in an evolutionary loop. LLMs are certainly an enabler, but IMO, evolutionary optimization is what deserves credit in this case. [1]: https://arxiv.org/abs/2511.02864 https://arxiv.org/abs/2511.02864

ants_everywhere 11mo ago
all mathematicians and scientists work with a feedback loop. that's what the scientific method is.
- omnicognate 11mo ago
  Not one that amounts to a literal, pre-supplied objective function that's run on a computer to evaluate their outputs.
  - ants_everywhere 11mo ago
    That's exactly how a great deal of research level math is done. In fact all open conjectures can be cast this way: the objective function is just the function that checks whether a written proof is a valid proof of the statement. Is there a solution to this PDE? Is there a solution to this algebraic equation? Is there an optimal solution (i.e. we add an optimality condition to the objective function). Does there exist a nontrivial zero that is not equal to 1/2, etc. I can't tell you how many talks I've seen from mathematicians, including Fields Medal winners, that are heavily driven by computations done in Mathematica notebooks which are then cleaned up and formalized. That means that -- even for problems where we don't know the statement in advance -- the actual leg work is done via the evaluation of computable functions against a (explicit or implicit) objective function.
    - omnicognate 11mo ago
      Existence problems are not optimisation problems and can't, AIUI, be tackled by AlphaEvolve. It needs an optimisation function that can be incrementally improved in order to work towards an optimal result, not a binary yes/no. More importantly, a research mathematician is not trapped in a loop, mutating candidates for an evolutionary optimiser loop like the LLM is in AlphaEvolve. They have the agency to decide what questions to explore and can tackle a much broader range of tasks than well-defined optimisation problems, most of which (as the article says) can be approached using traditional optimisation techniques with similar results.
      - ants_everywhere 11mo ago
        > Existence problems are not optimisation problems Several of the problems were existence problems, such as finding geometric constructions. > It needs an optimisation function that can be incrementally improved in order to work towards an optimal result, not a binary yes/no. This is not correct. The evaluation function is arbitrary. To quote the AlphaEvolve paper: > or example, when wishing to find largest possible graphs satisfying a given property, ℎ invokes the evolved code to generate a graph, checks whether the property holds, and then simply returns the size of the graph as the score. In more complicated cases, the function ℎ might involve performing an evolved search algorithm, or training and evaluating a machine learning model The evaluation function is a black box that outputs metrics. The feedback that you've constructed a graph of size K with some property does not tell you what you need to do to construct a graph of size K + M with the same property. > a research mathematician is not trapped in a loop, mutating candidates for an evolutionary optimiser loop like the LLM is in AlphaEvolve. Yes they are in a loop called the scientific method or the research loop. They try things out and check them. This is a basic condition of anything that does research. > They have the agency to decide what questions to explore This is unrelated to the question of whether LLMs can solve novel problems > most of which (as the article says) can be approached using traditional optimisation techniques with similar results. This is a mischaracterization. The article says that an expert human working with an optimizer might achieve similar results. In practice that's how research is done by humans as I mentioned above: it is human plus computer program. The novelty here is that the LLM replaces the human expert.
  - jcelerier 11mo ago
    I don't know, it feels exactly like how I work
    - omnicognate 11mo ago
      I'll ask you the same question as ants_everywhere, then. What objective function do you give AlphaEvolve to ask it to attempt to prove or disprove the Collatz conjecture?
adastra22 11mo ago
Yes, and that’s what a coding agent is too. Technically an agent is not (just) an LLM, but the two have become synonymous is discussions.
- woadwarrior01 11mo ago
  > that’s what a coding agent is too. Not quite.
  - adastra22 11mo ago
    An AI agent mutates state (the repository) in order to get some metric (e.g. test suites) closer to passing. Same thing, really.
    - woadwarrior01 11mo ago
      An AI agent isn't an evolutionary optimization loop.
      - adastra22 11mo ago
        It is when you’re using it right. The most effective way to use an AI agent is to give it a metric for success (ideally tests, but even a description of success), a state upon which it can effect changes (the repo) and it moves the state towards success by executing tool calls and evaluating the success criteria at each step.

ants_everywhere 11mo ago

all mathematicians and scientists work with a feedback loop. that's what the scientific method is.

omnicognate 11mo ago
Not one that amounts to a literal, pre-supplied objective function that's run on a computer to evaluate their outputs.
- ants_everywhere 11mo ago
  That's exactly how a great deal of research level math is done. In fact all open conjectures can be cast this way: the objective function is just the function that checks whether a written proof is a valid proof of the statement. Is there a solution to this PDE? Is there a solution to this algebraic equation? Is there an optimal solution (i.e. we add an optimality condition to the objective function). Does there exist a nontrivial zero that is not equal to 1/2, etc. I can't tell you how many talks I've seen from mathematicians, including Fields Medal winners, that are heavily driven by computations done in Mathematica notebooks which are then cleaned up and formalized. That means that -- even for problems where we don't know the statement in advance -- the actual leg work is done via the evaluation of computable functions against a (explicit or implicit) objective function.
  - omnicognate 11mo ago
    Existence problems are not optimisation problems and can't, AIUI, be tackled by AlphaEvolve. It needs an optimisation function that can be incrementally improved in order to work towards an optimal result, not a binary yes/no. More importantly, a research mathematician is not trapped in a loop, mutating candidates for an evolutionary optimiser loop like the LLM is in AlphaEvolve. They have the agency to decide what questions to explore and can tackle a much broader range of tasks than well-defined optimisation problems, most of which (as the article says) can be approached using traditional optimisation techniques with similar results.
    - ants_everywhere 11mo ago
      > Existence problems are not optimisation problems Several of the problems were existence problems, such as finding geometric constructions. > It needs an optimisation function that can be incrementally improved in order to work towards an optimal result, not a binary yes/no. This is not correct. The evaluation function is arbitrary. To quote the AlphaEvolve paper: > or example, when wishing to find largest possible graphs satisfying a given property, ℎ invokes the evolved code to generate a graph, checks whether the property holds, and then simply returns the size of the graph as the score. In more complicated cases, the function ℎ might involve performing an evolved search algorithm, or training and evaluating a machine learning model The evaluation function is a black box that outputs metrics. The feedback that you've constructed a graph of size K with some property does not tell you what you need to do to construct a graph of size K + M with the same property. > a research mathematician is not trapped in a loop, mutating candidates for an evolutionary optimiser loop like the LLM is in AlphaEvolve. Yes they are in a loop called the scientific method or the research loop. They try things out and check them. This is a basic condition of anything that does research. > They have the agency to decide what questions to explore This is unrelated to the question of whether LLMs can solve novel problems > most of which (as the article says) can be approached using traditional optimisation techniques with similar results. This is a mischaracterization. The article says that an expert human working with an optimizer might achieve similar results. In practice that's how research is done by humans as I mentioned above: it is human plus computer program. The novelty here is that the LLM replaces the human expert.
- jcelerier 11mo ago
  I don't know, it feels exactly like how I work
  - omnicognate 11mo ago
    I'll ask you the same question as ants_everywhere, then. What objective function do you give AlphaEvolve to ask it to attempt to prove or disprove the Collatz conjecture?

adastra22 11mo ago

Yes, and that’s what a coding agent is too. Technically an agent is not (just) an LLM, but the two have become synonymous is discussions.

woadwarrior01 11mo ago
> that’s what a coding agent is too. Not quite.
- adastra22 11mo ago
  An AI agent mutates state (the repository) in order to get some metric (e.g. test suites) closer to passing. Same thing, really.
  - woadwarrior01 11mo ago
    An AI agent isn't an evolutionary optimization loop.
    - adastra22 11mo ago
      It is when you’re using it right. The most effective way to use an AI agent is to give it a metric for success (ideally tests, but even a description of success), a state upon which it can effect changes (the repo) and it moves the state towards success by executing tool calls and evaluating the success criteria at each step.