5 ms·
They put an LLM in a loop that mimics how people do real math, and it did research-level math. Like humans, it wasn't equally capable across all mathematical d
by ants_everywhere 11mo ago
They put an LLM in a loop that mimics how people do real math, and it did research-level math.
Like humans, it wasn't equally capable across all mathematical domains.
The experiment was set up to mimic mathematicians who are excellent at proving inequalities, bounds, finding optimal solutions, etc. So more like Ramanujan and Erdős in their focus on a computationally-driven and problem-focused approach.
- j2kun 11mo ago> that mimics how people do real math Real people do not do math like AlphaEvolve...
- ants_everywhere 11mo agoWhat do you feel is fundamentally different about the feedback loop in AlphaEvolve compared to, say, Einstein and Grossman repeatedly running calculations until they found the right tensor setup for General Relativity? Or Euler filling his notebooks with computations? Or Ramanujan? Or Newton working out infinite series? Or Kepler, etc etc. They are all doing iterative search with feedback from a function that tells them whether they're getting closer or farther from their goal. They try different things, see what works, and keep the stuff that works.
- youoy 11mo agoThe reward functions in the problems that they proposed alphaevolve are easy. The reward funtions of at least 50% of maths are not. You can say that validating if a proof is correct is a straightforward reward, but the size of interesting theorems over the space of all theorems is very small. And also what does "interesting" could even mean?
- j2kun 11mo ago"All human work is just an iterative search with a feedback function" is the kind of trite and ignorant take that belongs only in the private group chats of VC bros.
- griffzhowl 11mo agoAn LLM as a component of a tool, plus a team of research mathematicians, did research level math
- ants_everywhere 11mo agoYes mathematicians chose problems for the LLM to solve, and then the LLM solved them. That's how we know they were good open problems and that this is research level math.
- griffzhowl 11mo agoThe LLMs generated candidate solutions which were evaluated by a scoring function written by hand by the mathematicians. No LLM produced these results by itself
- ants_everywhere 11mo agoyes the LLM produced the results themselves
- griffzhowl 11mo agoYes, some good ones and some garbage, and the LLMs had no idea which was which. The good solutions were arrived at by an iterative procedure which depended on the scoring function written by the mathematicians, and it seems plenty of other ingenuity besides. It's a fascinating use of LLMs by mathematicians to produce new results, but the LLMs are just one component of the tools used to get the results.