8 ms·

Hopefully this will finally stop the continuing claims[1] that LLMs can only solve problems they have seen before! If you listen carefully to the people who bu

• mariusor 1y ago
  For the less mathematically inclined of us, what is in that discussion that qualifies as a problem that has not been seen before? (I don't mean this combatively, I'd like to have a more mundane explanation)
  • looobay 1y ago
    It means something that is too out-of-data. For example if you try to make an LLM write a program in a strange or very new language it will struggle in non-trivial tasks.
    • mariusor 1y ago
      I understand what "a new problem for an LLM is", my question is about what in the math discussion qualifies as a one. I see references to "improvements", "optimizing" and what I would describe as "iterating over existing solutions" work, not something that's "new". But as I'm not well versed into maths I was hoping that someone that considers the thread as definite proof for that, like parent seems to be, is capable of offering a dumbed down explanation for the five year olds among us. :)
  • griffzhowl 1y ago
    This is a useful summary given by another poster here https://news.ycombinator.com/item?id=45833892 https://news.ycombinator.com/item?id=45833892 The novel results seem to be incremental improvements on some obscurely-named inequalities that I'm not personally familiar with, but I'm far from this field of maths
    • mariusor 1y ago
      I had seen that post, and frankly it didn't pass my smell test of "LLM solves entirely new problems". But again, I'm not a mathematician to know what "unsolved problems" means in the context of the ones they gave AlphaEvolve, and more than that I'm entirely a skeptic on LLM solves problems that don't exist in their training data, so I'm biased. However the request for details is genuine. :)
• topaz0 1y ago
  I think it's disingenuous to characterize these solutions as "LLMs solving problems", given the dependence on a hefty secondary apparatus to choose optimal solutions from the LLM proposals. And an important point here is that this tool does not produce any optimality proofs, so even if they do find the optimal result, you may not be any closer to showing that that's the case.
  • ineedasername 1y ago
    Well, there's the goal posts moved and a Scotsman denied. It's got an infrastructure in which it operates and "didn't show its work" so it takes an F in maths.
    • wizzwizz4 1y ago
      A random walk can do mathematics, with this kind of infrastructure. Isabelle/HOL has a tool called Sledgehammer, which is the hackiest hack that ever hacked[0], basically amounting to "run a load of provers in parallel, with as much munging as it takes". (Plumbing them together is a serious research contribution, which I'm not at all belittling.) I've yet to see ChatGPT achieve anything like what it's capable of. [0]: https://lawrencecpaulson.github.io/2022/04/13/Sledgehammer.html https://lawrencecpaulson.github.io/2022/04/13/Sledgehammer.h...
      • DroneBetter 1y ago
        yeah but random walks can't improve upon the state of the art on many-dimensional numerical optimisation problems of the nature discussed here, on account of they're easy enough to to implement to have been tried already and had their usefulness exhausted; this does present a meaningful improvement over them in its domain.
        • wizzwizz4 1y ago
          When I see announcements that say "we used a language model for X, and got novel results!", I play a little game where I identify the actual function of the language model in the system, and then replace it with something actually suited for that task. Here, the language model is used as the mutation / crossover component of a search through the space of computer programs. What you really want here is represent the programs using an information-dense scheme, endowed with a pseudoquasimetric such that semantically-similar programs are nearby (and vice versa); then explore the vicinity of successful candidates. Ordinary compression algorithms satisfy "information-dense", but the metrics they admit aren't that great. Something that does work pretty well is embedding the programs into the kind of high-dimensional vector space you get out of a predictive text model: there may be lots of non-programs in the space, but (for a high-quality model) those are mostly far away from the programs, so exploring the neighbourhood of programs won't encounter them often. Because I'm well aware of the flaws of such embeddings, I'd add some kind of token-level fuzzing to the output, biased to avoid obvious syntax errors: that usually won't move the embedding much, but will occasionally jump further (in vector space) than the system would otherwise search. So, an appropriate replacement for this generative language model would be some kind of… generative language model. Which is why I'm impressed by this paper. There are enough other contributions in this paper that slotting a bog-standard genetic algorithm over program source in place of the language model could achieve comparable results; but I wouldn't expect it to be nearly as effective in each generation. If the language model is a particularly expensive part of the runtime (as the paper suggests might be the case), then I expect it's worth trying to replace it with a cruder-but-cheaper bias function; but otherwise, you'd need something more sophisticated to beat it. (P.S.: props for trying to bring this back on-topic, but this subthread was merely about AI hype, not actually about the paper.) Edit: Just read §3.2 of the paper. The empirical observations match the theory I've described here.
• swannodette 1y ago
  I don't see how anything about what's presented here that refutes such claims. This mostly confirms that LLM based approaches need some serious baby-sitting from experts and those experts can derive some value from them but generally with non-trivial levels of effort and non-LLM supported thinking.
  • dpflan 1y ago
    Yes, applied research has yielded the modern expert system, which is really useful to experts who know what they are doing.
    • wizzwizz4 1y ago
      It's not the "modern expert system", unless you're throwing away the existing definition of "expert system" entirely, and re-using the term-of-art to mean "system that has something to do with experts".
      • HarHarVeryFunny 1y ago
        I don't know what the parent was referring to, but IMO "expert system" is one of the more accurate and insightful ways of describing LLMs. An expert system is generically a system of declarative rules, capturing an expert's knowledge, that can be used to solve problems. Traditionally expert systems are symbolic systems, representing the rules in a language such as Prolog, with these rules having been laboriously hand derived, but none of this seems core to the definition. A pre-trained LLM can be considered as an expert system that captures the rules of auto-regressive language generation needed to predict the training data. These rules are represented by the weights of a transformer, and were learnt by SGD rather than hand coded, but so what?
        • wizzwizz4 1y ago
          If you can extract anything resembling a declarative rule from the weights of a transformer, I will put you in for a Turing award. Expert systems are a specific kind of thing (see https://en.wikipedia.org/wiki/Expert_system#Software_architecture https://en.wikipedia.org/wiki/Expert_system#Software_archite...): any definition you've read is a description. If the definition includes GPT models, the definition is imprecise.
• deleted 1y ago
  [deleted]
• ants_everywhere 1y ago
  > Hopefully this will finally stop the continuing claims[1] that LLMs can only solve problems they have seen before! The AlphaEvolve paper has been out since May. I don't think the people making these claims are necessarily primarily motivated by the accuracy of what they're saying.
• woadwarrior01 1y ago
  Please read section 2 of the paper[1] cited in the blog post. LLMs are used as a mutation function in an evolutionary loop. LLMs are certainly an enabler, but IMO, evolutionary optimization is what deserves credit in this case. [1]: https://arxiv.org/abs/2511.02864 https://arxiv.org/abs/2511.02864
  • ants_everywhere 1y ago
    all mathematicians and scientists work with a feedback loop. that's what the scientific method is.
    • omnicognate 1y ago
      Not one that amounts to a literal, pre-supplied objective function that's run on a computer to evaluate their outputs.
      • ants_everywhere 1y ago
        That's exactly how a great deal of research level math is done. In fact all open conjectures can be cast this way: the objective function is just the function that checks whether a written proof is a valid proof of the statement. Is there a solution to this PDE? Is there a solution to this algebraic equation? Is there an optimal solution (i.e. we add an optimality condition to the objective function). Does there exist a nontrivial zero that is not equal to 1/2, etc. I can't tell you how many talks I've seen from mathematicians, including Fields Medal winners, that are heavily driven by computations done in Mathematica notebooks which are then cleaned up and formalized. That means that -- even for problems where we don't know the statement in advance -- the actual leg work is done via the evaluation of computable functions against a (explicit or implicit) objective function.
        • omnicognate 1y ago
          Existence problems are not optimisation problems and can't, AIUI, be tackled by AlphaEvolve. It needs an optimisation function that can be incrementally improved in order to work towards an optimal result, not a binary yes/no. More importantly, a research mathematician is not trapped in a loop, mutating candidates for an evolutionary optimiser loop like the LLM is in AlphaEvolve. They have the agency to decide what questions to explore and can tackle a much broader range of tasks than well-defined optimisation problems, most of which (as the article says) can be approached using traditional optimisation techniques with similar results.
• wizzwizz4 1y ago
  That's not what "world-model" means: see https://en.wiktionary.org/wiki/world_model https://en.wiktionary.org/wiki/world_model. Your [2] is equivocating in an attempt to misrepresent the state-of-the-art. Genie 3 is technically impressive, don't get me wrong, but it's strictly inferior to procedural generation techniques from the 20th century, physics simulation techniques from the 20th century, and PlayStation 2-era graphics engines. (Have you seen the character models in the 2001 PS2 port of Half-Life? That's good enough.)
  • ainch 1y ago
    Inferior in what sense? Genie 3 is addressing a fundamentally different problem to a physics sim or procgen: building a good-enough (and broad-enough) model of the real world to train agents that act in the real world. Sims are insufficient for that purpose, hence the "sim2real" gap that has stymied robotics development for years.
    • wizzwizz4 1y ago
      Genie 3 is inferior in the sense you just described: the sim2real gap would be greater, because it's a less accurate model of the aspects of the world that are relevant to robotics.
• ghm2180 1y ago
  >.. that LLMs can only solve problems they have seen before! This is a reductive argument. The set of problems they are solving are proposals that can be _verified_ quickly and bad solutions can be easily pruned. Software development by a human — and even more so teams — are not those kind of problems because the context cannot efficiently hold (1) Design bias of individuals (2) Slower evolution of "correct" solution and visibility over time. (3) Difficulty in "testing" proposals: You can't build 5 different types of infrastructure proposals by an LLM — which themselves are dozens of small sub proposals — _quickly_
• HarHarVeryFunny 1y ago
  AlphaEvolve isn't an LLM - it's an evolutionary coding agent that uses an LLM for code generation. https://deepmind.google/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/ https://deepmind.google/blog/alphaevolve-a-gemini-powered-co... This is part of Google/DeepMind's "Alpha" branding (AlphaGo, AlphaZero, AlphaFold) of bespoke machine learning solutions to tough problems. It sounds like AlphaEvolve might do well on Chollet's ARC-AGI test, where this sort of program synthesis seems to be the most successful approach. I find Tao's use of "extremize" vs "maximize" a bit jarring - maybe this is a more normal term in mathematics?
  • lupire 1y ago
    Sometimes you want to minimize
• j2kun 1y ago
  Read https://www.argmin.net/p/lore-laundering-machines https://www.argmin.net/p/lore-laundering-machines Given time, we may find out that the solutions in this paper were also in the literature, as was the case in the anecdotes from the linked article :)
  • cantor_S_drug 1y ago
    Then its utility as a best search agent is even more. It proves the statement, LLM will find the needle in the haystack if the needle exists.
    • j2kun 1y ago
      > best search agent Except that, instead of telling you that the idea came from a particular source, it fools you into thinking it came up with the idea. In other words, it cannot be described as "search", but rather "laundering" as the article describes it.