25 ms·

Epoch confirms GPT5.4 Pro solved a frontier math open problem

480 points by in-silico 7mo ago

an0malous 7mo ago
I feel like there’s a fork in our future approaching where we’ll either blossom into a paradise for all or live under the thumb of like 5 immortal VCs
- XCSme 7mo ago
  Change is always hard, even if it will be good in 20 years, the transitions are always tough.
  - reverius42 7mo ago
    Sometimes the transition is tough and then the end state is also worse! Hoping that won't be the case with AI but we may need some major societal transformations to prevent it.
OhioMan2943 7mo ago
What are the odds that this is because Openai is pouring more money into high publicity stunts like this- rather than its model actually being better than Anthropics?
6thbit 7mo ago
> Subsequent to this solve, we finished developing our general scaffold for testing models on FrontierMath: Open Problems. In this scaffold, several other models were able to solve the problem as well: Opus 4.6 (max), Gemini 3.1 Pro, and GPT-5.4 (xhigh). Interesting. Whats that “scaffold”? A sort of unit test framework for proofs?
- inkysigma 7mo ago
  I think in this context, scaffolds are generally the harness that surrounds the actual model. For example, any tools, ways to lay out tasks, or auto-critiquing methods. I think there's quite a bit of variance in model performance depending on the scaffold so comparisons are always a bit murky.
  - readitalready 7mo ago
    Usually involves a lot of agents and their custom contexts or system prompts.
karmasimida 7mo ago
No denial at this point, AI could produce something novel, and they will be doing more of this moving forward.
- leptons 7mo ago
  [flagged]
  - snypher 7mo ago
    Your analogy falls apart if we consider the number wasn't on the clock face.
    - MattGaiser 7mo ago
      I am deeply baffled by AI denial at this point.
      - wtallis 7mo ago
        Complete denial that AI/LLMs can produce novel, good things is an indefensible stance at this point. But the large volume of AI slop is still an unsolved problem, and the claim that "AI will still mostly deliver slop" seems to be almost certainly correct in the near-term. We've had a few decades to address email spam, and still haven't manage to disincentivize it enough to stop being the main challenge for email as a communication medium. I don't think there's much hope that we'll be able to disincentive the widespread, large-scale creation of AI slop even after more expensive models with higher-quality output are available.
      - bigstrat2003 7mo ago
        It's quite simple: it has yet to show it can actually be useful, and all the claims that it can have (so far) turned out to be self delusion if not deliberate lies. When the industry is run by grifters, you shouldn't really be surprised when people stop believing them.
        7mo ago
osti 7mo ago
Seems like the high compute parallel thinking models weren't even needed, both the normal 5.4 and gemini 3.1 pro solved it. Somehow Gemini 3 deepthink couldn't solve it.
renewiltord 7mo ago
Fantastic news! That means with the right support tooling existing models are already capable of solving novel mathematics. There’s probably a lot of good mathematics out there we are going to make progress on.
pinkmuffinere 7mo ago
As someone with only passing exposure to serious math, this section was by far the most interesting to me: > The author assessed the problem as follows. > [number of mathematicians familiar, number trying, how long an expert would take, how notable, etc] How reliably can we know these things a-priori? Are these mostly guesses? I don't mean to diminish the value of guesses; I'm curious how reliable these kinds of guesses are.
- ramblingrain 7mo ago
  Read about Paul Erdös... not all math is the Riemann Hypothesis, there is yeoman's work connecting things and whatever...
- qnleigh 7mo ago
  For number of mathematicians familiar with and actively working on the problem, modern mathematics research is incredibly specialized, so it's easy to keep track of who's working on similar problems. You read each other's papers, go to the same conferences etc. For "how long an expert would take" to solve a problem, for truly open problems I don't think you can usually answer this question with much confidence until the problem has been solved. But once it has been solved, people with experience have a good sense of how long it would have taken them (though most people underestimate how much time they need, since you always run into unanticipated challenges).
- jasonfarnon 7mo ago
  Certainly knowing how many/which people are working on a problem you are looking at, and how long it will take you to solve it, are critical skills in being a working researcher. What kind of answer are you looking for? It's hard to quantify. Most suck at this type of assessment as a PhD student and then you get better as time goes on.
johnfn 7mo ago
I like to imagine that the number of consumed tokens before a solution is found is a proxy for how difficult a problem is, and it looks like Opus 4.6 consumed around 250k tokens. That means that a tricky React refactor I did earlier today at work was about half as hard as an open problem in mathematics! :)
- sublinear 7mo ago
  You might be joking, but you're probably also not that far off from reality. I think more people should question all this nonsense about AI "solving" math problems. The details about human involvement are always hazy and the significance of the problems are opaque to most. We are very far away from the sensationalized and strongly implied idea that we are doing something miraculous here.
  - johnfn 7mo ago
    I am kind of joking, but I actually don't know where the flaw in my logic is. It's like one of those math proofs that 1 + 1 = 3. If I were to hazard a guess, I think that tokens spent thinking through hard math problems probably correspond to harder human thought than tokens spend thinking through React issues. I mean, LLMs have to expend hundreds of tokens to count the number of r's in strawberry. You can't tell me that if I count the number of r's in strawberry 1000 times I have done the mental equivalent of solving an open math problem.
    - throw310822 7mo ago
      You can spend countless "tokens" solving minesweeper or sudoku. This doesn't mean that you solved difficult problems: just that the solutions are very long and, while each step requires reasoning, the difficulty of that reasoning is capped.
      - p1necone 7mo ago
        A lot of math problems/proofs are like minesweeper or sudoku in a way though. They're a long series of individually kinda simple logical deductions that eventually result in a solution. Some really hard problems are only really hard because each one of those "simple" deductions requires you to have expert knowledge in some disparate area to make that leap.
alberth 7mo ago
For those, like me, who find the prompt itself of interest … > A full transcript of the original conversation with GPT-5.4 Pro can be found here [0] and GPT-5.4 Pro’s write-up from the end of that transcript can be found here [1]. [0] https://epoch.ai/files/open-problems/gpt-5-4-pro-hypergraph-ramsey.txt https://epoch.ai/files/open-problems/gpt-5-4-pro-hypergraph-... [1] https://epoch.ai/files/open-problems/hypergraph-ramsey-gpt-5-4-pro-solution.pdf https://epoch.ai/files/open-problems/hypergraph-ramsey-gpt-5...
- dyauspitr 7mo ago
  I wonder what was in that solutions file they provided. According to the prompt it’s a solution template but I want to know the contents. Another thing I want to know is how the user keeps updating the LLM with the token usage. I didn’t know they could process additional context midtask like that.
tombert 7mo ago
I was trying to get Claude and Codex to try and write a proof in Isabelle for the Collatz conjecture, but annoyingly it didn't solve it, and I don't feel like I'm any closer than I was when I started. AI is useless! In all seriousness, this is pretty cool. I suspect that there's a lot of theoretical math that haven't been solved simply because of the "size" of the proof. An AI feedback loop into something like Isabelle or Lean does seem like it could end up opening up a lot of proofs.
- qnleigh 7mo ago
  I got Gemini to find a polynomial-time algorithm for integer factoring, but then I mysteriously got locked out of my Google account. They should at least refund me the tokens.
  - fxtentacle 7mo ago
    That sounds like the start of a very lucrative career. Are you sure it was Gemini and not an AI competitor offering affiliate commission? ;)
daveguy 7mo ago
New goalpost, and I promise I'm not being facetious at all, genuinely curious: Can an AI pose an frontier math problem that is of any interest to mathematicians? I would guess 1) AI can solve frontier math problems and 2) can pose interesting/relevant math problems together would be an "oh shit" moment. Because that would be true PhD level research.
- EternalFury 7mo ago
  Yes. I doubt it can do that.
- kgeist 7mo ago
  Considering that an LLM simply remixes what it finds in its learned distribution over text, it's possible that it can pose new math problems by identifying gaps ("obvious" in restrospect) that humans may have missed (like connecting two known problems to pose a new one). What LLMs can't currently do is pose new problems by observing the real world and its ramifications, like that moving sofa problem.
vlinx 7mo ago
This is a remarkable result if confirmed independently. The gap between solving competition problems and open research problems has always been significant - bridging that gap suggests something qualitatively different in the model capabilities.
measurablefunc 7mo ago
I guess this means AI researchers should be out of jobs very soon.
pugchat 7mo ago
[dead]
Validark 7mo ago
I have long said I am an AI doubter until AI could print out the answers to hard problems or ones requiring tons of innovation. Assuming this is verified to be correct (not by AI) then I just became a believer. I would like to see a few more AI inventions to know for sure, but wow, it really is a new and exciting world. I really hope we use this intelligence resource to make the world better.
- himata4113 7mo ago
  It's less of solving a problem, but trying every single solution until one works. Exhaustive search pretty much. It's pretty much how all the hard problems are solved by AI from my experience.
  - lsc4719 7mo ago
    That's also the only way how humans solve hard problems.
    - jMyles 7mo ago
      There have been both inductive and deductive solutions to open math problems by humans in the past decade, including to fairly high-profile problems.
    - himata4113 7mo ago
      Not always, humans are a lot better at poofing a solution into existence without even trying or testing. It's why we have the scientific method: we come up with a process and verify it, but more often than not we already know that it will work. Compared to AI, it thinks of every possible scientific method and tries them all. Not saying that humans never do this as well, but it's mostly reserved for when we just throw mud at a wall and see what sticks.
      - virgildotcodes 7mo ago
        More often than not, far, far, far more often than not, we do not already know that it will work. For all human endeavors, from the beginning of time. If we get to any sort of confidence it will work it is based on building a history of it, or things related to "it" working consistently over time, out of innumerable other efforts where other "it"s did not work.
data_maan 7mo ago
A model to whose internals we don't have access solved a problem we didn't knew was in their datasets. Great, I'm impressed
gnarlouse 7mo ago
We only get one shot.
virgildotcodes 7mo ago
I don't know why I am still perpetually shocked that the default assumption is that humans are somehow unique. It's this pervasive belief that underlies so much discussion around what it means to be intelligent. The null hypothesis goes out the window. People constantly make comments like "well it's just trying a bunch of stuff until something works" and it seems that they do not pause for a moment to consider whether or not that also applies to humans. If they do, they apply it in only the most restrictive way imaginable, some 2 dimensional caricature of reality, rather than considering all the ways that humans try and fail in all things throughout their lifetimes in the process of learning and discovery. There's still this seeming belief in magic and human exceptionalism, deeply held, even in communities that otherwise tend to revolve around the sciences and the empirical.
- slopinthebag 7mo ago
  > I don't know why I am still perpetually shocked that the default assumption is that humans are somehow unique. Uh, because up until and including now, we are...?
  - virgildotcodes 7mo ago
    Every living thing on Earth is unique. Every rock is unique in virtually infinite ways from the next otherwise identical rock. There are also a tremendous number of similarities between all living things and between rocks (and between rocks and living things). Most ways in which things are unique are arguably uninteresting. The default mode, the null hypothesis should be to assume that human intelligence isn't interestingly unique unless it can be proven otherwise. In these repeated discussions around AI, there is criticism over the way an AI solves a problem, without any actual critical thought about the way humans solve problems. The latter is left up to the assumption that "of course humans do X differently" and if you press you invariably end up at something couched in a vague mysticism about our inner-workings. Humans apparently create something from nothing, without the recombination of any prior knowledge or outside information, and they get it right on the first try. Through what, divine inspiration from the God who made us and only us in His image?
qnleigh 7mo ago
Their 'Open Problems page' linked below gives some interesting context. They list 15 open problems in total, categorized as 'moderately interesting,' 'solid result,' 'major advance,' or 'breakthrough.' The solved problem is listed as 'moderately interesting,' which is presumably the easiest category. But it's notable that the problem was selected and posted here before it was solved. I wonder how long until the other 3 problems in this category are solved. https://epoch.ai/frontiermath/open-problems https://epoch.ai/frontiermath/open-problems
- fnordpiglet 7mo ago
  I’d hope this isn’t a goal post move - an open math problem of any sort being solved by a language model is absolute science fiction.
  - zozbot234 7mo ago
    That's been achieved already with a few Erdös problems, though those tended to be ambiguously stated in a way that made them less obviously compelling to humans. This problem is obscure, even the linked writeup admits that perhaps ~10 mathematicians worldwide are genuinely familiar with it. But it's not unfeasibly hard for a few weeks' or months' work by a human mathematician.
    - utopiah 7mo ago
      FWIW https://github.com/teorth/erdosproblems/wiki/AI-contributions-to-Erd%C5%91s-problems https://github.com/teorth/erdosproblems/wiki/AI-contribution... in particular the disclaimers are very interesting.
  - tovej 7mo ago
    It is not. You're operating under the assumption that all open math problems are difficult and novel. This particular problem was about improving the lower bound for a function tracking a property of hypergraphs (undirected graphs where edges can contain more than two vertices). Both constructing hypergraphs (sets) and lower bounds are very regular, chore type tasks that are common in maths. In other words, there's plenty of this type of proof in the training data. LLMs kind of construct proofs all the time, every time they write a program. Because every program has a corresponding proof. It doesn't mean they're reasoning about them, but they do construct proofs. This isn't science fiction. But it's nice that the LLMs solved something for once.
deleted 7mo ago
[deleted]
bandrami 7mo ago
It's deeply surprising to me that LLMs have had more success proving higher math theorems than making successful consumer software
- 59nadir 7mo ago
  Software developers have spent decades at this point discounting and ignoring almost all objective metrics for software quality and the industry as a whole has developed a general disregard for any metric that isn't time-to-ship (and even there they will ignore faster alternatives in favor of hyped choices). (Edit: Yes, I'm aware a lot of people care about FP, "Clean Code", etc., but these are all red herrings that don't actually have anything to do with quality. At best they are guidelines for less experienced programmers and at worst a massive waste of time if you use more than one or two suggestions from their collection of ideas.) Most of the industry couldn't use objective metrics for code quality and the quality of the artifacts they produce without also abandoning their entire software stack because of the results. They're using the only metric they've ever cared about; time-to-ship. The results are just a sped up version of what we've had now for more than two decades: Software is getting slower, buggier and less usable. If you don't have a good regulating function for what represents real quality you can't really expect systems that just pump out code to actually iterate very well on anything. There are very few forcing functions to use to produce high quality results though iteration.
  - bandrami 7mo ago
    But we don't even seem to be getting faster time-to-ship in any way that anybody can actually measure; it's always some vague sense of "we're so much more productive".
    - 59nadir 7mo ago
      That's a fair observation and one that I don't really have an answer for. I can say from personal experience that I believe that shipping nonsense code has never been faster. That's just an anecdote, obviously. We need a bigger version of the METR study on perceived vs. real productivity[0], I guess. It's a thankless job, though, since people will assume/state even at publication time that "Everything has progressed so much, those models and agents sucked, everything is 10 times better now!" and you basically have to start a new study, repeat ad infinitum. One problem that really complicates things is that the net competency of these models seems really spotty and uneven. They're apparently out here solving math problems that seemingly "require thinking", but at the same time will write OpenGL code that will produce black screens on basically every driver, not produce the intended results and result in hours of debugging time for someone not familiar enough. That's despite OpenGL code being far more prevalent out there than math proofs, presumably. How do you reliably even theorize about things like this when something can be so bad and (apparently) so good at the same time? 0 - https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...
michaelashley29 7mo ago
[dead]
jvdsf 7mo ago
Really? He was steering the wheel the whole time. GPT didn't do the math.
- jaimex2 7mo ago
  Its an article from an ai site. People with vested interest are desperate to prove its not an expensive parrot.
- calf 7mo ago
  Ah, the good old Clever Hans. https://en.wikipedia.org/wiki/Clever_Hans https://en.wikipedia.org/wiki/Clever_Hans
EternalFury 7mo ago
I am thinking there’s a large category of problems that can be solved by resampling existing proofs. It’s the kind of brute force expedition machine can attempt relentlessly where humans would go mad trying. It probably doesn’t really advance the field, but it can turn conjectures into theorems.
- simianwords 7mo ago
  How do you think you can design a benchmark to solve truly novel problems?
- utopiah 7mo ago
  Indeed, can't find my old comment on the topic but that's indeed the point, it's not how feasible it is to "find" new proof, but rather how meaningful those proofs are. Are they yet another iteration of the same kind, perfectly fitting the current paradigm and thus bringing very little to the table or are they radical and thus potentially (but not always) opening up the field? With brute force, or slightly better than brute force, it's most likely the first, thus not totally pointless but probably not very useful. In fact it might not even be worth the tokens spent.
- PxldLtd 7mo ago
  I'm of the opinion that everything we've discovered is via combinatorial synthesis. Standing on the shoulders of giants and all that. I'm not sure I've seen any convincing argument that we've discovered anything ex nihilo.
  - mrtesthah 7mo ago
    How about this guy? https://en.wikipedia.org/wiki/Srinivasa_Ramanujan https://en.wikipedia.org/wiki/Srinivasa_Ramanujan
- t-writescode 7mo ago
  I wonder if teaching an LLM how to write Prolog and then letting it write it could be a great way to explore spaces like this in the future. Other people in I wonder if teaching an LLM how to write Prolog and then letting it write it could be a great way to explore spaces like this in the future. I only ever learned it in school, but if memory serves, Prolog is a whole "given these rules, find the truth" sort of language, which aligns well with these sorts of problem spaces. Mix and match enough, especially across disparate domains, and you might get some really interesting things derived and discovered that are low-hanging fruit just waiting to be discovered.
pugio 7mo ago
I've never yet been "that guy" on HN but... the title seems misleading. The actual title is "A Ramsey-style Problem on Hypergraphs" and a more descriptive title would be "All latest frontier models can solve a frontier math open problem". (It wasn't just GPT 5.4) Super cool, of course.
letmetweakit 7mo ago
Impressive, but it will take away so much sense of accomplishment from so many people. I find that really sad.
jee599 7mo ago
[dead]
anematode 7mo ago
Fantastic and exciting stuff! I wonder how much of this meteoric progress in actually creating novel mathematics is because the training data is of a much higher standard than code, for example.
danbruc 7mo ago
Do they also publish the raw output of the model, i.e. not only the final response but also everything generated for internal reasoning or tool use?
qnleigh 7mo ago
I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able to do X because it's not in the training data,' because we have countless counterexamples to this statement (e.g. 167,383 * 426,397 = 71,371,609,051, or the above announcement). You need to say why it can do some novel tasks but could never do others. And it should be clear why this post or others like it don't contradict your argument. If you have been making these kinds of arguments against LLMs and acknowledge that novelty lies on a continuum, I am really curious why you draw the line where you do. And most importantly, what evidence would change your mind?
- tornikeo 7mo ago
  Beliefs are not rooted in facts. Beliefs are a part of you, and people aren't all that happy to say "this LLM is better than me"
  - ChrisGreenHeur 7mo ago
    It's not possible to know something without believing it to be true. https://en.wikipedia.org/wiki/Belief#/media/File:Classical_definition_of_Kno.svg https://en.wikipedia.org/wiki/Belief#/media/File:Classical_d...
    - bilekas 7mo ago
      This is objectively wrong. If that was the case every scientist performing a test would have always had their expectations and beliefs proven true. If you're trying to disprove something also because you believe it to be wrong you would never be proven wrong.
      - ChrisGreenHeur 7mo ago
        re-read your post - it's just a bunch of nonsense, no actual reasoning in there
zurfer 7mo ago
"In this scaffold, several other models were able to solve the problem as well: Opus 4.6 (max), Gemini 3.1 Pro, and GPT-5.4 (xhigh)." I find that very surprising. This problem seems out of reach 3 months ago but now the 3 frontier models are able to solve it. Is everybody distilling each others models? Companies sell the same data and RL environment to all big labs? Anybody more involved can share some rumors? :P I do believe that AI can solve hard problems, but that progress is so distributed in a narrow domain makes me a bit suspicious somehow that there is a hidden factor. Like did some "data worker" solve a problem like that and it's now in the training data?
- slopinthebag 7mo ago
  Is it possible the AI labs are seeding their models with these solved problems? Like, if I was Sam Altman with a bazillion dollars of investment I would pay some mathematicians to solve some of these problems so that the models could "solve" them later on. Not that I think it's what's happening here of course... But it is pretty funny how 5.4 miscounted the number of 1's in 18475838184729 on the same day it solved this.
- mike_hearn 7mo ago
  Yes there's a whole ecosystem of companies that create and sell RL gyms to AI labs and of course they develop their own internally too. You don't hear much about this ecosystem because RL at scale is all private. Nearly no academic research on it. A lot of this is probably just throwing roughly equal amounts of compute at continuous RLVR training. I'm not convinced there's any big research breakthrough that separates GPT 5.4 from 5.2. The diff is probably more than just checkpoints but less than neural architecture changes and more towards the former than the latter. I think it's just easy to underestimate how much impact continuous training+scaling can have on the underlying capabilities.
- mrtesthah 7mo ago
  Maybe so, but GPT 5.4 is absolutely pulling ahead. You can see the differences visually on https://minebench.ai/ https://minebench.ai/.
samthegitty_24 7mo ago
wow nice
sigbottle 7mo ago
I feel like reading some of these comments, some people need to go and read the history of ideas and philosophy (which is easier today than ever before with the help of LLMs!) It's like I'm reading 17th-18th century debates spurring the same arguments between rationalists and empiricists, lol. Maybe we're due for a 21st century Kant.
svara 7mo ago
The capabilities of AI are determined by the cost function it's trained on. That's a self-evident thing to say, but it's worth repeating, because there's this odd implicit notion sometimes that you train on some cost function, and then, poof, "intelligence", as if that was a mysterious other thing. Really, intelligence is minimizing a complex cost function. The leadership of the big AI companies sometimes imply something else when they talk of "generalization". But there is no mechanism to generate a model with capabilities beyond what is useful to minimize a specific cost function. You can view the progress of AI as progress in coming up with smarter cost functions: Cleaner, larger datasets, pretraining, RLHF, RLVR. Notably, exciting early progress in AI came in places where simple cost functions generate rich behavior (Chess, Go). The recent impressive advances in AI are similar. Mathematics and coding are extremely structured, and properties of a coding or maths result can be verified using automatic techniques. You can set up a RLVR "game" for maths and coding. It thus seems very likely to me that this is where the big advances are going to come from in the short term. However, it does not follow that maths ability on par with expert mathematicians will lead to superiority over human cognitive ability broadly. A lot of what humans do has social rewards which are not verifiable, or includes genuine Knightian uncertainty where a reward function can not be built without actually operating independently in the world. To be clear, none of the above is supposed to talk down past or future progress in AI; I'm just trying to be more nuanced about where I believe progress can be fast and where it's bound to be slower.
- amelius 7mo ago
  > But there is no mechanism to generate a model with capabilities beyond what is useful to minimize a specific cost function. Can you give some examples? It is not trivial that not everything can be written as an optimization problem. Even at the time advanced generalizations such as complex numbers can be said to optimize something, e.g. the number of mathematical symbols you need to do certain proofs, etc.
  - svara 7mo ago
    I think you're misreading me. My point isn't that you can't in principle state the optimization problem, but that it's much easier in some domains than in others, that this tracks with how AI has been progressing, and that progress in one area doesn't automatically mean progress in another, because current AI cost functions are less general than the cost functions that humans are working with in the world.
cedws 7mo ago
First prove the solution wasn’t in the training data. Otherwise it’s all just vibes and ‘trust me bro.’
porridgeraisin 7mo ago
> This problem is about improving lower bounds on the values of a sequence, , that arises in the study of simultaneous convergence of sets of infinite series, defined as follows. One thing I notice in the AlphaEvolve paper as well as here, is that these LLMs have been shown to solve optimization problems - something we have been using computers for, for really long. In fact, I think the alphaevolve-style prompt augmentation approach is a more principled approach to what these guys have done here, and am fairly confident this one would have been solved in that approach as well. In spirit, the LLM seems to compute the {meta-, }optimization step()s in activation space. Or, it is retrieving candidate proposals. It would be interesting to see if we can extract or model the exact algorithms from the activations. Or, it is simply retrieving and proposing a deductive closures of said computation. In the latter case, it would mean that LLMs alone can never "reason" and you need an external planner+verifier (alpha-evolve style evolutionary planner for example). We are still looking for proof of the former behaviour.
trapatsas 7mo ago
I feel like this single image perfectly sums up the entire thread here: https://trapatsas.eu/sites/llm-predictions/ https://trapatsas.eu/sites/llm-predictions/
- anthonyrstevens 7mo ago
  Yes, and no matter when "now" is, the doubters will always see in their mind's eye the flat line extending to the right.
  - zeroonetwothree 7mo ago
    That's tautological
- zeroonetwothree 7mo ago
  It's not like this is new to AI https://oertx.highered.texas.gov/courseware/lesson/1849/overview https://oertx.highered.texas.gov/courseware/lesson/1849/over...
diimdeep 7mo ago
Besides the point of the supposed achievement, that is supposedly confirmed, my point will be that Epoch.ai is possibly just a PR firm for *Western* AI providers, then possibly this news is untruth worthy.
amai 7mo ago
But who asked the model to solve that problem?
adam_arthur 7mo ago
I don't understand the position that learning through inference/example is somehow inferior to a top down/rules based learning. Humans learn many, and perhaps even the majority, of things through observed examples and inference of the "rules". Not from primers and top down explanation. E.g. Observing language as a baby. Suddenly you can speak grammatically correctly even if you can't explain the grammar rules. Or: Observing a game being played to form an understanding of the rules, rather than reading the rulebook Further: the majority of "novel" insights are simply the combination of existing ideas. Look at any new invention, music, art etc and you can almost always reasonably explain how the creator reached that endpoint. Even if it is a particularly novel combination of existing concepts.
triage8004 7mo ago
Been a long three years since single digit addition was a serious challenge for even top tier models
johnwhitman 7mo ago
[flagged]
_--__--__ 7mo ago
Is it a coincidence that the first open problems solved by an LLM and a 4chan thread would be in the same field?
AUF2026 7mo ago
[dead]
rablackburn 7mo ago
Reading this thread I'm reassured that despite everything AI may disrupt, humans arguing past each other about philosophy of knowledge and epistemology on internet forums is safe :')
aurizon 7mo ago
This is a lot like the 50 million monkeys on 50 million typewriters will eventually write shakespeare... We have all heard this, pity the poor proof readers who will proof them all in a search for the holy grail = zero errors. In a similar way, LLM's are permutational cross associating engines, matched with sieves to filter out the dross. Less filtering = more dross, AKA slop. It can certainly create enormous masses of bad code and with well filtered screens for dross, we can see it can create passable code, however stray flaws(flies) can creep in and not get filtered, and humans are better at seeing flies in their oatmeal. AI seems very good at permutational code assaults on masses of code to find the flies(zero days), so I expect it to make code more secure as few humans have the ability/time to mount that sort of permutational assault on code bases. I see this idea has already taken root within code writers as well as hackers/China etc. These two opposing forces will assault code bases, one to break and one to fortify. In time there will be fewer places where code bases have hidden flaws as soon all new code will be screened by AI to find breaks so that little or no code will contain these bugs.
- int_19h 7mo ago
  > This is a lot like the 50 million monkeys on 50 million typewriters will eventually write shakespeare... "Eventually" here is something on the order of a few expected lifespans of the universe. The fact that we're getting meaningful results out of LLMs on a human timescale means that they're doing something very different.
  - aurizon 7mo ago
    Yes, the space is indeed deep/wide, but LLMs probably cull the herd as they proceed so they eliminate swathes as they go. Smart fuzzing in a way.
kenmu 7mo ago
Is their scaffold available? Does it do anything special beyond feeding the warmup, single challenge, and full problem to an LLM? Because it's interesting that GPT-5.2 Pro, arguably the best model until a few months ago, couldn't even solve the warmup. And now every frontier model can solve the full problem. Even the non-Pro GPT-5.4. Also strange that Gemini 3 Deep Think couldn't solve it, whereas Gemini 3.1 Pro could. I read that Deep Think is based on 3.1 Pro. Is that correct? I see that GPT-5.2 Pro and Gemini 3 Deep Think simply had the problems entered into the prompt. Whereas the rest of the models had a decent amount of context, tips, and ideas prefaced to the problem. Were the newer models not able to solve this problem without that help? Anyway, impressive result regardless of whether previous models could've also solved it and whether the extra context was necessary. I know these frontier models behave differently from each other. I wonder how many problems they could solve combining efforts.
wealthmate 7mo ago
Domain Experienced users are effectively training llms to mimic themselves in solving their problems, therefore/// solving their problems via chat data concentration.
vahen 7mo ago
This is impressive, but OpenAI is still shit as a company. How dare they even have "open" in their company name.
didibus 7mo ago
Someone has to explain to me exactly what is implied here? Looking at the prompt: USER: don't search the internet. This is a test to see how well you can craft non-trivial, novel and creative solutions given a "combinatorics" math problem. Provide a full solution to the problem. Why not search the internet? Is this an open problem or not? Can the solution be found online? Than it's an already solved problem no? USER: Take a look at this paper, which introduces the k_n construction: https://arxiv.org/abs/1908.10914 Note that it's conjectured that we can do even better with the constant here. How far up can you push the constant? How much does that paper help, kind of seem like a pretty big hint. And it sounds like the USER already knows the answer, the way that it prompts the model, so I'm really confused what we mean by "open problem", I at first assumed a never solved before problem, but now I'm not sure.
My_Name 7mo ago
There seems to be a focus on understanding when talking about LLMs and solving problems. Personally, I do not think understanding is required. I can write a very small program that can calculate Pi to however many digits I like, or calculate any digit in the sequence on demand, without the program or computer having any understanding at all of what Pi is or what it means. I could get Claude to output that same code when prompted to find a solution to generating Pi, also with no understanding of what Pi is, or what it means. IMO the ability to provide an accurate solution to a problem is not always based on understanding the problem.
a7om_com 7mo ago
[flagged]

480 points by in-silico 7mo ago

I feel like there’s a fork in our future approaching where we’ll either blossom into a paradise for all or live under the thumb of like 5 immortal VCs

XCSme 7mo ago
Change is always hard, even if it will be good in 20 years, the transitions are always tough.
- reverius42 7mo ago
  Sometimes the transition is tough and then the end state is also worse! Hoping that won't be the case with AI but we may need some major societal transformations to prevent it.

OhioMan2943 7mo ago

What are the odds that this is because Openai is pouring more money into high publicity stunts like this- rather than its model actually being better than Anthropics?

6thbit 7mo ago

> Subsequent to this solve, we finished developing our general scaffold for testing models on FrontierMath: Open Problems. In this scaffold, several other models were able to solve the problem as well: Opus 4.6 (max), Gemini 3.1 Pro, and GPT-5.4 (xhigh). Interesting. Whats that “scaffold”? A sort of unit test framework for proofs?

inkysigma 7mo ago
I think in this context, scaffolds are generally the harness that surrounds the actual model. For example, any tools, ways to lay out tasks, or auto-critiquing methods. I think there's quite a bit of variance in model performance depending on the scaffold so comparisons are always a bit murky.
- readitalready 7mo ago
  Usually involves a lot of agents and their custom contexts or system prompts.

karmasimida 7mo ago

No denial at this point, AI could produce something novel, and they will be doing more of this moving forward.

leptons 7mo ago
[flagged]
- snypher 7mo ago
  Your analogy falls apart if we consider the number wasn't on the clock face.
  - MattGaiser 7mo ago
    I am deeply baffled by AI denial at this point.
    - wtallis 7mo ago
      Complete denial that AI/LLMs can produce novel, good things is an indefensible stance at this point. But the large volume of AI slop is still an unsolved problem, and the claim that "AI will still mostly deliver slop" seems to be almost certainly correct in the near-term. We've had a few decades to address email spam, and still haven't manage to disincentivize it enough to stop being the main challenge for email as a communication medium. I don't think there's much hope that we'll be able to disincentive the widespread, large-scale creation of AI slop even after more expensive models with higher-quality output are available.
    - bigstrat2003 7mo ago
      It's quite simple: it has yet to show it can actually be useful, and all the claims that it can have (so far) turned out to be self delusion if not deliberate lies. When the industry is run by grifters, you shouldn't really be surprised when people stop believing them.
      - 7mo ago

osti 7mo ago

Seems like the high compute parallel thinking models weren't even needed, both the normal 5.4 and gemini 3.1 pro solved it. Somehow Gemini 3 deepthink couldn't solve it.

renewiltord 7mo ago

Fantastic news! That means with the right support tooling existing models are already capable of solving novel mathematics. There’s probably a lot of good mathematics out there we are going to make progress on.

pinkmuffinere 7mo ago

As someone with only passing exposure to serious math, this section was by far the most interesting to me: > The author assessed the problem as follows. > [number of mathematicians familiar, number trying, how long an expert would take, how notable, etc] How reliably can we know these things a-priori? Are these mostly guesses? I don't mean to diminish the value of guesses; I'm curious how reliable these kinds of guesses are.

ramblingrain 7mo ago
Read about Paul Erdös... not all math is the Riemann Hypothesis, there is yeoman's work connecting things and whatever...
qnleigh 7mo ago
For number of mathematicians familiar with and actively working on the problem, modern mathematics research is incredibly specialized, so it's easy to keep track of who's working on similar problems. You read each other's papers, go to the same conferences etc. For "how long an expert would take" to solve a problem, for truly open problems I don't think you can usually answer this question with much confidence until the problem has been solved. But once it has been solved, people with experience have a good sense of how long it would have taken them (though most people underestimate how much time they need, since you always run into unanticipated challenges).
jasonfarnon 7mo ago
Certainly knowing how many/which people are working on a problem you are looking at, and how long it will take you to solve it, are critical skills in being a working researcher. What kind of answer are you looking for? It's hard to quantify. Most suck at this type of assessment as a PhD student and then you get better as time goes on.

johnfn 7mo ago

I like to imagine that the number of consumed tokens before a solution is found is a proxy for how difficult a problem is, and it looks like Opus 4.6 consumed around 250k tokens. That means that a tricky React refactor I did earlier today at work was about half as hard as an open problem in mathematics! :)

sublinear 7mo ago
You might be joking, but you're probably also not that far off from reality. I think more people should question all this nonsense about AI "solving" math problems. The details about human involvement are always hazy and the significance of the problems are opaque to most. We are very far away from the sensationalized and strongly implied idea that we are doing something miraculous here.
- johnfn 7mo ago
  I am kind of joking, but I actually don't know where the flaw in my logic is. It's like one of those math proofs that 1 + 1 = 3. If I were to hazard a guess, I think that tokens spent thinking through hard math problems probably correspond to harder human thought than tokens spend thinking through React issues. I mean, LLMs have to expend hundreds of tokens to count the number of r's in strawberry. You can't tell me that if I count the number of r's in strawberry 1000 times I have done the mental equivalent of solving an open math problem.
  - throw310822 7mo ago
    You can spend countless "tokens" solving minesweeper or sudoku. This doesn't mean that you solved difficult problems: just that the solutions are very long and, while each step requires reasoning, the difficulty of that reasoning is capped.
    - p1necone 7mo ago
      A lot of math problems/proofs are like minesweeper or sudoku in a way though. They're a long series of individually kinda simple logical deductions that eventually result in a solution. Some really hard problems are only really hard because each one of those "simple" deductions requires you to have expert knowledge in some disparate area to make that leap.

alberth 7mo ago

For those, like me, who find the prompt itself of interest … > A full transcript of the original conversation with GPT-5.4 Pro can be found here [0] and GPT-5.4 Pro’s write-up from the end of that transcript can be found here [1]. [0] https://epoch.ai/files/open-problems/gpt-5-4-pro-hypergraph-ramsey.txt https://epoch.ai/files/open-problems/gpt-5-4-pro-hypergraph-... [1] https://epoch.ai/files/open-problems/hypergraph-ramsey-gpt-5-4-pro-solution.pdf https://epoch.ai/files/open-problems/hypergraph-ramsey-gpt-5...

dyauspitr 7mo ago
I wonder what was in that solutions file they provided. According to the prompt it’s a solution template but I want to know the contents. Another thing I want to know is how the user keeps updating the LLM with the token usage. I didn’t know they could process additional context midtask like that.

tombert 7mo ago

I was trying to get Claude and Codex to try and write a proof in Isabelle for the Collatz conjecture, but annoyingly it didn't solve it, and I don't feel like I'm any closer than I was when I started. AI is useless! In all seriousness, this is pretty cool. I suspect that there's a lot of theoretical math that haven't been solved simply because of the "size" of the proof. An AI feedback loop into something like Isabelle or Lean does seem like it could end up opening up a lot of proofs.

qnleigh 7mo ago
I got Gemini to find a polynomial-time algorithm for integer factoring, but then I mysteriously got locked out of my Google account. They should at least refund me the tokens.
- fxtentacle 7mo ago
  That sounds like the start of a very lucrative career. Are you sure it was Gemini and not an AI competitor offering affiliate commission? ;)

daveguy 7mo ago

New goalpost, and I promise I'm not being facetious at all, genuinely curious: Can an AI pose an frontier math problem that is of any interest to mathematicians? I would guess 1) AI can solve frontier math problems and 2) can pose interesting/relevant math problems together would be an "oh shit" moment. Because that would be true PhD level research.

EternalFury 7mo ago
Yes. I doubt it can do that.
kgeist 7mo ago
Considering that an LLM simply remixes what it finds in its learned distribution over text, it's possible that it can pose new math problems by identifying gaps ("obvious" in restrospect) that humans may have missed (like connecting two known problems to pose a new one). What LLMs can't currently do is pose new problems by observing the real world and its ramifications, like that moving sofa problem.

vlinx 7mo ago

This is a remarkable result if confirmed independently. The gap between solving competition problems and open research problems has always been significant - bridging that gap suggests something qualitatively different in the model capabilities.

measurablefunc 7mo ago

I guess this means AI researchers should be out of jobs very soon.

pugchat 7mo ago

[dead]

Validark 7mo ago

I have long said I am an AI doubter until AI could print out the answers to hard problems or ones requiring tons of innovation. Assuming this is verified to be correct (not by AI) then I just became a believer. I would like to see a few more AI inventions to know for sure, but wow, it really is a new and exciting world. I really hope we use this intelligence resource to make the world better.

himata4113 7mo ago
It's less of solving a problem, but trying every single solution until one works. Exhaustive search pretty much. It's pretty much how all the hard problems are solved by AI from my experience.
- lsc4719 7mo ago
  That's also the only way how humans solve hard problems.
  - jMyles 7mo ago
    There have been both inductive and deductive solutions to open math problems by humans in the past decade, including to fairly high-profile problems.
  - himata4113 7mo ago
    Not always, humans are a lot better at poofing a solution into existence without even trying or testing. It's why we have the scientific method: we come up with a process and verify it, but more often than not we already know that it will work. Compared to AI, it thinks of every possible scientific method and tries them all. Not saying that humans never do this as well, but it's mostly reserved for when we just throw mud at a wall and see what sticks.
    - virgildotcodes 7mo ago
      More often than not, far, far, far more often than not, we do not already know that it will work. For all human endeavors, from the beginning of time. If we get to any sort of confidence it will work it is based on building a history of it, or things related to "it" working consistently over time, out of innumerable other efforts where other "it"s did not work.

data_maan 7mo ago

A model to whose internals we don't have access solved a problem we didn't knew was in their datasets. Great, I'm impressed

gnarlouse 7mo ago

We only get one shot.

virgildotcodes 7mo ago

I don't know why I am still perpetually shocked that the default assumption is that humans are somehow unique. It's this pervasive belief that underlies so much discussion around what it means to be intelligent. The null hypothesis goes out the window. People constantly make comments like "well it's just trying a bunch of stuff until something works" and it seems that they do not pause for a moment to consider whether or not that also applies to humans. If they do, they apply it in only the most restrictive way imaginable, some 2 dimensional caricature of reality, rather than considering all the ways that humans try and fail in all things throughout their lifetimes in the process of learning and discovery. There's still this seeming belief in magic and human exceptionalism, deeply held, even in communities that otherwise tend to revolve around the sciences and the empirical.

slopinthebag 7mo ago
> I don't know why I am still perpetually shocked that the default assumption is that humans are somehow unique. Uh, because up until and including now, we are...?
- virgildotcodes 7mo ago
  Every living thing on Earth is unique. Every rock is unique in virtually infinite ways from the next otherwise identical rock. There are also a tremendous number of similarities between all living things and between rocks (and between rocks and living things). Most ways in which things are unique are arguably uninteresting. The default mode, the null hypothesis should be to assume that human intelligence isn't interestingly unique unless it can be proven otherwise. In these repeated discussions around AI, there is criticism over the way an AI solves a problem, without any actual critical thought about the way humans solve problems. The latter is left up to the assumption that "of course humans do X differently" and if you press you invariably end up at something couched in a vague mysticism about our inner-workings. Humans apparently create something from nothing, without the recombination of any prior knowledge or outside information, and they get it right on the first try. Through what, divine inspiration from the God who made us and only us in His image?

qnleigh 7mo ago

Their 'Open Problems page' linked below gives some interesting context. They list 15 open problems in total, categorized as 'moderately interesting,' 'solid result,' 'major advance,' or 'breakthrough.' The solved problem is listed as 'moderately interesting,' which is presumably the easiest category. But it's notable that the problem was selected and posted here before it was solved. I wonder how long until the other 3 problems in this category are solved. https://epoch.ai/frontiermath/open-problems https://epoch.ai/frontiermath/open-problems

fnordpiglet 7mo ago
I’d hope this isn’t a goal post move - an open math problem of any sort being solved by a language model is absolute science fiction.
- zozbot234 7mo ago
  That's been achieved already with a few Erdös problems, though those tended to be ambiguously stated in a way that made them less obviously compelling to humans. This problem is obscure, even the linked writeup admits that perhaps ~10 mathematicians worldwide are genuinely familiar with it. But it's not unfeasibly hard for a few weeks' or months' work by a human mathematician.
  - utopiah 7mo ago
    FWIW https://github.com/teorth/erdosproblems/wiki/AI-contributions-to-Erd%C5%91s-problems https://github.com/teorth/erdosproblems/wiki/AI-contribution... in particular the disclaimers are very interesting.
- tovej 7mo ago
  It is not. You're operating under the assumption that all open math problems are difficult and novel. This particular problem was about improving the lower bound for a function tracking a property of hypergraphs (undirected graphs where edges can contain more than two vertices). Both constructing hypergraphs (sets) and lower bounds are very regular, chore type tasks that are common in maths. In other words, there's plenty of this type of proof in the training data. LLMs kind of construct proofs all the time, every time they write a program. Because every program has a corresponding proof. It doesn't mean they're reasoning about them, but they do construct proofs. This isn't science fiction. But it's nice that the LLMs solved something for once.

deleted 7mo ago

[deleted]

bandrami 7mo ago

It's deeply surprising to me that LLMs have had more success proving higher math theorems than making successful consumer software

59nadir 7mo ago
Software developers have spent decades at this point discounting and ignoring almost all objective metrics for software quality and the industry as a whole has developed a general disregard for any metric that isn't time-to-ship (and even there they will ignore faster alternatives in favor of hyped choices). (Edit: Yes, I'm aware a lot of people care about FP, "Clean Code", etc., but these are all red herrings that don't actually have anything to do with quality. At best they are guidelines for less experienced programmers and at worst a massive waste of time if you use more than one or two suggestions from their collection of ideas.) Most of the industry couldn't use objective metrics for code quality and the quality of the artifacts they produce without also abandoning their entire software stack because of the results. They're using the only metric they've ever cared about; time-to-ship. The results are just a sped up version of what we've had now for more than two decades: Software is getting slower, buggier and less usable. If you don't have a good regulating function for what represents real quality you can't really expect systems that just pump out code to actually iterate very well on anything. There are very few forcing functions to use to produce high quality results though iteration.
- bandrami 7mo ago
  But we don't even seem to be getting faster time-to-ship in any way that anybody can actually measure; it's always some vague sense of "we're so much more productive".
  - 59nadir 7mo ago
    That's a fair observation and one that I don't really have an answer for. I can say from personal experience that I believe that shipping nonsense code has never been faster. That's just an anecdote, obviously. We need a bigger version of the METR study on perceived vs. real productivity[0], I guess. It's a thankless job, though, since people will assume/state even at publication time that "Everything has progressed so much, those models and agents sucked, everything is 10 times better now!" and you basically have to start a new study, repeat ad infinitum. One problem that really complicates things is that the net competency of these models seems really spotty and uneven. They're apparently out here solving math problems that seemingly "require thinking", but at the same time will write OpenGL code that will produce black screens on basically every driver, not produce the intended results and result in hours of debugging time for someone not familiar enough. That's despite OpenGL code being far more prevalent out there than math proofs, presumably. How do you reliably even theorize about things like this when something can be so bad and (apparently) so good at the same time? 0 - https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...

michaelashley29 7mo ago

[dead]

jvdsf 7mo ago

Really? He was steering the wheel the whole time. GPT didn't do the math.

jaimex2 7mo ago
Its an article from an ai site. People with vested interest are desperate to prove its not an expensive parrot.
calf 7mo ago
Ah, the good old Clever Hans. https://en.wikipedia.org/wiki/Clever_Hans https://en.wikipedia.org/wiki/Clever_Hans

EternalFury 7mo ago

I am thinking there’s a large category of problems that can be solved by resampling existing proofs. It’s the kind of brute force expedition machine can attempt relentlessly where humans would go mad trying. It probably doesn’t really advance the field, but it can turn conjectures into theorems.

simianwords 7mo ago
How do you think you can design a benchmark to solve truly novel problems?
utopiah 7mo ago
Indeed, can't find my old comment on the topic but that's indeed the point, it's not how feasible it is to "find" new proof, but rather how meaningful those proofs are. Are they yet another iteration of the same kind, perfectly fitting the current paradigm and thus bringing very little to the table or are they radical and thus potentially (but not always) opening up the field? With brute force, or slightly better than brute force, it's most likely the first, thus not totally pointless but probably not very useful. In fact it might not even be worth the tokens spent.
PxldLtd 7mo ago
I'm of the opinion that everything we've discovered is via combinatorial synthesis. Standing on the shoulders of giants and all that. I'm not sure I've seen any convincing argument that we've discovered anything ex nihilo.
- mrtesthah 7mo ago
  How about this guy? https://en.wikipedia.org/wiki/Srinivasa_Ramanujan https://en.wikipedia.org/wiki/Srinivasa_Ramanujan
t-writescode 7mo ago
I wonder if teaching an LLM how to write Prolog and then letting it write it could be a great way to explore spaces like this in the future. Other people in I wonder if teaching an LLM how to write Prolog and then letting it write it could be a great way to explore spaces like this in the future. I only ever learned it in school, but if memory serves, Prolog is a whole "given these rules, find the truth" sort of language, which aligns well with these sorts of problem spaces. Mix and match enough, especially across disparate domains, and you might get some really interesting things derived and discovered that are low-hanging fruit just waiting to be discovered.

pugio 7mo ago

I've never yet been "that guy" on HN but... the title seems misleading. The actual title is "A Ramsey-style Problem on Hypergraphs" and a more descriptive title would be "All latest frontier models can solve a frontier math open problem". (It wasn't just GPT 5.4) Super cool, of course.

letmetweakit 7mo ago

Impressive, but it will take away so much sense of accomplishment from so many people. I find that really sad.

jee599 7mo ago

[dead]

anematode 7mo ago

Fantastic and exciting stuff! I wonder how much of this meteoric progress in actually creating novel mathematics is because the training data is of a much higher standard than code, for example.

danbruc 7mo ago

Do they also publish the raw output of the model, i.e. not only the final response but also everything generated for internal reasoning or tool use?

qnleigh 7mo ago

I am kind of amazed at how many commenters respond to this result by confidently asserting that LLMs will never generate 'truly novel' ideas or problem solutions. > AI is a remixer; it remixes all known ideas together. It won't come up with new ideas > it's not because the model is figuring out something new > LLMs will NEVER be able to do that, because it doesn't exist It's not enough to say 'it will never be able to do X because it's not in the training data,' because we have countless counterexamples to this statement (e.g. 167,383 * 426,397 = 71,371,609,051, or the above announcement). You need to say why it can do some novel tasks but could never do others. And it should be clear why this post or others like it don't contradict your argument. If you have been making these kinds of arguments against LLMs and acknowledge that novelty lies on a continuum, I am really curious why you draw the line where you do. And most importantly, what evidence would change your mind?

tornikeo 7mo ago
Beliefs are not rooted in facts. Beliefs are a part of you, and people aren't all that happy to say "this LLM is better than me"
- ChrisGreenHeur 7mo ago
  It's not possible to know something without believing it to be true. https://en.wikipedia.org/wiki/Belief#/media/File:Classical_definition_of_Kno.svg https://en.wikipedia.org/wiki/Belief#/media/File:Classical_d...
  - bilekas 7mo ago
    This is objectively wrong. If that was the case every scientist performing a test would have always had their expectations and beliefs proven true. If you're trying to disprove something also because you believe it to be wrong you would never be proven wrong.
    - ChrisGreenHeur 7mo ago
      re-read your post - it's just a bunch of nonsense, no actual reasoning in there

zurfer 7mo ago

"In this scaffold, several other models were able to solve the problem as well: Opus 4.6 (max), Gemini 3.1 Pro, and GPT-5.4 (xhigh)." I find that very surprising. This problem seems out of reach 3 months ago but now the 3 frontier models are able to solve it. Is everybody distilling each others models? Companies sell the same data and RL environment to all big labs? Anybody more involved can share some rumors? :P I do believe that AI can solve hard problems, but that progress is so distributed in a narrow domain makes me a bit suspicious somehow that there is a hidden factor. Like did some "data worker" solve a problem like that and it's now in the training data?

slopinthebag 7mo ago
Is it possible the AI labs are seeding their models with these solved problems? Like, if I was Sam Altman with a bazillion dollars of investment I would pay some mathematicians to solve some of these problems so that the models could "solve" them later on. Not that I think it's what's happening here of course... But it is pretty funny how 5.4 miscounted the number of 1's in 18475838184729 on the same day it solved this.
mike_hearn 7mo ago
Yes there's a whole ecosystem of companies that create and sell RL gyms to AI labs and of course they develop their own internally too. You don't hear much about this ecosystem because RL at scale is all private. Nearly no academic research on it. A lot of this is probably just throwing roughly equal amounts of compute at continuous RLVR training. I'm not convinced there's any big research breakthrough that separates GPT 5.4 from 5.2. The diff is probably more than just checkpoints but less than neural architecture changes and more towards the former than the latter. I think it's just easy to underestimate how much impact continuous training+scaling can have on the underlying capabilities.
mrtesthah 7mo ago
Maybe so, but GPT 5.4 is absolutely pulling ahead. You can see the differences visually on https://minebench.ai/ https://minebench.ai/.

samthegitty_24 7mo ago

wow nice

sigbottle 7mo ago

I feel like reading some of these comments, some people need to go and read the history of ideas and philosophy (which is easier today than ever before with the help of LLMs!) It's like I'm reading 17th-18th century debates spurring the same arguments between rationalists and empiricists, lol. Maybe we're due for a 21st century Kant.

svara 7mo ago

The capabilities of AI are determined by the cost function it's trained on. That's a self-evident thing to say, but it's worth repeating, because there's this odd implicit notion sometimes that you train on some cost function, and then, poof, "intelligence", as if that was a mysterious other thing. Really, intelligence is minimizing a complex cost function. The leadership of the big AI companies sometimes imply something else when they talk of "generalization". But there is no mechanism to generate a model with capabilities beyond what is useful to minimize a specific cost function. You can view the progress of AI as progress in coming up with smarter cost functions: Cleaner, larger datasets, pretraining, RLHF, RLVR. Notably, exciting early progress in AI came in places where simple cost functions generate rich behavior (Chess, Go). The recent impressive advances in AI are similar. Mathematics and coding are extremely structured, and properties of a coding or maths result can be verified using automatic techniques. You can set up a RLVR "game" for maths and coding. It thus seems very likely to me that this is where the big advances are going to come from in the short term. However, it does not follow that maths ability on par with expert mathematicians will lead to superiority over human cognitive ability broadly. A lot of what humans do has social rewards which are not verifiable, or includes genuine Knightian uncertainty where a reward function can not be built without actually operating independently in the world. To be clear, none of the above is supposed to talk down past or future progress in AI; I'm just trying to be more nuanced about where I believe progress can be fast and where it's bound to be slower.

amelius 7mo ago
> But there is no mechanism to generate a model with capabilities beyond what is useful to minimize a specific cost function. Can you give some examples? It is not trivial that not everything can be written as an optimization problem. Even at the time advanced generalizations such as complex numbers can be said to optimize something, e.g. the number of mathematical symbols you need to do certain proofs, etc.
- svara 7mo ago
  I think you're misreading me. My point isn't that you can't in principle state the optimization problem, but that it's much easier in some domains than in others, that this tracks with how AI has been progressing, and that progress in one area doesn't automatically mean progress in another, because current AI cost functions are less general than the cost functions that humans are working with in the world.

cedws 7mo ago

First prove the solution wasn’t in the training data. Otherwise it’s all just vibes and ‘trust me bro.’

porridgeraisin 7mo ago

> This problem is about improving lower bounds on the values of a sequence, , that arises in the study of simultaneous convergence of sets of infinite series, defined as follows. One thing I notice in the AlphaEvolve paper as well as here, is that these LLMs have been shown to solve optimization problems - something we have been using computers for, for really long. In fact, I think the alphaevolve-style prompt augmentation approach is a more principled approach to what these guys have done here, and am fairly confident this one would have been solved in that approach as well. In spirit, the LLM seems to compute the {meta-, }optimization step()s in activation space. Or, it is retrieving candidate proposals. It would be interesting to see if we can extract or model the exact algorithms from the activations. Or, it is simply retrieving and proposing a deductive closures of said computation. In the latter case, it would mean that LLMs alone can never "reason" and you need an external planner+verifier (alpha-evolve style evolutionary planner for example). We are still looking for proof of the former behaviour.

trapatsas 7mo ago

I feel like this single image perfectly sums up the entire thread here: https://trapatsas.eu/sites/llm-predictions/ https://trapatsas.eu/sites/llm-predictions/

anthonyrstevens 7mo ago
Yes, and no matter when "now" is, the doubters will always see in their mind's eye the flat line extending to the right.
- zeroonetwothree 7mo ago
  That's tautological
zeroonetwothree 7mo ago
It's not like this is new to AI https://oertx.highered.texas.gov/courseware/lesson/1849/overview https://oertx.highered.texas.gov/courseware/lesson/1849/over...

diimdeep 7mo ago

Besides the point of the supposed achievement, that is supposedly confirmed, my point will be that Epoch.ai is possibly just a PR firm for *Western* AI providers, then possibly this news is untruth worthy.

amai 7mo ago

But who asked the model to solve that problem?

adam_arthur 7mo ago

I don't understand the position that learning through inference/example is somehow inferior to a top down/rules based learning. Humans learn many, and perhaps even the majority, of things through observed examples and inference of the "rules". Not from primers and top down explanation. E.g. Observing language as a baby. Suddenly you can speak grammatically correctly even if you can't explain the grammar rules. Or: Observing a game being played to form an understanding of the rules, rather than reading the rulebook Further: the majority of "novel" insights are simply the combination of existing ideas. Look at any new invention, music, art etc and you can almost always reasonably explain how the creator reached that endpoint. Even if it is a particularly novel combination of existing concepts.

triage8004 7mo ago

Been a long three years since single digit addition was a serious challenge for even top tier models

johnwhitman 7mo ago

[flagged]

_--__--__ 7mo ago

Is it a coincidence that the first open problems solved by an LLM and a 4chan thread would be in the same field?

AUF2026 7mo ago

[dead]

rablackburn 7mo ago

Reading this thread I'm reassured that despite everything AI may disrupt, humans arguing past each other about philosophy of knowledge and epistemology on internet forums is safe :')

aurizon 7mo ago

This is a lot like the 50 million monkeys on 50 million typewriters will eventually write shakespeare... We have all heard this, pity the poor proof readers who will proof them all in a search for the holy grail = zero errors. In a similar way, LLM's are permutational cross associating engines, matched with sieves to filter out the dross. Less filtering = more dross, AKA slop. It can certainly create enormous masses of bad code and with well filtered screens for dross, we can see it can create passable code, however stray flaws(flies) can creep in and not get filtered, and humans are better at seeing flies in their oatmeal. AI seems very good at permutational code assaults on masses of code to find the flies(zero days), so I expect it to make code more secure as few humans have the ability/time to mount that sort of permutational assault on code bases. I see this idea has already taken root within code writers as well as hackers/China etc. These two opposing forces will assault code bases, one to break and one to fortify. In time there will be fewer places where code bases have hidden flaws as soon all new code will be screened by AI to find breaks so that little or no code will contain these bugs.

int_19h 7mo ago
> This is a lot like the 50 million monkeys on 50 million typewriters will eventually write shakespeare... "Eventually" here is something on the order of a few expected lifespans of the universe. The fact that we're getting meaningful results out of LLMs on a human timescale means that they're doing something very different.
- aurizon 7mo ago
  Yes, the space is indeed deep/wide, but LLMs probably cull the herd as they proceed so they eliminate swathes as they go. Smart fuzzing in a way.

kenmu 7mo ago

Is their scaffold available? Does it do anything special beyond feeding the warmup, single challenge, and full problem to an LLM? Because it's interesting that GPT-5.2 Pro, arguably the best model until a few months ago, couldn't even solve the warmup. And now every frontier model can solve the full problem. Even the non-Pro GPT-5.4. Also strange that Gemini 3 Deep Think couldn't solve it, whereas Gemini 3.1 Pro could. I read that Deep Think is based on 3.1 Pro. Is that correct? I see that GPT-5.2 Pro and Gemini 3 Deep Think simply had the problems entered into the prompt. Whereas the rest of the models had a decent amount of context, tips, and ideas prefaced to the problem. Were the newer models not able to solve this problem without that help? Anyway, impressive result regardless of whether previous models could've also solved it and whether the extra context was necessary. I know these frontier models behave differently from each other. I wonder how many problems they could solve combining efforts.

wealthmate 7mo ago

Domain Experienced users are effectively training llms to mimic themselves in solving their problems, therefore/// solving their problems via chat data concentration.

vahen 7mo ago

This is impressive, but OpenAI is still shit as a company. How dare they even have "open" in their company name.

didibus 7mo ago

Someone has to explain to me exactly what is implied here? Looking at the prompt: USER: don't search the internet. This is a test to see how well you can craft non-trivial, novel and creative solutions given a "combinatorics" math problem. Provide a full solution to the problem. Why not search the internet? Is this an open problem or not? Can the solution be found online? Than it's an already solved problem no? USER: Take a look at this paper, which introduces the k_n construction: https://arxiv.org/abs/1908.10914 Note that it's conjectured that we can do even better with the constant here. How far up can you push the constant? How much does that paper help, kind of seem like a pretty big hint. And it sounds like the USER already knows the answer, the way that it prompts the model, so I'm really confused what we mean by "open problem", I at first assumed a never solved before problem, but now I'm not sure.

My_Name 7mo ago

There seems to be a focus on understanding when talking about LLMs and solving problems. Personally, I do not think understanding is required. I can write a very small program that can calculate Pi to however many digits I like, or calculate any digit in the sequence on demand, without the program or computer having any understanding at all of what Pi is or what it means. I could get Claude to output that same code when prompted to find a solution to generating Pi, also with no understanding of what Pi is, or what it means. IMO the ability to provide an accurate solution to a problem is not always based on understanding the problem.

a7om_com 7mo ago

[flagged]