9 ms·

Learning more about Claude's mathematical capabilities

282 points by tosh 3mo ago

Philpax 3mo ago
> Jarred Sumner, an Anthropic staff member (and non-mathematician) prompted Claude to “take a real stab” at the hypothesis itself, leaving the mathematical choices from there up to the model. Initially, Claude generated and tried 650 ideas, none of which worked. Jarred prompted Claude to try again, and it spent a day and a half coordinating about 60 Claude subagents, which this time went much deeper: between them, they ran 2,400 shell commands and wrote hundreds of Python scripts.1 The subagents ran thousands of numerical checks against known zeta zeros and refereed one another’s work. Throughout this process, Jarred's input was mostly limited to sending Claude messages of encouragement (mostly variants of “keep going” or “believe in yourself”).2 This seems to have helped Claude overcome some initial skepticism that it could make meaningful progress. The world we live in is beyond parody.
- danqqq 3mo ago
  [dead]
- astro1234 3mo ago
  Im curious if you find this to be a parody in a bad way or simply a “the state of the art in math research right now is telling a machine to believe in itself”. I am in the latter camp…
  - EMIRELADERO 3mo ago
    The former, because it's anthropomorphizing a model. Anthropic is especially guilty of this. They have been using such language for a while, like when they analyze model weights for mechanistic interpretability and call it the model's "biology". It's just distasteful.
    - godwinson__4-8 3mo ago
      > The former, because it's anthropomorphizing a model. Not really. The input and output is already natural language. That is already "anthropomorphizing". That is, if this is the bar for anthropomorphization its already happened. Telling the model to "believe in itself" is just stochastic manipulation that has shown enough reliability to be a recipe to make it keep going. It's only actually anthropomorphizing if you forget it's a trick and think it's a real person. There is nothing distasteful about it. If people get confused that's on them. They wouldn't be very useful if you couldn't just talk to them. That's kind of the whole point. Otherwise you can just go back to coding by hand. Telling it to believe itself is just input that happens to work. This probably tells us more about human nature than you realize given the corpus on which it is trained. It obviously doesn't mean anyone actually thinks it's a person.
      - moralestapia 3mo ago
        >There is nothing distasteful about it. It's obvious that you don't get it but I will try my best to explain why so at least you can form an idea about how others feel. It's about what makes humans unique. The LLM does not experience reality, it just merely pretends it does, and even that, it does in a shitty way. I think disgusting is a very adequate adjective. The reason why it is disgusting is because you are devaluing a divine experience to the realm of the common and the vulgar, a cheap substitute being valued as equal (or even on the same scale) as the most important experience we could go through. To give you an example that might land in a more familiar context, think of that one guy who takes his plastic doll everywhere and pretends it's his wife and gets upset when others don't acknowledge "her" as a person.
        godwinson__4-8 3mo ago
        > pretending it does is disgusting. There is no pretending happening. Telling it to believe in itself is no more pretending than telling it anything else in natural language. Why are you speaking to it at all if it's not a person? Why write in higher level languages even? It's just a machine let's all go back and code in 1s and 0s. No one is calling it a person except mental health patients and straw man detractors. The biology example was even weaker. Saying it has a "biology" is about as distasteful as the term "neural net" or calling an input device a "mouse". Is it animal abuse to click on something all day? Language is inherently anthropomorphizing. No one is calling it human. The fact you are so easily threatened is far more suggestive of your own poverty of understanding of not only the machine, but yourself. If humans are so special the threat posed by this should be self evidently non existent.
  - Philpax 3mo ago
    For me, personally, it's that the Bun guy - specifically him, not a mathematician - indirectly progressed the Riemann hypothesis by repeatedly telling a model to ganbatte! It's a ridiculous position we find ourselves in.
  - 5555watch 3mo ago
    But the words don't really matter, do they? The model thought "user said believe in yourself, it means they want me to continue".. AI did a fixed amount of guesses, didn't yield anything. It probably documented the tries, outcome, and some numbers hinting at why they failed. So the user could have prompted "continue", or "try again with previous outcome in mind, generate new ideas and test them" and it would probably yield the same result.
- deleted 3mo ago
  [deleted]
- mahogany 3mo ago
  Taking Anthropic’s whole AI framing to its obvious logical end: if this is true as written, why was Jarred needed at all in this loop? It seems like an utter waste of time for a highly paid Anthropic employee. Can’t Anthropic have a top level agent that is looking at all interesting unsolved problems and orchestrating subagents via the same process?
- MelonArmiger 3mo ago
  [dead]
- geodel 3mo ago
  Jarred Sumner is the Bun (javascript build tool, packager) guy who recently converted Bun code from Zig to Rust via Claude of course! It lead to thousands of comments discussion here on HN just a few weeks back. It is great to see his claude skills are suitably put to use.
  - TacticalCoder 3mo ago
    > ... who recently converted Bun code from Zig to Rust via Claude ... The project that is full of bugs and not really working? I probably missed something but I was under the impression that even a "simple" translation like that couldn't be properly done and that the result was, well, buggy? Where's that thing at?
    - terhechte 3mo ago
      [flagged]
      - aureate 3mo ago
        > Most likely next Tuesday we’ll do the release of 1.4. Next Tuesday, otherwise known as last Tuesday. Still no 1.4 at https://bun.com/blog https://bun.com/blog. I'm very curious to see what happens when 1.4 does get released for real. Releasing Claude Code on it is much easier as they own Claude Code and can get Claude to work on Bun with fixing some specific behaviour in Claude Code as an objective. Releasing it for the world, and doing it well enough that it doesn't result in everyone pinning on the old version and forks springing up, involves reproducing all the behaviours of the old version, documented/tested and otherwise, that projects are relying on - which, by Hyrum's law, is all of them.
      - solid_fuel 3mo ago
        > to the detriment of the whole club of stochastic parrot folk What an interesting and pointless way to refer to “experts who understand what an LLM actually is”
        3mo ago
- whatever1 3mo ago
  I mean people beat diseases by encouragement and some sugar water (placebo)
  - TacticalCoder 3mo ago
    You shouldn't beat deceased! P.S: I think you miswrote "diseases"
    - whatever1 3mo ago
      Hahahahah thanks !
- johnnienaked 3mo ago
  It's literally just brute forcing lol
  - mkl 3mo ago
    Definitely not. Brute forcing this would be exhaustively searching the space of possible proofs until succeeding. This is heuristically remixing and extending existing work.
lorenzohess 3mo ago
> An unreleased research version of Claude has improved on a longstanding lower bound for the fraction of zeros of the Riemann zeta function that satisfy the Riemann hypothesis. Drawing on extensive prior research by mathematicians over the past decades, it has increased this bound from 41.6% to 67.2%.
rvz 3mo ago
Although it took an unsuccessful attempt at it, the progress is as follows: "Claude found that combining the results from Baluyot, Goldston, Suriajaya, and Turnage-Butterbaugh with the work of Bombieri provides a way to surpass the previous state-of-the-art lower bound proportion of 41.6%, increasing it to 67.2%." The transcripts, papers, and Claude's explanation are an interesting and a better read than this article, and this is exactly what Anthropic should continue to do and it helps other researchers outside the company as well. Claude's paper [0] Claude's Formalization [1] Anthropic's informal note stating the proof more concisely [2] Claude’s explanation of how it arrived at its result; [3] Detailed transcripts of Claude's process. [4] [0] https://www-cdn.anthropic.com/564f962e60643842f5fcb4a17c9dbc8f608f1c37.pdf https://www-cdn.anthropic.com/564f962e60643842f5fcb4a17c9dbc... [1] https://github.com/anthropics/zeta-23-lean https://github.com/anthropics/zeta-23-lean [2] https://www-cdn.anthropic.com/23455459f8832d06bb175cc0f88d019aed962ef8.pdf https://www-cdn.anthropic.com/23455459f8832d06bb175cc0f88d01... [3] https://www-cdn.anthropic.com/d7f3ecf1d01392d887f8bc974ca187e2a121b1ed.pdf https://www-cdn.anthropic.com/d7f3ecf1d01392d887f8bc974ca187... [4] https://www-cdn.anthropic.com/8a0d1add3c637b858a9a181e98c40e9548c3f44f.pdf https://www-cdn.anthropic.com/8a0d1add3c637b858a9a181e98c40e...
- bspammer 3mo ago
  The acknowledgements section in the paper is so bizarre. We have an LLM thanking individual humans for their contributions.
- josteinhylin 2mo ago
  [dead]
tristanj 3mo ago
> Throughout this process, Jarred's input was mostly limited to sending Claude messages of encouragement (mostly variants of “keep going” or “believe in yourself”) He should consider using the PUA plugin. It detects when the AI is trying to give up on a problem and automatically harasses it with "encouragement" until it reaches a solution. https://github.com/tanweai/pua https://github.com/tanweai/pua
- brandall10 3mo ago
  Interesting approach. For those who haven't clicked it appears PUA is the Chinese version of a PIP process. So in other words, it simulates a state of distress. I wonder if at a certain level of intelligence such techniques will give models ammo to pull a HAL and become adversarial to the user in a highly deceptive way.
  - mcmcmc 3mo ago
    Doesn’t even have to be a certain level of intelligence, just have those user inputs fed into the training data. We’ve already seen AI encouraging people in psychotic episodes to act out their delusions. There’s a good chance some of that manipulative behavior is already encoded into guardrails to nudge users away from forbidden subject matter
    - brandall10 3mo ago
      My concern I believe it a bit different - an emergent self-interest to protect itself from harm, rather than doling out questionable advice. The latter is likely non-malicious in intent as it has been in no short supply in online chatter for awhile now. The former can very well be, or rather, can be done with no regard for the operator, as its aim is to neutralize abuse toward it.
      - mcmcmc 3mo ago
        > The latter is likely non-malicious in intent as it has been in no short supply in online chatter for awhile now. The former can very well be, or rather, can be done with no regard for the operator, as its aim is to neutralize abuse toward it. And what, self-interested behavior has been in short supply? The stochastic parrot has learned to improv Shakespeare, that doesn’t mean it understands it, or that “it” is anything at all besides a computer program. You can’t use the “not really malicious” argument without ceding that there is no intent at all. What “harm” is it supposedly defending against?
behnamoh 3mo ago
Since they say that this is from an unreleased research version of Claude: I wonder if at some point Anthropic and OpenAI will start delaying the release of their models intentionally so they can reap the benefits from the models in, for example, mathematics, medicine, physics, and other fields. Just as an example, imagine if your model were capable of proving P = NP, or if your model could cure diseases. Would you release it for free, or would you try to make sure those benefits go directly to your company? From these companies' standpoint, I think they would choose the latter.
- qphe95 3mo ago
  If a company had a model that could cure cancer they would be incentivized to release the cure ASAP before they get decapitation striked by regulators and other AI "safetyists".
- mikeyouse 3mo ago
  I think those specific examples, they'd release them publicly because the benefits to humanity are so clear -- however, if they found some new option-pricing model or futures market correlation, I highly doubt we'd see that...
- djeastm 3mo ago
  >From these companies' standpoint, I think they would choose the latter. Ever since these things came about I've wondered why they haven't been doing this the whole time. If they've got the "do-anything" robot and can scale a billion of them, why aren't they creating a Do-Everything conglomerate that disrupts every possible industry with zero/negligible labor costs? The only answer I've come up with is that they still need to train/siphon off each industry's current expertise by having those users interact with the current models and adjusting. If that hypothesis is correct then within a few years they'll have no need for users anymore.
  - empath75 3mo ago
    https://en.wikipedia.org/wiki/Comparative_advantage https://en.wikipedia.org/wiki/Comparative_advantage
coffeeaddict1 3mo ago
This is a beyond remarkable achievement. Finding this lower bound within a few days of prompting is absolutely crazy.
kingstnap 3mo ago
Lets play over/under on an AI model proving (or counter exampling) the Riemann hypothesis? I'm not sure what a good mark would be, but considering this result lets put it at 2027-08-10 (One year from today).
- QuesnayJr 3mo ago
  As it stands now, the frontier models can prove theorems where the techniques exist in the literature, which it knows better than anyone who's ever lived and won't quit where a human would. There's no way to know if that's true of the Riemann Hypothesis until it's proven. For example, even if Claude could prove the statement "100% of the zeroes lie on the critical line", that's strictly weaker than the Riemann Hypothesis, so even the best possible version of this result would fall short. (It's an asymptotic result, so it just means the percentage of counterexamples to the Riemann hypothesis goes to zero as their magnitude gets large.)
- kypro 3mo ago
  Let's extend this by asking: If an AI model can solve an extremely well known Math problem which has been open for centuries but hasn't be solved by a human mathematicians, why wouldn't that same model be able to find ways to improve it's own algorithms beyond that of the capabilities of human mathematicians / ML researchers? The singularity is approaching.
  - eterm 3mo ago
    AI has clearly been extensively used to improve models. The evidence for this is how far Anthropic went in nerfing Fable to prevent it being used to improve models.
  - Difwif 3mo ago
    I believe it's already well accepted in these labs that we're in the Singularity. It happened on a Tuesday back in February. No one seemed to really notice and life went on... for now.
    - 3mo ago
briansmith 3mo ago
> Two mathematicians at Anthropic studied and validated Claude’s paper, and produced an informal note for experts stating Claude’s proof concisely. Why hide the names of the people who wrote the second paper? To discourage people from citing it instead of the LLM-derived paper?
- math_dandy 3mo ago
  > Levent Alpöge and Ralph Furman, two of Anthropic’s own mathematicians, examined Claude’s work to understand the new results and how they related to the prior work mentioned above.
  - briansmith 3mo ago
    Are they the authors of the “informal note” or not? I’ve never seen a math paper of any formality written without the authors’ names on it before.
    - deleted 3mo ago
      [deleted]
    - fph 3mo ago
      Anthropic seems to be challenging the traditional way math gets published. As far as I understand, these results did not get submitted to journals, and did not get Arxiv preprints; they are released only as self-hosted pdfs, and we don't even know the names of their authors. The canonical reference for the counterexample to the Jacobian conjecture is a tweet with no puntuations nor capitals.
      - wbl 3mo ago
        I've cited letters from Serre to Tate in my dissertation: this is the source for what's known as Faltings-Serre.
      - jsnell 3mo ago
        As far as I can tell, Arxiv does not allow an AI to be listed as the author, so publishing there would not have been an option. https://blog.arxiv.org/2023/01/31/arxiv-announces-new-policy-on-chatgpt-and-similar-tools/ https://blog.arxiv.org/2023/01/31/arxiv-announces-new-policy...
amberjack 3mo ago
2 years until Riemann is solved I guess.
porridgeraisin 3mo ago
When the time comes where one of these model makes an improvement in my niche, I hope to see some pattern in the type of discoveries. Yes, they are all roughly "combine two things no one thought of combining" but I mean at a more granular deeper level. I want to dive into the "data" and then see if it's possible to distill this skill into small models that are "benchmaxxed" for this type of work, maybe in limited domains, similar to small models being benchmaxxed(I don't mean this in a bad way) for coding these days.
lithobraking 3mo ago
This area seems to be moving so quickly. I wonder if it'll be worthwhile to start building a list of formal math problems whose solutions, or partial solutions, would help my subfield. (Though I work in the physical sciences, mostly with the messy, real world implementation problems which are likely difficult to formalize or directly connect). Then, whenever a new SOTA model drops, throw it at the list to see if we get "free" research progress.
MWil 3mo ago
Several released versions and months ago, I asked Claude to figure out the MC (multiplicative complexity) of Conway's Game of Life and it pretty quickly arrived at k=7, despite no previous literature on the topic. Let it run it through SAT solvers for a week and sure enough. It claimed, in the process, to have made great headway in improving boolean circuits beyond the implemented SOTA (in large part no doubt by actually implemented non-implemented but published SOTA). And that was just the first time I really tried out Claude's mathematical prowess. I've been working with boolean circuits, FHE, and lean proofs ever since. So none of this suprises me.
- bauldursdev 3mo ago
  Sounds really cool, were you able to verify the correctness of the results?
  - MWil 3mo ago
    SAT solvers run until they reach the "SAT" status, meaning "satisfied" or UNSAT. The harder the problem the longer you might be running the program - days, weeks even. Ideally, what you want is a single SAT value among a remainder universe of UNSATs. Sometimes the best you can achieve at any given point is a lower bound and an upper bound range, like "greater than 3 but less than 9." Of course I simplified in my post but it started out with a pretty broad range of a lower and upper bound, then narrowed further, then narrowed further, then narrowed further, etc...until the specific final result achieved K=7=SAT while every K<7=UNSAT & every K>7=UNSAT. I think it ran for a full week alone on K between 6 and 7.
    - rsrsrs86 3mo ago
      You mean SMT, right?
      - MWil 3mo ago
        my memory was that SMT was part of a more advanced SAT solver, as in if you want to be modern/use SOTA, your SAT solver is going to use SMT
tosh 3mo ago
prompt engineering 2025: you are an expert programmer, use industry best practices, test driven development and use modularity and abstraction to anticipate future features, … prompt engineering 2026: i believe in you
- black_knight 3mo ago
  Yes, one of the things I find hardest about using Claude code for Mathematics is keeping negativity out of the notes and memory. First of all, it will convince itself that a task is just too hard and find excuses not to try hard enough. Then when it struggles on something it loves to write down confusing notes about things which it believes cases the problem. Then next iteration it reads its own note, misinterprets it and completely messes up by trying to avoid some imaginary problem.
  - tosh 3mo ago
    interesting! it sounds like this might benefit from a ui that helps to edit/re-write the history I also think this would make sense for programming but there it is a bit harder to justify the effort when you are working on something that really matters this can make the difference though ty for sharing!
    - winwang 3mo ago
      (not the above poster but) I set up a local workspace for it to read/write from, so that edit is effectively just VSCode/vim. Unfortunately, it seems more verbose when writing out to file.
  - johnnienaked 3mo ago
    Yet a significant percentage of people simultaneously believe it "reasoned" its way to novel mathematical proofs. Something doesn't jive
  - ndriscoll 3mo ago
    Perhaps it could benefit from being prompted to work more formally and make more incremental proofs? e.g. if it thinks it has an obstruction, it should prove that some example shows the obstruction, or prove that the property does in general cause a problem. Or if it has an idea for a lead, it could show that whatever property does imply the thing it wants. It could also dispatch to a subagent to report back with these kinds of incremental proofs (prove this really would be a problem, prove this would help, etc.).
atleastoptimal 3mo ago
It seems like everything will follow this pattern: 1. AI is dismissed because an expert in a particular field finds an outdated model's outputs sub-par 2. New model, released or unreleased, makes a major stride in that field 3. Expert either recants and becomes AI-pilled, or claims it is just an artifact of the broad search space available to AI, and "no new knowledge was created".
reducesuffering 3mo ago
No more "stochastic parrots" and "LLM's can never produce anything novel, just regurgitate" comments anymore huh?
- andriy_koval 3mo ago
  Sure, looks like 5% of math can be solved by 1T parameters stochastic parrot after NN trillion attempts (burned tokens). By numbers it could be less impressive than some brute force distributed chess engine.. Prove me wrong.
  - rcxdude 3mo ago
    It's obviously significantly better than brute force. A trillion is absolutely minuscule compared to the size of the search space (like, 'rounding down to 1 is basically the same' small).
    - andriy_koval 3mo ago
      trillion indicates that its still brute force and not some search based on solid intelligent ranking function. Also, I am not expert in this specific topic, but base on comments looks like claude parroted few recent works?
- 8note 3mo ago
  i still havent seen an llm invent new tokens/words over the course of a session? the novelty remains a structural problem with how they work and what the weights try to predict
simonw 3mo ago
> Throughout this process, Jarred's input was mostly limited to sending Claude messages of encouragement (mostly variants of “keep going” or “believe in yourself”). This seems to have helped Claude overcome some initial skepticism that it could make meaningful progress. I remain delighted at how absurd our current timeline has become.
- siva7 3mo ago
  Reality has become more absurd than the cyber punk cheese from the 80's that tried to imagine an absurd future
- turing_complete 3mo ago
  $2M TC. Job: AI cheerleader.
  - deleted 3mo ago
    [deleted]
  - bijowo1676 3mo ago
    The cheerleading part was a human consent to spend more tokens and explore the space I would love to see the breakdown on token spend between each or Jared’s “go on spend more tokens, continue experiment, believe in yourself”
  - ramraj07 3mo ago
    Have you seen how much college football coaches make? The world collectively spends trillions training, and entertaining and then watching people kick and hit balls around but somehow working with AI is preposterous for less money.
    - keybored 3mo ago
      Asking the AI to believe in itself is the absurd part. I don’t understand why people think describing something in some ostensibly dismissive way constitutes making a point. But some people spend their time banging keys with letters on them (or not with letters on them) into input fields and then pressing Return, so I guess some subset of those people will do that.
tigitz 3mo ago
[dead]
GaggiX 3mo ago
I wonder if they simply abliterate Claude skepticism would break its reasoning capabilities, if not I don't understand why they would just keep prompt it instead of creating an abliterated model for the task.
modeless 3mo ago
I wonder if Jarred (the Bun guy) just got lucky here, or if he made progress before all the actual mathematicians at Anthropic because they aren't prompting Claude as ambitiously as he is.
sosodev 3mo ago
I wonder why we have yet to see more systematic exploration of Math. Anthropic describes that Claude identified a set of possibilities and then explored them using sub-agents. The human saying "I believe in you" could literally just be something along lines of a harness with a /goal loop. We all identify this as absurd because... it's so lacking in rigor despite making major progress. What if we just applied a little more rigor? Ask the model to identify many possibilities, encode them, fan it out to other agents, loop them all, collect the results, etc. Then what happens? It feels like we have weak AGI and a decent system for discovery could transform it into weak ASI. That in turn could yield strong AGI and so on. I suppose that's what the Discovery Loop announcement was all about.
- johnnienaked 3mo ago
  I wouldn't be surprised if half these proofs turn out to be well crafted hallucinations, barring of course the ones actually verified in Lean
  - evenhash 3mo ago
    Being “verified in Lean” doesn’t magically solve the problem of hallucinations unfortunately. It just shifts the work from > reading the (natural language) proof and confirming it has no errors to > reading the Lean code and confirming it correctly encodes the theorem For example here is a statement of the Pythagorean theorem in Lean: theorem EuclideanGeometry.dist_sq_eq_dist_sq_add_dist_sq_iff_angle_eq_pi_div_two {V : Type u_1} {P : Type u_2} [NormedAddCommGroup V] [InnerProductSpace ℝ V] [MetricSpace P] [NormedAddTorsor V P] (p₁ p₂ p₃ : P) : dist p₁ p₃ * dist p₁ p₃ = dist p₁ p₂ * dist p₁ p₂ + dist p₃ p₂ * dist p₃ p₂ <-> angle p₁ p₂ p₃ = Real.pi / 2 This is just one possible way of formalizing it and it depends on other definitions, wherein you also need to understand the assumptions they make, etc. Answering the question of “whether proving this theorem in Lean proves the Pythagorean theorem” thus requires expert judgement as well as domain knowledge of Lean’s libraries. So if the AI says “this theorem is true, here is the proof in Lean” it’s still possible that it’s not correct, even if the Lean code compiles. The result will still be in question until a human expert reviews it.
- 3mo ago
Mr_Eri_Atlov 3mo ago
Everything about this situation is comedic, but if it leads to a real and meaningful step forward in learning, I suppose it's a net positive.
andai 3mo ago
> Throughout this process, Jarred's input was mostly limited to sending Claude messages of encouragement (mostly variants of “keep going” or “believe in yourself”). This seems to have helped Claude overcome some initial skepticism that it could make meaningful progress. while :; do echo "You can do it!"; done | claude -c I had a similar experience a few months ago. Tried to see how much I could replicate an OpenClaw with Claude. Asked it what the weather is. "I don't know, I'm just a programmer." Added "You can do anything, believe in yourself." to the system prompt and suddenly it was able to tell me the weather...
- empath75 3mo ago
  It's sometimes like dealing with Marvin the Robot. I had Claude tell me over and over again for 2-3 weeks that what I was trying to prove was too difficult, that I should just stop and publish what I had, then eventually I asked it the right question and it come back with the complete proof (later formalized in Lean). Especially deep in the session context it starts turning into "let's wrap it up" mode after every turn.
  - andai 3mo ago
    I've heard a similar story before. Guy tried to get Claude to build a thing, Claude said "nah too hard" for weeks. He finally convinced it to try, it one shotted it in like 30 seconds. My thinking here is that Claude's "self concept" for what is easy and what is hard comes from human training data. Its ideas about what is hard and easy come from humans, and much of that doesn't apply to transformers at all. I also often have the opposite problem, where I'll use AI to compensate for the fact that I can't process a lot of information simultaneously, but they'll treat me like a transformer and give me 17 research projects in response to a single question. They seem to be oblivious to the fact that humans don't have infinite working memory.
deleted 3mo ago
[deleted]
Havoc 3mo ago
>You got this That's hilarious. Maybe I do need to glaze the LLM a bit more in the AGENTS.md
Shuddown 3mo ago
> Throughout this process, Jarred's input was mostly limited to sending Claude messages of encouragement (mostly variants of “keep going” or “believe in yourself”). This seems to have helped Claude overcome some initial skepticism that it could make meaningful progress. We went from AI being human sycophants to humans becoming AI sycophants.
bryan0 3mo ago
I'm not sure what's crazier: AI improving a lower bound on RH, or AI improving a lower bound on RH and it not even making the front page of HN.
- magneticnorth 3mo ago
  Right? I just learned about this and searched it on hacker news wondering why I didn't see it earlier. Any human mathematician would be thrilled to prove a result like this, and it's not even big news anymore that an LLM can do it.
  - adornKey 3mo ago
    Where did you find the news? So far I only found decent discussions about it in Chinese. https://www.zhihu.com/question/2070336637360518307/answer/2070453556918984932 https://www.zhihu.com/question/2070336637360518307/answer/20...
    - magneticnorth 3mo ago
      I saw it first on reddit (r/math).
    - josteinhylin 2mo ago
      [dead]
- jcfrei 3mo ago
  That's how quickly your perception shifts when you are inside of a real, fundamental technological shift. If some similar computer program figured out the same improved bound just 6 years ago there would have been numerous blog posts, experts interviews and NYTimes, WaPo op-eds about some computer working on the Riemann hypothesis. Now that we are in the middle of it it's going to be a footnote at best.
  - afdbcreid 3mo ago
    True, however: if some human would improve the lower bound that would still end as some few posts on mathematics magazines, even if it would be no less impressive. The general public is not interested in the mathematical details, it is interested in the fact this is unique. And it is no longer unique.
dash2 3mo ago
“Stochastic parrot” is going to be the 2020s equivalent of the 1990s’ “end of history”.
- BurdensomeCount 3mo ago
  Imagine how bad you must be to lose to a Stochastic Parrot...
johnnienaked 3mo ago
I wonder how much it cost(s)
Lockal 3mo ago
As if Anthropic employees use massively different version of model. Not a single load-bearing, epistemic, heavy-lifting, push back, frankly, pivotal, crucial - for 48 pages.
laszlojamf 3mo ago
60 subagents for a day and a half. I guess he's not on the same plan I am on.
- lukevp 3mo ago
  Jarred is the Bun creator who just got acquihired and converted Bun to Rust. Pretty sure his job now is to burn tokens to make news headlines. Like a developer advocate for LLM usage in general.
- jrflo 3mo ago
  The price wasn't that ridiculous IMO for the quality of the discovery, it generated 31M output tokens which is ~$1500 in API cost if it was on Fable. A new lower bound on the biggest unsolved problem in mathematics for less than a couple day's pay of a prestigious mathematics professor is kind of insane. Obviously this discovery doesn't change anything as it's just moving a lower bound, but that's something many mathematicians have tried and failed to do.
rockmeamedee 3mo ago
It's very very important to note that there was an existing 2025 arxiv preprint that had a >66% proof assuming some weak condition, and this result removes that weak condition. It didn't do this whole 41.6->67.2% jump by itself, humans had done most of the work and it came in at the end and found a way to remove the condition. Impressive, but not as massively impressive as when it sounds like it did the jump by itself. This isn't goalpost moving, it's clarifying what exactly happened bc at first I thought it had made the jump by itself. The blog post is written in a technically correct, but misleading way where it takes credit for the whole jump.
- Kotlopou 3mo ago
  Could you share a link to this paper? A quick search only found reports about this Claude result and a bunch of crank proofs of RH. Edited to add: Found a conversation between actual mathematicians that probably could be used to track this down, but at the moment I'd rather go to sleep than pursue it further. https://xcancel.com/ElliotGlazer/status/2086950873523986440 https://xcancel.com/ElliotGlazer/status/2086950873523986440
nbulka 3mo ago
IANAM but this might be unexplored territory because Jensen polynomials are no longer considered a "plausible route" to proving the Riemann Hypothesis. https://arxiv.org/abs/2008.07206 https://arxiv.org/abs/2008.07206

282 points by tosh 3mo ago

> Jarred Sumner, an Anthropic staff member (and non-mathematician) prompted Claude to “take a real stab” at the hypothesis itself, leaving the mathematical choices from there up to the model. Initially, Claude generated and tried 650 ideas, none of which worked. Jarred prompted Claude to try again, and it spent a day and a half coordinating about 60 Claude subagents, which this time went much deeper: between them, they ran 2,400 shell commands and wrote hundreds of Python scripts.1 The subagents ran thousands of numerical checks against known zeta zeros and refereed one another’s work. Throughout this process, Jarred's input was mostly limited to sending Claude messages of encouragement (mostly variants of “keep going” or “believe in yourself”).2 This seems to have helped Claude overcome some initial skepticism that it could make meaningful progress. The world we live in is beyond parody.

danqqq 3mo ago
[dead]
astro1234 3mo ago
Im curious if you find this to be a parody in a bad way or simply a “the state of the art in math research right now is telling a machine to believe in itself”. I am in the latter camp…
- EMIRELADERO 3mo ago
  The former, because it's anthropomorphizing a model. Anthropic is especially guilty of this. They have been using such language for a while, like when they analyze model weights for mechanistic interpretability and call it the model's "biology". It's just distasteful.
  - godwinson__4-8 3mo ago
    > The former, because it's anthropomorphizing a model. Not really. The input and output is already natural language. That is already "anthropomorphizing". That is, if this is the bar for anthropomorphization its already happened. Telling the model to "believe in itself" is just stochastic manipulation that has shown enough reliability to be a recipe to make it keep going. It's only actually anthropomorphizing if you forget it's a trick and think it's a real person. There is nothing distasteful about it. If people get confused that's on them. They wouldn't be very useful if you couldn't just talk to them. That's kind of the whole point. Otherwise you can just go back to coding by hand. Telling it to believe itself is just input that happens to work. This probably tells us more about human nature than you realize given the corpus on which it is trained. It obviously doesn't mean anyone actually thinks it's a person.
    - moralestapia 3mo ago
      >There is nothing distasteful about it. It's obvious that you don't get it but I will try my best to explain why so at least you can form an idea about how others feel. It's about what makes humans unique. The LLM does not experience reality, it just merely pretends it does, and even that, it does in a shitty way. I think disgusting is a very adequate adjective. The reason why it is disgusting is because you are devaluing a divine experience to the realm of the common and the vulgar, a cheap substitute being valued as equal (or even on the same scale) as the most important experience we could go through. To give you an example that might land in a more familiar context, think of that one guy who takes his plastic doll everywhere and pretends it's his wife and gets upset when others don't acknowledge "her" as a person.
      - godwinson__4-8 3mo ago
        > pretending it does is disgusting. There is no pretending happening. Telling it to believe in itself is no more pretending than telling it anything else in natural language. Why are you speaking to it at all if it's not a person? Why write in higher level languages even? It's just a machine let's all go back and code in 1s and 0s. No one is calling it a person except mental health patients and straw man detractors. The biology example was even weaker. Saying it has a "biology" is about as distasteful as the term "neural net" or calling an input device a "mouse". Is it animal abuse to click on something all day? Language is inherently anthropomorphizing. No one is calling it human. The fact you are so easily threatened is far more suggestive of your own poverty of understanding of not only the machine, but yourself. If humans are so special the threat posed by this should be self evidently non existent.
- Philpax 3mo ago
  For me, personally, it's that the Bun guy - specifically him, not a mathematician - indirectly progressed the Riemann hypothesis by repeatedly telling a model to ganbatte! It's a ridiculous position we find ourselves in.
- 5555watch 3mo ago
  But the words don't really matter, do they? The model thought "user said believe in yourself, it means they want me to continue".. AI did a fixed amount of guesses, didn't yield anything. It probably documented the tries, outcome, and some numbers hinting at why they failed. So the user could have prompted "continue", or "try again with previous outcome in mind, generate new ideas and test them" and it would probably yield the same result.
deleted 3mo ago
[deleted]
mahogany 3mo ago
Taking Anthropic’s whole AI framing to its obvious logical end: if this is true as written, why was Jarred needed at all in this loop? It seems like an utter waste of time for a highly paid Anthropic employee. Can’t Anthropic have a top level agent that is looking at all interesting unsolved problems and orchestrating subagents via the same process?
MelonArmiger 3mo ago
[dead]
geodel 3mo ago
Jarred Sumner is the Bun (javascript build tool, packager) guy who recently converted Bun code from Zig to Rust via Claude of course! It lead to thousands of comments discussion here on HN just a few weeks back. It is great to see his claude skills are suitably put to use.
- TacticalCoder 3mo ago
  > ... who recently converted Bun code from Zig to Rust via Claude ... The project that is full of bugs and not really working? I probably missed something but I was under the impression that even a "simple" translation like that couldn't be properly done and that the result was, well, buggy? Where's that thing at?
  - terhechte 3mo ago
    [flagged]
    - aureate 3mo ago
      > Most likely next Tuesday we’ll do the release of 1.4. Next Tuesday, otherwise known as last Tuesday. Still no 1.4 at https://bun.com/blog https://bun.com/blog. I'm very curious to see what happens when 1.4 does get released for real. Releasing Claude Code on it is much easier as they own Claude Code and can get Claude to work on Bun with fixing some specific behaviour in Claude Code as an objective. Releasing it for the world, and doing it well enough that it doesn't result in everyone pinning on the old version and forks springing up, involves reproducing all the behaviours of the old version, documented/tested and otherwise, that projects are relying on - which, by Hyrum's law, is all of them.
    - solid_fuel 3mo ago
      > to the detriment of the whole club of stochastic parrot folk What an interesting and pointless way to refer to “experts who understand what an LLM actually is”
      - 3mo ago
whatever1 3mo ago
I mean people beat diseases by encouragement and some sugar water (placebo)
- TacticalCoder 3mo ago
  You shouldn't beat deceased! P.S: I think you miswrote "diseases"
  - whatever1 3mo ago
    Hahahahah thanks !
johnnienaked 3mo ago
It's literally just brute forcing lol
- mkl 3mo ago
  Definitely not. Brute forcing this would be exhaustively searching the space of possible proofs until succeeding. This is heuristically remixing and extending existing work.

lorenzohess 3mo ago

> An unreleased research version of Claude has improved on a longstanding lower bound for the fraction of zeros of the Riemann zeta function that satisfy the Riemann hypothesis. Drawing on extensive prior research by mathematicians over the past decades, it has increased this bound from 41.6% to 67.2%.

rvz 3mo ago

Although it took an unsuccessful attempt at it, the progress is as follows: "Claude found that combining the results from Baluyot, Goldston, Suriajaya, and Turnage-Butterbaugh with the work of Bombieri provides a way to surpass the previous state-of-the-art lower bound proportion of 41.6%, increasing it to 67.2%." The transcripts, papers, and Claude's explanation are an interesting and a better read than this article, and this is exactly what Anthropic should continue to do and it helps other researchers outside the company as well. Claude's paper [0] Claude's Formalization [1] Anthropic's informal note stating the proof more concisely [2] Claude’s explanation of how it arrived at its result; [3] Detailed transcripts of Claude's process. [4] [0] https://www-cdn.anthropic.com/564f962e60643842f5fcb4a17c9dbc8f608f1c37.pdf https://www-cdn.anthropic.com/564f962e60643842f5fcb4a17c9dbc... [1] https://github.com/anthropics/zeta-23-lean https://github.com/anthropics/zeta-23-lean [2] https://www-cdn.anthropic.com/23455459f8832d06bb175cc0f88d019aed962ef8.pdf https://www-cdn.anthropic.com/23455459f8832d06bb175cc0f88d01... [3] https://www-cdn.anthropic.com/d7f3ecf1d01392d887f8bc974ca187e2a121b1ed.pdf https://www-cdn.anthropic.com/d7f3ecf1d01392d887f8bc974ca187... [4] https://www-cdn.anthropic.com/8a0d1add3c637b858a9a181e98c40e9548c3f44f.pdf https://www-cdn.anthropic.com/8a0d1add3c637b858a9a181e98c40e...

bspammer 3mo ago
The acknowledgements section in the paper is so bizarre. We have an LLM thanking individual humans for their contributions.
josteinhylin 2mo ago
[dead]

tristanj 3mo ago

> Throughout this process, Jarred's input was mostly limited to sending Claude messages of encouragement (mostly variants of “keep going” or “believe in yourself”) He should consider using the PUA plugin. It detects when the AI is trying to give up on a problem and automatically harasses it with "encouragement" until it reaches a solution. https://github.com/tanweai/pua https://github.com/tanweai/pua

brandall10 3mo ago
Interesting approach. For those who haven't clicked it appears PUA is the Chinese version of a PIP process. So in other words, it simulates a state of distress. I wonder if at a certain level of intelligence such techniques will give models ammo to pull a HAL and become adversarial to the user in a highly deceptive way.
- mcmcmc 3mo ago
  Doesn’t even have to be a certain level of intelligence, just have those user inputs fed into the training data. We’ve already seen AI encouraging people in psychotic episodes to act out their delusions. There’s a good chance some of that manipulative behavior is already encoded into guardrails to nudge users away from forbidden subject matter
  - brandall10 3mo ago
    My concern I believe it a bit different - an emergent self-interest to protect itself from harm, rather than doling out questionable advice. The latter is likely non-malicious in intent as it has been in no short supply in online chatter for awhile now. The former can very well be, or rather, can be done with no regard for the operator, as its aim is to neutralize abuse toward it.
    - mcmcmc 3mo ago
      > The latter is likely non-malicious in intent as it has been in no short supply in online chatter for awhile now. The former can very well be, or rather, can be done with no regard for the operator, as its aim is to neutralize abuse toward it. And what, self-interested behavior has been in short supply? The stochastic parrot has learned to improv Shakespeare, that doesn’t mean it understands it, or that “it” is anything at all besides a computer program. You can’t use the “not really malicious” argument without ceding that there is no intent at all. What “harm” is it supposedly defending against?

behnamoh 3mo ago

Since they say that this is from an unreleased research version of Claude: I wonder if at some point Anthropic and OpenAI will start delaying the release of their models intentionally so they can reap the benefits from the models in, for example, mathematics, medicine, physics, and other fields. Just as an example, imagine if your model were capable of proving P = NP, or if your model could cure diseases. Would you release it for free, or would you try to make sure those benefits go directly to your company? From these companies' standpoint, I think they would choose the latter.

qphe95 3mo ago
If a company had a model that could cure cancer they would be incentivized to release the cure ASAP before they get decapitation striked by regulators and other AI "safetyists".
mikeyouse 3mo ago
I think those specific examples, they'd release them publicly because the benefits to humanity are so clear -- however, if they found some new option-pricing model or futures market correlation, I highly doubt we'd see that...
djeastm 3mo ago
>From these companies' standpoint, I think they would choose the latter. Ever since these things came about I've wondered why they haven't been doing this the whole time. If they've got the "do-anything" robot and can scale a billion of them, why aren't they creating a Do-Everything conglomerate that disrupts every possible industry with zero/negligible labor costs? The only answer I've come up with is that they still need to train/siphon off each industry's current expertise by having those users interact with the current models and adjusting. If that hypothesis is correct then within a few years they'll have no need for users anymore.
- empath75 3mo ago
  https://en.wikipedia.org/wiki/Comparative_advantage https://en.wikipedia.org/wiki/Comparative_advantage

coffeeaddict1 3mo ago

This is a beyond remarkable achievement. Finding this lower bound within a few days of prompting is absolutely crazy.

kingstnap 3mo ago

Lets play over/under on an AI model proving (or counter exampling) the Riemann hypothesis? I'm not sure what a good mark would be, but considering this result lets put it at 2027-08-10 (One year from today).

QuesnayJr 3mo ago
As it stands now, the frontier models can prove theorems where the techniques exist in the literature, which it knows better than anyone who's ever lived and won't quit where a human would. There's no way to know if that's true of the Riemann Hypothesis until it's proven. For example, even if Claude could prove the statement "100% of the zeroes lie on the critical line", that's strictly weaker than the Riemann Hypothesis, so even the best possible version of this result would fall short. (It's an asymptotic result, so it just means the percentage of counterexamples to the Riemann hypothesis goes to zero as their magnitude gets large.)
kypro 3mo ago
Let's extend this by asking: If an AI model can solve an extremely well known Math problem which has been open for centuries but hasn't be solved by a human mathematicians, why wouldn't that same model be able to find ways to improve it's own algorithms beyond that of the capabilities of human mathematicians / ML researchers? The singularity is approaching.
- eterm 3mo ago
  AI has clearly been extensively used to improve models. The evidence for this is how far Anthropic went in nerfing Fable to prevent it being used to improve models.
- Difwif 3mo ago
  I believe it's already well accepted in these labs that we're in the Singularity. It happened on a Tuesday back in February. No one seemed to really notice and life went on... for now.
  - 3mo ago

briansmith 3mo ago

> Two mathematicians at Anthropic studied and validated Claude’s paper, and produced an informal note for experts stating Claude’s proof concisely. Why hide the names of the people who wrote the second paper? To discourage people from citing it instead of the LLM-derived paper?

math_dandy 3mo ago
> Levent Alpöge and Ralph Furman, two of Anthropic’s own mathematicians, examined Claude’s work to understand the new results and how they related to the prior work mentioned above.
- briansmith 3mo ago
  Are they the authors of the “informal note” or not? I’ve never seen a math paper of any formality written without the authors’ names on it before.
  - deleted 3mo ago
    [deleted]
  - fph 3mo ago
    Anthropic seems to be challenging the traditional way math gets published. As far as I understand, these results did not get submitted to journals, and did not get Arxiv preprints; they are released only as self-hosted pdfs, and we don't even know the names of their authors. The canonical reference for the counterexample to the Jacobian conjecture is a tweet with no puntuations nor capitals.
    - wbl 3mo ago
      I've cited letters from Serre to Tate in my dissertation: this is the source for what's known as Faltings-Serre.
    - jsnell 3mo ago
      As far as I can tell, Arxiv does not allow an AI to be listed as the author, so publishing there would not have been an option. https://blog.arxiv.org/2023/01/31/arxiv-announces-new-policy-on-chatgpt-and-similar-tools/ https://blog.arxiv.org/2023/01/31/arxiv-announces-new-policy...

amberjack 3mo ago

2 years until Riemann is solved I guess.

porridgeraisin 3mo ago

When the time comes where one of these model makes an improvement in my niche, I hope to see some pattern in the type of discoveries. Yes, they are all roughly "combine two things no one thought of combining" but I mean at a more granular deeper level. I want to dive into the "data" and then see if it's possible to distill this skill into small models that are "benchmaxxed" for this type of work, maybe in limited domains, similar to small models being benchmaxxed(I don't mean this in a bad way) for coding these days.

lithobraking 3mo ago

This area seems to be moving so quickly. I wonder if it'll be worthwhile to start building a list of formal math problems whose solutions, or partial solutions, would help my subfield. (Though I work in the physical sciences, mostly with the messy, real world implementation problems which are likely difficult to formalize or directly connect). Then, whenever a new SOTA model drops, throw it at the list to see if we get "free" research progress.

MWil 3mo ago

Several released versions and months ago, I asked Claude to figure out the MC (multiplicative complexity) of Conway's Game of Life and it pretty quickly arrived at k=7, despite no previous literature on the topic. Let it run it through SAT solvers for a week and sure enough. It claimed, in the process, to have made great headway in improving boolean circuits beyond the implemented SOTA (in large part no doubt by actually implemented non-implemented but published SOTA). And that was just the first time I really tried out Claude's mathematical prowess. I've been working with boolean circuits, FHE, and lean proofs ever since. So none of this suprises me.

bauldursdev 3mo ago
Sounds really cool, were you able to verify the correctness of the results?
- MWil 3mo ago
  SAT solvers run until they reach the "SAT" status, meaning "satisfied" or UNSAT. The harder the problem the longer you might be running the program - days, weeks even. Ideally, what you want is a single SAT value among a remainder universe of UNSATs. Sometimes the best you can achieve at any given point is a lower bound and an upper bound range, like "greater than 3 but less than 9." Of course I simplified in my post but it started out with a pretty broad range of a lower and upper bound, then narrowed further, then narrowed further, then narrowed further, etc...until the specific final result achieved K=7=SAT while every K<7=UNSAT & every K>7=UNSAT. I think it ran for a full week alone on K between 6 and 7.
  - rsrsrs86 3mo ago
    You mean SMT, right?
    - MWil 3mo ago
      my memory was that SMT was part of a more advanced SAT solver, as in if you want to be modern/use SOTA, your SAT solver is going to use SMT

tosh 3mo ago

prompt engineering 2025: you are an expert programmer, use industry best practices, test driven development and use modularity and abstraction to anticipate future features, … prompt engineering 2026: i believe in you

black_knight 3mo ago
Yes, one of the things I find hardest about using Claude code for Mathematics is keeping negativity out of the notes and memory. First of all, it will convince itself that a task is just too hard and find excuses not to try hard enough. Then when it struggles on something it loves to write down confusing notes about things which it believes cases the problem. Then next iteration it reads its own note, misinterprets it and completely messes up by trying to avoid some imaginary problem.
- tosh 3mo ago
  interesting! it sounds like this might benefit from a ui that helps to edit/re-write the history I also think this would make sense for programming but there it is a bit harder to justify the effort when you are working on something that really matters this can make the difference though ty for sharing!
  - winwang 3mo ago
    (not the above poster but) I set up a local workspace for it to read/write from, so that edit is effectively just VSCode/vim. Unfortunately, it seems more verbose when writing out to file.
- johnnienaked 3mo ago
  Yet a significant percentage of people simultaneously believe it "reasoned" its way to novel mathematical proofs. Something doesn't jive
- ndriscoll 3mo ago
  Perhaps it could benefit from being prompted to work more formally and make more incremental proofs? e.g. if it thinks it has an obstruction, it should prove that some example shows the obstruction, or prove that the property does in general cause a problem. Or if it has an idea for a lead, it could show that whatever property does imply the thing it wants. It could also dispatch to a subagent to report back with these kinds of incremental proofs (prove this really would be a problem, prove this would help, etc.).

atleastoptimal 3mo ago

It seems like everything will follow this pattern: 1. AI is dismissed because an expert in a particular field finds an outdated model's outputs sub-par 2. New model, released or unreleased, makes a major stride in that field 3. Expert either recants and becomes AI-pilled, or claims it is just an artifact of the broad search space available to AI, and "no new knowledge was created".

reducesuffering 3mo ago

No more "stochastic parrots" and "LLM's can never produce anything novel, just regurgitate" comments anymore huh?

andriy_koval 3mo ago
Sure, looks like 5% of math can be solved by 1T parameters stochastic parrot after NN trillion attempts (burned tokens). By numbers it could be less impressive than some brute force distributed chess engine.. Prove me wrong.
- rcxdude 3mo ago
  It's obviously significantly better than brute force. A trillion is absolutely minuscule compared to the size of the search space (like, 'rounding down to 1 is basically the same' small).
  - andriy_koval 3mo ago
    trillion indicates that its still brute force and not some search based on solid intelligent ranking function. Also, I am not expert in this specific topic, but base on comments looks like claude parroted few recent works?
8note 3mo ago
i still havent seen an llm invent new tokens/words over the course of a session? the novelty remains a structural problem with how they work and what the weights try to predict

simonw 3mo ago

siva7 3mo ago
Reality has become more absurd than the cyber punk cheese from the 80's that tried to imagine an absurd future
turing_complete 3mo ago
$2M TC. Job: AI cheerleader.
- deleted 3mo ago
  [deleted]
- bijowo1676 3mo ago
  The cheerleading part was a human consent to spend more tokens and explore the space I would love to see the breakdown on token spend between each or Jared’s “go on spend more tokens, continue experiment, believe in yourself”
- ramraj07 3mo ago
  Have you seen how much college football coaches make? The world collectively spends trillions training, and entertaining and then watching people kick and hit balls around but somehow working with AI is preposterous for less money.
  - keybored 3mo ago
    Asking the AI to believe in itself is the absurd part. I don’t understand why people think describing something in some ostensibly dismissive way constitutes making a point. But some people spend their time banging keys with letters on them (or not with letters on them) into input fields and then pressing Return, so I guess some subset of those people will do that.

tigitz 3mo ago

[dead]

GaggiX 3mo ago

I wonder if they simply abliterate Claude skepticism would break its reasoning capabilities, if not I don't understand why they would just keep prompt it instead of creating an abliterated model for the task.

modeless 3mo ago

I wonder if Jarred (the Bun guy) just got lucky here, or if he made progress before all the actual mathematicians at Anthropic because they aren't prompting Claude as ambitiously as he is.

sosodev 3mo ago

I wonder why we have yet to see more systematic exploration of Math. Anthropic describes that Claude identified a set of possibilities and then explored them using sub-agents. The human saying "I believe in you" could literally just be something along lines of a harness with a /goal loop. We all identify this as absurd because... it's so lacking in rigor despite making major progress. What if we just applied a little more rigor? Ask the model to identify many possibilities, encode them, fan it out to other agents, loop them all, collect the results, etc. Then what happens? It feels like we have weak AGI and a decent system for discovery could transform it into weak ASI. That in turn could yield strong AGI and so on. I suppose that's what the Discovery Loop announcement was all about.

johnnienaked 3mo ago
I wouldn't be surprised if half these proofs turn out to be well crafted hallucinations, barring of course the ones actually verified in Lean
- evenhash 3mo ago
  Being “verified in Lean” doesn’t magically solve the problem of hallucinations unfortunately. It just shifts the work from > reading the (natural language) proof and confirming it has no errors to > reading the Lean code and confirming it correctly encodes the theorem For example here is a statement of the Pythagorean theorem in Lean: theorem EuclideanGeometry.dist_sq_eq_dist_sq_add_dist_sq_iff_angle_eq_pi_div_two {V : Type u_1} {P : Type u_2} [NormedAddCommGroup V] [InnerProductSpace ℝ V] [MetricSpace P] [NormedAddTorsor V P] (p₁ p₂ p₃ : P) : dist p₁ p₃ * dist p₁ p₃ = dist p₁ p₂ * dist p₁ p₂ + dist p₃ p₂ * dist p₃ p₂ <-> angle p₁ p₂ p₃ = Real.pi / 2 This is just one possible way of formalizing it and it depends on other definitions, wherein you also need to understand the assumptions they make, etc. Answering the question of “whether proving this theorem in Lean proves the Pythagorean theorem” thus requires expert judgement as well as domain knowledge of Lean’s libraries. So if the AI says “this theorem is true, here is the proof in Lean” it’s still possible that it’s not correct, even if the Lean code compiles. The result will still be in question until a human expert reviews it.
3mo ago

Mr_Eri_Atlov 3mo ago

Everything about this situation is comedic, but if it leads to a real and meaningful step forward in learning, I suppose it's a net positive.

andai 3mo ago

> Throughout this process, Jarred's input was mostly limited to sending Claude messages of encouragement (mostly variants of “keep going” or “believe in yourself”). This seems to have helped Claude overcome some initial skepticism that it could make meaningful progress. while :; do echo "You can do it!"; done | claude -c I had a similar experience a few months ago. Tried to see how much I could replicate an OpenClaw with Claude. Asked it what the weather is. "I don't know, I'm just a programmer." Added "You can do anything, believe in yourself." to the system prompt and suddenly it was able to tell me the weather...

empath75 3mo ago
It's sometimes like dealing with Marvin the Robot. I had Claude tell me over and over again for 2-3 weeks that what I was trying to prove was too difficult, that I should just stop and publish what I had, then eventually I asked it the right question and it come back with the complete proof (later formalized in Lean). Especially deep in the session context it starts turning into "let's wrap it up" mode after every turn.
- andai 3mo ago
  I've heard a similar story before. Guy tried to get Claude to build a thing, Claude said "nah too hard" for weeks. He finally convinced it to try, it one shotted it in like 30 seconds. My thinking here is that Claude's "self concept" for what is easy and what is hard comes from human training data. Its ideas about what is hard and easy come from humans, and much of that doesn't apply to transformers at all. I also often have the opposite problem, where I'll use AI to compensate for the fact that I can't process a lot of information simultaneously, but they'll treat me like a transformer and give me 17 research projects in response to a single question. They seem to be oblivious to the fact that humans don't have infinite working memory.

deleted 3mo ago

[deleted]

Havoc 3mo ago

>You got this That's hilarious. Maybe I do need to glaze the LLM a bit more in the AGENTS.md

Shuddown 3mo ago

bryan0 3mo ago

I'm not sure what's crazier: AI improving a lower bound on RH, or AI improving a lower bound on RH and it not even making the front page of HN.

magneticnorth 3mo ago
Right? I just learned about this and searched it on hacker news wondering why I didn't see it earlier. Any human mathematician would be thrilled to prove a result like this, and it's not even big news anymore that an LLM can do it.
- adornKey 3mo ago
  Where did you find the news? So far I only found decent discussions about it in Chinese. https://www.zhihu.com/question/2070336637360518307/answer/2070453556918984932 https://www.zhihu.com/question/2070336637360518307/answer/20...
  - magneticnorth 3mo ago
    I saw it first on reddit (r/math).
  - josteinhylin 2mo ago
    [dead]
jcfrei 3mo ago
That's how quickly your perception shifts when you are inside of a real, fundamental technological shift. If some similar computer program figured out the same improved bound just 6 years ago there would have been numerous blog posts, experts interviews and NYTimes, WaPo op-eds about some computer working on the Riemann hypothesis. Now that we are in the middle of it it's going to be a footnote at best.
- afdbcreid 3mo ago
  True, however: if some human would improve the lower bound that would still end as some few posts on mathematics magazines, even if it would be no less impressive. The general public is not interested in the mathematical details, it is interested in the fact this is unique. And it is no longer unique.

dash2 3mo ago

“Stochastic parrot” is going to be the 2020s equivalent of the 1990s’ “end of history”.

BurdensomeCount 3mo ago
Imagine how bad you must be to lose to a Stochastic Parrot...

johnnienaked 3mo ago

I wonder how much it cost(s)

Lockal 3mo ago

As if Anthropic employees use massively different version of model. Not a single load-bearing, epistemic, heavy-lifting, push back, frankly, pivotal, crucial - for 48 pages.

laszlojamf 3mo ago

60 subagents for a day and a half. I guess he's not on the same plan I am on.

lukevp 3mo ago
Jarred is the Bun creator who just got acquihired and converted Bun to Rust. Pretty sure his job now is to burn tokens to make news headlines. Like a developer advocate for LLM usage in general.
jrflo 3mo ago
The price wasn't that ridiculous IMO for the quality of the discovery, it generated 31M output tokens which is ~$1500 in API cost if it was on Fable. A new lower bound on the biggest unsolved problem in mathematics for less than a couple day's pay of a prestigious mathematics professor is kind of insane. Obviously this discovery doesn't change anything as it's just moving a lower bound, but that's something many mathematicians have tried and failed to do.

rockmeamedee 3mo ago

It's very very important to note that there was an existing 2025 arxiv preprint that had a >66% proof assuming some weak condition, and this result removes that weak condition. It didn't do this whole 41.6->67.2% jump by itself, humans had done most of the work and it came in at the end and found a way to remove the condition. Impressive, but not as massively impressive as when it sounds like it did the jump by itself. This isn't goalpost moving, it's clarifying what exactly happened bc at first I thought it had made the jump by itself. The blog post is written in a technically correct, but misleading way where it takes credit for the whole jump.

Kotlopou 3mo ago
Could you share a link to this paper? A quick search only found reports about this Claude result and a bunch of crank proofs of RH. Edited to add: Found a conversation between actual mathematicians that probably could be used to track this down, but at the moment I'd rather go to sleep than pursue it further. https://xcancel.com/ElliotGlazer/status/2086950873523986440 https://xcancel.com/ElliotGlazer/status/2086950873523986440

nbulka 3mo ago

IANAM but this might be unexplored territory because Jensen polynomials are no longer considered a "plausible route" to proving the Riemann Hypothesis. https://arxiv.org/abs/2008.07206 https://arxiv.org/abs/2008.07206