7 ms·
I like how everyone laughed when OpenAI said their models will have "PhD-Level Intelligence" and now the goalpost has been moved to if AI can create new math (i
by raincole 5mo ago
I like how everyone laughed when OpenAI said their models will have "PhD-Level Intelligence" and now the goalpost has been moved to if AI can create new math (i.e., not PhD-Level, but Leibniz/Euler/Galois level.)
- zeofig 5mo agoI still laugh.
- johnfn 5mo agoHave you updated your priors after this announcement? If not, why not?
- xyzsparetimexyz 5mo agoPrior whats?
- Jblx2 5mo agohttps://en.wikipedia.org/wiki/Prior_probability https://en.wikipedia.org/wiki/Prior_probability
- se4u 5mo agoWhen a qualifying noun is absent , then priors means prior beliefs.
- xyzsparetimexyz 5mo agoPrior to what? Why not just say beliefs?
- niek_pas 5mo ago"Update your priors" is a common expression in English: https://en.wiktionary.org/wiki/update_one%27s_priors#English https://en.wiktionary.org/wiki/update_one%27s_priors#English
- staticman2 5mo agoYour wiktionary link indicates it is not a common expression in English but instead something "rationalist community" people say.
- pocksuppet 5mo agoHN is a rationalist community hangout.
- repparw 5mo agowe're reading comments on a post about math proofs
- rob 5mo agoNo it's not. Where do you come up with this? Just because you searched the phrase on Google and there's a single result for it on a wiki? Who do you know that's using this expression regularly?
- eudamoniac 5mo agohttps://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=update%20priors&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
- happytoexplain 5mo ago"Common" is an exaggeration.
- zeofig 5mo agoI don't have enough information about the announcement for it to mean much to me. I don't know much about this field of maths. I don't know how many mathematicians were actively working on this problem. It could be zero, which would indicate it's not really that interesting. The article gushes about how it's a Very Important Problem, but it's not even mentioned on https://en.wikipedia.org/wiki/List_of_conjectures_by_Paul_Erd%C5%91s https://en.wikipedia.org/wiki/List_of_conjectures_by_Paul_Er.... I'm sure the busy folk at openAI will fix that soon however. Furthermore the extensive dishonesty of companies like openAI makes me suspicious of just how this was achieved. Overall the announcement is of little interest to my "priors", although I don't typically think in such terms.
- signatoremo 5mo agoYou don't have enough knowledge to dismiss them, but you still laugh? For?
- zeofig 5mo agoDo you have enough knowledge? I laugh at everyone who accepts these claims in the light they're presented despite knowing so little.
- pickleRick243 5mo agoYou don't know the names of the mathematicians who've given their thoughts on this? If not, you really should just not comment on anything mathematical ever again.
- zeofig 5mo agoI do know their names. However I'm not in the field and there are many cases in recent years of high-profile scientists putting their weight behind highly dubious claims. Thanks for the advice, by the way. Note that I'm not disputing the validity of the counterexample itself.
- ex-aws-dude 5mo agoYes let me calculate the exact change it’s 0.004748394 probability now based on my own made up statistical vibes that I feel
- deleted 5mo ago[deleted]
- dawnerd 5mo agoYet it still codes like a junior developer that memorized all of stack overflow.
- dilap 5mo agoPersonally I don't find this to be true anymore! It's not always great and does still will often tend towards unneeded complexity (especially if not pushed a bit), but I often find GPT 5.5 writing code I would have written myself. This was very much not true with earlier models (who make something that worked, but I'd always have to rewrite to make it "good code").
- dawnerd 5mo agoPersonally I found 5.5 a massive step back from 5.4. Both of them still use way too many fallbacks and unnecessary checks, especially if you're having it output php. It's fine if you're just one person and checking everything and able to catch and correct. But it's really bad when you have a team all using it, not checking the output and trusting it's output leading to spaghetti code. Technically works, but very messy and will no doubt lead to buggy code. It still writes like a junior dev, in that despite AI being able to get a picture of an entire repo, it's changes are typically confined to the task it's working on and will opt to duplicate logic to keep changes contained. Again, technically works, not ideal.
- esikich 5mo agoIdk man, I think at this point, if you can't get good code out of frontier models, you're doing something wrong. Plenty of resources out there for you to familiarize yourself with the workflows if you can be bothered.
- _zoltan_ 5mo ago100%. this is what I tell people who fail at using these models. the model isn't the problem anymore.
- 5mo ago
- xgulfie 5mo agolarge language models do not have pigeon-level intelligence. They can't even feed themselves.
- staticman2 5mo agoWhat's laughable is an OpenAI employee invented the term "PHD level intelligence" and you think that " PHD Level intelligence" is a real term that describes a real thing and you are repeating it here.
- mathisfun123 5mo ago[flagged]
- perching_aix 5mo agoThinking you're magically smarter than others is indeed an essential part of the NPC trend, to the extent that it in itself becomes an NPC thing to say. It's pretty much a 1:1 match to the "we're all unique snowflakes" meme, with an army of Buzz Lightyear toys repeating the same in the background.
- ptrl600 5mo agoIt's clearly smarter than any PhD and dumber than any ant.
- zamadatix 5mo agoMy only complaint is the claims always start spreading 6-12 months before the delivery. A little patience goes a long way in what's possible with AI and we all just have to wait and see what parts actually grow this next cycle or not. Guessing at it based on trend lines only leads to people getting excited when it matches their particular guess and ignoring it when it doesn't.
- melagonster 5mo agoThey said that their specific version of model had has this ability one year ago.
- zamadatix 5mo agoI can't open all of the links in the article because of some Cloudflare issue (perhaps related to me being on a plane) but is the version and sub iteration of the model they actually used for this the same as the one they announced the capability on a year ago? If so, did they comment why didn't they just show this a year ago (they seem to have been publishing successively better results slowly instead).
- melagonster 5mo agoIn my memory, it was OpenAI o1. OpenAI o1 was released in September 2024. It seems the difference came from the difference between benchmark performance and propagation.
- no-name-here 5mo ago>> OpenAI said their models will have "PhD-Level Intelligence" > My only complaint is the claims always start spreading 6-12 months before the delivery. If delivering on such promises "always" occurs 6-12 months after the promise, is that pretty good?
- zamadatix 5mo agoAgain, the promise isn't _always_ delivered, people just more often focus on when the particular result aligns with their view. When it is though, it's all too commonly 6-12 months later. Which is nice but a bit annoying - why not wait 6-12 months and claim when you can actually show it? Or just say that's where it might be soon instead of talking like it is now. I generally like AI and use it plenty often, it does many things well and I'm curious to see how far it keeps going, but that doesn't mean I have to like overhyped marketing about it.
- necovek 5mo agoPhDs used to mean publishing a novel mathematical result: when has that changed?
- bananaflag 5mo agoThey mean "new math" in the sense of more than a novel mathematical result, a new math paradigm or so.
- kamaal 5mo agoThats coming too. Some times when you go some distance with a subject generates data for new ideas. Once math gets done fast, newer ideas and paradigms also arrive.
- tedbradley 5mo agoMy good sir or madam, disproving a decades-old conjecture produced by Erdos that has had armies of people in that field have their go at it IS a novel mathematical result.
- deleted 5mo ago[deleted]
- raincole 5mo agoThe gap between novel result and "new math" is as wide as the pacific ocean.
- melagonster 5mo agoSo finally they reach a part of PhDs level. Current version rely human to integrate results from they model and writting the papper. If LLMs/AIs can do all thing above, we can exactly get a PhDs level model.
- golol 5mo agoNo it is not Leibniz/Euler/Galois. More like writing good papers that contribute to the broader understanding of a theory. I think if one evaluated a mathematicians research output and it consisted of mostly the kinds of problems AI has solved so far, it would give the impression that this person is somehow very good at picking accessible problems to target, but has not made a larger impact on the field.
- bananaflag 5mo agoAs a mathematician, new, conceptual math is when I'll become interested in reading LLM output. I appreciate very much the work done so far, but this sort of asymptotic/quantitative result didn't interest me much even when it was done by humans. (This is not snobbery, just a personal preference.)
- kamaal 5mo agoWell that's coming. As a matter of fact more logic and structure to your work, the more easy it is for AI to conquer it. Due to this programming was the first thing that got solved, but pure sciences are next. If what you do, and how you do can be written down on a piece of paper, then AI can do it. I do believe programming getting solved will be double assault on these fields. >>This is not snobbery This is good for the species, what sense does it make to keep treating these fields like they are reserved for the top most intelligent micro percentage of humans? Getting LLM to these things gives some scale to these subjects and thats good.
- alt227 5mo ago> Well that's coming. So is AGI, but we may be hundreds of years off still.
- toilet 5mo agoI have no idea about research in mathematics: How will mathematicians judge what constitutes new conceptual math that is actually useful, vs a hallucination that might be novel but doesn't introduce anything actual meaningful?
- RedCinnabar 5mo agoIt’s basically up to the domain experts. What I found interesting in mathematical optimization/combinatorics (my fields of interest) when an AI proved some major results some time ago was probably dismissed as a boring fact by someone else. What OP is mentioning is just their personal preference and doesn’t reflect the actual opinion of the mathematical world.
- turzmo 5mo agoNot denying that these advances are impressive, but it is important to consider that this is a cherry-picked result. This doesn’t mean that AI can now be expected to do problems of similar or lower difficulty, but that it happened to work well on one problem. What you won’t see is how many others they had to try to get this result.
- cma 5mo agoEarlier of their systems have solve other Erdos problems that people had worked on, this one was more monumental and had had a lot more prior effort that didn't solve, but this isn't a one-off.
- turzmo 5mo agoThis is true, but I still think the relevant question is, how many did they try before they found one that yielded to LLMs? The conclusion is very different if they tried 100 open problems and succeeded at one.
- YeGoblynQueenne 5mo agoYeah, maybe it's just the Texas Sharpshooter Fallacy basically, but with AI. And if it isn't, we should find out very soon. If AI has got so good as OpenAI's post implies, then we should soon see a veritable blooming in the production of mathematical results, by lay people no less. No mathematicians needed! OpenAI say that their secret LLM solved the planar unit distance problem "autonomously" and the companion remarks say it one-shotted it; and while the companion remarks make it clear that there was a lot of refinement and improvement work done by humans, everyone seems to agree that the AI did the job by itself. If that's true, if we're really at that level of autonomous mathematical reasoning ability, then we should see hundreds, even thousands, of open problems suddenly solved in a matter of years if not months. We'll just have to wait and see.
- cma 5mo ago#326 solved https://github.com/teorth/erdosproblems/wiki/AI-contributions-to-Erd%C5%91s-problems#sect-1 https://github.com/teorth/erdosproblems/wiki/AI-contribution...
- Valakas_ 5mo agoAnd the goalposts will keep getting moved all the way to the singularity. And then those people will/would say "Oops. I was wrong."
- InsideOutSanta 5mo agoWhat were you imagining when OpenAI said that their models would have "PhD-Level Intelligence"? Were you imagining that there were specific tasks they could do that were on par with what a human with a PhD could do? Because by that definition, many computer tools have "PhD-Level Intelligence". By that definition, Wolfram Alpha has "PhD-Level Intelligence". What I assumed they were saying is that their LLMs would be as intelligent as a human with a PhD across all, or at least most, knowledge tasks, and they clearly are not.
- xigoi 5mo agoIt would be more impressive if it wasn’t behind the closed doors of a rich company. For all we know, they could’ve paid some mathematicians to work on the problem and pretend that their results are from ChatGPT.