5 ms·
I’m generally a bit of an LLM skeptic, at least as concerns extravagant claims around “AGI” or self-improvement of any of that nonsense. But there are activiti
by benreesman 2y ago
I’m generally a bit of an LLM skeptic, at least as concerns extravagant claims around “AGI” or self-improvement of any of that nonsense.
But there are activities where an inherently high-temperature assistant (e.g. some Instruct-inspired tune) are a great fit, and those are almost definitionally at the boundary of “objectively faithful” and “a bit stochastic”.
I’ve never done any work in novel mathematics myself, but I’m a big fan of those who do, and any study of those who do paints a picture of roughly that boundary: those who are proficient can easily spot a falsehood but only with difficulty generate inspiration for a new idea.
Of all the highly optimistic applications for 2024 LLMs, mathematics sounds pretty plausible?
- kevinventullo 2y agoFWIW as a former mathematician, I believe that if LLM’s can crack mathematics in the sense of say, solving the Riemann Hypothesis through chains of logical reasoning, then we’d be in the back half of reaching “AGI” in the sense of solving broad classes of general problems like drug discovery, or getting fusion to work, or even improving itself.
- benreesman 2y agoOh sure, I think that’s pretty uncontroversial. An LLM that can with any consistency start at “prove or disprove P = NP” and arrive at a proof is in some sense super-intelligence. Even the best funded boosters are making no such claims AFAIK, I think a much more reasonable and sober assertion might be like “they are useful to doing real mathematics”.
- adroniser 2y agoThe curious thing is if you can ever hit a snowball type point, because by proving theorems you are generating training data, and maybe you can get to a point where you effectively have limitless data.
- quonn 2y agoScience needs experiments and those will not go faster because of AI. So why would fusion arrive any faster?
- kevinventullo 2y agoYeah I did say “back half”. I just think that if we can build an AI that can logically reason better than any human, combined with the total knowledge of all humans, it can start running experiments and doing “science” itself.
- benreesman 2y agoIn general I think I agree with all your points. The word “if” in both the parent and GP is doing a lot of lifting here though, if they can do amazing feat X, then they will likewise do amazing feat Y. And I think in every instance you’ve made a credible argument. But I don’t think we have the if yet. If I could just slip GPT-4 an executive summary of the code I meant to write and the emails I meant to send on a given day I would be doing it. But it can’t: it’s catastrophically wrong with extreme confidence routinely. These things are easy to cherry-pick in citation but still fuck up on HellaSwag in instances a child would not.
- kevinventullo 2y agoOh, I 100% agree we are not there. Every time a new model comes out I cook up a few undergrad math homework questions to see if they’ve gotten any better, and so far they haven’t. But, the thing I like about “reducing” AGI to solving math is that it kind of sidesteps these thorny questions about interacting with the real world to conduct experiments, etc. It’s something that, in principle, can be done with the current text interfaces, and it’s pretty easy to tell whether or not we’ve gotten there.
- aleph_minus_one 2y ago> Science needs experiments and those will not go faster because of AI. Even though I am deeply skeptical whether AIs (as they are commonly understood today) will be helpful for hard scientific problems (for quite different reasons), I don't think this argument necessarily holds: couldn't a (hypothetical!) AI propose much better experiments and experimental designs?
- 77pt77 2y agoMost likely it will just generate bullshit proofs, claim otherwise and then just flat out lie to your face.
- mindcrime 2y agoI’m generally a bit of an LLM skeptic, at least as concerns extravagant claims around “AGI” or self-improvement of any of that nonsense. Of all the highly optimistic applications for 2024 LLMs, mathematics sounds pretty plausible? You probably already know this, but I think it bears pointing out explicitly: in this context we're not talking (only) about LLM's. There's a LOT more to AI than just Large Language Models, and the list of resources linked reflects that.
- benreesman 2y agoI’m an advocate for shifting resources from pure-scale transformer work to other promising avenues in both representation and reinforcement learning.
- pineaux 2y agoNever going to happen until upscaling does not return easy gains.
- kubrickslair 2y agoThe impact extends beyond mathematics. We have seen some pretty positive outcomes for LLMs as an assistant for scientists - with a lot of custom agent dev though - and not off-the-shelf models. This is already being used by a few hundred scientists in a lab. We are aiming to extend to thousands next year with a focus on CS, climate & bio.
- benreesman 2y agoI think this readership would be highly interested by anything hundreds of serious scientists are doing in serious work. Can you share any details?
- kubrickslair 2y agoThanks! Most of the current work is not yet out as press or blogs - we generally do that post code release/ publications/ arXiv submissions. Will share them as they are made public. Meanwhile, this is the blog of our earliest work from January that will be published in ICML24: https://blog.allenai.org/data-driven-discovery-with-large-generative-models-e1a062e99390 https://blog.allenai.org/data-driven-discovery-with-large-ge... Also if you are interested - happy to correspond more by email. Lot more to share privately.
- benreesman 2y agohttps://arxiv.org/pdf/2402.00159 https://arxiv.org/pdf/2402.00159 MIT, the Steel Factory, Cal, UDub and what, 37 authors pushed a preprint and no one heard about it? @dang I can’t be the only one who has had it on this GPT-5 test flight shit. I don’t care how many clean Azure IPv4 blocks Altman has, throw this motherfucker out.
- kubrickslair 2y agoHaha, I think it was covered at HN. But did not get as much attention as it should have.