7 ms·

Well math solving is exactly what the rumored Q* is aiming towards too. I don't think it'll take more than 2 years before some LLM + RL system can take the gold

Well math solving is exactly what the rumored Q* is aiming towards too. I don't think it'll take more than 2 years before some LLM + RL system can take the gold medal. I think companies like OpenAI are aiming for something far more ambitious, like solving a millennium prize problem (even with human assistance). That's the kind of news release that'll add another $100 billion to your market cap.

dimask 3y ago
Their current ambition is to be able to solve school math, which is quite far away from solving unsolved conjectures or math olympiads. I really doubt that any of this is within LLM/transformer scope, except maybe in some auxiliary sense to other, much different architectures.
- anonylizard 3y ago
  Art isn't an easier problem than math. An artbot would have sounded more sci-fi than a mathbot only 2 years ago. Yet it only took the AI world 1.5 years to go from drawing child scribbles to replicating top artists with like 90% similarity (I can barely tell the difference between AI and human drawn art anymore with the new NovelAI model). It won't be long before AI starts to go superhuman in art skills. It won't take long from a school-math model to math olympiad model (I'd say 1 year is enough), and going to unsolved conjectures won't be that long either (2-3 years?). We know from AlphaGo that its possible to make AI systems far superhuman at solving some abstract math problem.
  - RandomLensman 3y ago
    AlphaGo solved Go?
    - anonylizard 3y ago
      "Superhuman at solving" != "solved". AlphaGo didn't solve go (Ie, can the first mover guarantee a win?). However, it understood go at a far, far superior level to any human. A mathbot doens't have to solve math in general. It merely has to be better at solving math than any human mathematician to be considered ASI. And it only has to be better than the 'average' human mathematician to be extremely useful in accelerating math research.
      - RandomLensman 3y ago
        What mathematics was AlphaGo solving? What do you mean by solving there?
        dimask 3y ago
        Proving that the first player has a winning strategy, or the optimal strategy for both players leads to draw.
        falcor84 3y ago
        Just to nitpick, while unlikely, it is theoretically possible that the second player has a winning strategy
        k2enemy 3y ago
        To nitpick a little further, it actually is not possible that the second player has a winning strategy. For that to be the case, P2 would need a winning path of play no matter what P1's first move is. Suppose that P1 passes on their first move (which is a valid move). Then P2 has a winning path of play in which they put down the first stone. But P1 could have made that move and then they would be on the winning path.
        falcor84 3y ago
        Good point, I actually wasn't that they could pass the first move
        godelski 3y ago
        I'm not a game theorist, in RL, nor a big Go player; but I am having a hard time finding this argument convincing. Isn't the whole reason Go is impressive is because the enormous set of possible moves? Like we know that no computer could run ever game in the lifetime of the universe were it to even perform millions of moves a second. So of the I understand this number to be north of 10^500 for possible legal and playable games. So the difference of 1 doesn't seem meaningful. Is there something I'm missing or a more convincing argument? Because even if player 1 always locks out 90% of those possible future moves, that's still an absurdly large search space and it doesn't seem like it is meaningfully different.
  - ekianjo 3y ago
    > Art isn't an easier problem than math Not sure. Art is about approximate pattern recognition and if you have a large enough dataset it seems that you can definitely reproduce some of that. For math... it involves consistent reasoning from A to Z - which does not allow for any kind of mistake in the way. In Art you won't feel too bad if the shadows or the lights are a little weird or if a character has 7 fingers instead of 5 on one hand, but this kind of mishaps break everything in Math.
    - anonylizard 3y ago
      Fingers are already mostly solved. The latest models draw hands better than most human artists. I think artists would disagree with your assessment of 'approximate pattern recognition'. Its more like： 1. Given a set of words describing what the user wants. 2. Arrange pixels in a grid 3. That maximizes the user rating On one hand it is tolerant of small errors. On the other hand its an extremely broad problem. Also to get a good user rating, it has to do 99 things right for every 1 thing it draws wrong.
      - larodi 3y ago
        Well this thing about fingers, etc in drawings. Lets put it this way - for us mere mortals the generative images look very much okay. To artists and people who actually draw something, well ... they very often spot inconsistencies in the whole production, including how fingers, arms, overall body posture, etc is presented. So it is exactly what we can expect - good enough on average, but actually a mediocre result of commonality. Thing is the wide audience chews in mediocrity all the time, and nobody seems to have been able to change this for ages...
        disgruntledphd2 3y ago
        It's a weighted average over the prompts and data. Mediocrity is exactly what we'd expect from such an approach. Now, if we end up seeing mastery, that would be extremely interesting.
        3y ago
  - japoco 3y ago
    I think art is much easier for LLM-style AI models to do compared to writing. To make a nice picture you just need to place pixels near each other in a way that looks good, and we all know LLMs are phenomenal at this. Good text on the other hand is not just text that has a good flow and fits the prompt. It must follow a line of thought, and LLMs don’t do that by design, even though we could argue wether they have that capability as an emergent one, but I don't believe that at all.
  - jszymborski 3y ago
    > Art isn't an easier problem than math. Someone can tell you when a math problem is solved. Someone else can't tell you when you've successfully art'd with remotely the same degree of confidence. In so far as they can, however, many experts claim that AI cannot, indeed "do an art". Also, it's easier to formulate a hard math question (there are plenty of unsolved problems), but that's (IMHO) harder to do for art. Sure, you may think this is the first time the phrase "Astronaut riding a Llama and holding an avocado" was writ, but those are all well represented concepts in the dataset. For more abstract prompts, there really isn't a way to verify "correctness".
  - jack_riminton 3y ago
    NovelAI isn't for drawing it's for writing, did you mean something else?
  - dxbydt 3y ago
    > Art isn't an easier problem than math. If I was asked to draw a. the emptiness in your heart b. the lack of furniture in your room c. your empty bank balance d. starvation e. object returned by a python function with no return ... I could just submit an empty sheet of paper, & an artist would argue that my empty sheet of paper represents any/all of the above. Now, if I turn in the same empty paper at a math qualifier and argue that it represents the infinite set of real and complex numbers, ergo the answer to the posed qual problem must be in there, I'll get kicked out of that phd program in a jiffy.
    - civilitty 3y ago
      > I could just submit an empty sheet of paper, & an artist would argue that my empty sheet of paper represents any/all of the above. And they'd be taken about as seriously as the ads taped to a urinal in a Museum of Modern Art washroom.
  - godelski 3y ago
    > Art isn't an easier problem than math. In a way it is, in a way it isn't. You have to remember what is easy for machine isn't going to correlate to what is easy for us humans. Look at AI art. Closely. No, closer than that. All the detail is fucked up. Not just the hands, but the tiniest of things. Strokes, lighting, reflections, and consistency, and all that. But can I turn my friend into a convincing werewolf? Yes. Can I turn my cat into a human or Wonder Woman? No. The system isn't a "fancy copier" but it is a compression algorithm and the aforementioned tasks were only possible because lots of work training LoRAs, textual inversions, control nets, and so on (you could seriously improve GANs, VAEs, hell, even Boltzman Machines could probably do pretty well were any of these given the same research investment that diffusion has received. GANs come close but nuances like GANs having a magnitude fewer parameters). But let's look at math, can I consistently add numbers? No. The problem is that in math, all those tiny intricate details matter. Not only that, they matter at every single step. The thing here is that these are still pattern recognition machines. But they aren't generalized machines. You can't really derive out all of math from probability distributions (or at least cleanly, but still not convinced you can). The thing is that for math to work in AI we have to address the elephants in the room: math. Yeah, math. ML people don't like it. But we gotta address the axioms in the room that we're operating under. How do we move on from machines operating on manifolds? How do we make it so data are not distributional? How do we move away from a number of unmentioned axioms remains a large open problem in AI research. One that does not get anywhere serious enough of a conversation, especially within the community. Sure, maybe transformer circuits can learn some addition by learning how to do FFTs and add in the FFT space, but you're not going to get to Abstract Algebra that way. Ideally the AI can solve problems that have no algorithms, pun intended.
- bilater 3y ago
  Exponential progress. Hard to wrap your mind around. An analogy: If it takes 20 years to create AGI equivalent to the village idiot. It take another couple of hours to go from that to Einstein.
  - dimask 3y ago
    I don't disagree, as in that if we achieve school math in a way that is not mere overfitting over a language-based training set, going to more advanced mathematics is definitely conceivable. My problem is that people are talking about solving unsolved conjectures while we are not even in a point where we know how to tackle math at all. Imo we are not currently in the beginning of an exponential curve re solving math with AI, and def not on the path of AGI. I understand that if one believes that we are on the path to AGI soon then we shall have these math-AI advancements quite soon, but I disagree with the premise.
antoinexp 3y ago
Might depend on the terms you have in mind, but current consensus seems to be more like 4-5 years as we speak on https://www.metaculus.com/questions/6728/ai-wins-imo-gold-medal/ https://www.metaculus.com/questions/6728/ai-wins-imo-gold-me...
- auntienomen 3y ago
  That's 4-5 years for solving Olympiad problems. Those are just very tricky high school math problems. They have solutions and can generally be solved by applying some combination of standard tricks. It's very much the sort of thing an LLM should be good at. Solving Millennium problems is a whole different ballgame. It's not known if these problems are solvable within ZFC axioms. (In one case, the Yang-Mills prize, stating the problem mathematically is part of the challenge.) All of the obvious applications of known tricks have been tried and failed. To solve such problems, one probably has to invent new and surprising mathematical definitions, building a framework in which the problem becomes solvable. This is something that LLMs will be crap at; the process of invention is not represented in any training data we have access to.
  - EVa5I7bHFq9mnYK 3y ago
    We can look at it this way: there are ~1000 chess Grandmasters and one World Champion. It took very short time for AI to go from beating an average GM to beating World Champion. There are ~1000 MO winners and 1 (one) Millenial problem solver ...
    - auntienomen 3y ago
      We can. But doing so frames math research as the same sort of activity as math problem solving.. it's not. Many imo champions struggle to do any successful math research. And many successful math researchers (e.g., all of the most recent batch of Fields medallists) never did the Oympiad at all.
      - EVa5I7bHFq9mnYK 3y ago
eddtries 3y ago
I have a couple friends who did the Math tripos at Cambridge (so a pretty high level!) who work in tech and have unanimously said they have 0% expectations of an LLM doing a millennium problem anytime soon
- Enginerrrd 3y ago
  Yeah, millennium problems almost certainly require truly novel nontrivial ideas to solve. That's a tough thing for AI to do. On the other hand, Terrence Tao had an interesting article on his blog a while back where he was trying to solve a problem and asked chatGPT about it in a high-level strategy sense. ChatGPT suggested several reasonable approaches, one of which turned out to work. That's nowhere near solving a millennium problem, but it is very interesting and suggests fairly sophisticated conceptual understanding of mathematics nevertheless. Current architecture and training methods I don't think are enough to get there. However, with enough compute, I can plausibly envision some sort of meta training of LLMs using an analogy to GANs where one network tries to synthesize new correct ideas and the other shoots them down as not novel, not correct, or not sufficiently interesting. Such an approach I think could perhaps work, but the compute needed would probably be pretty high.
  - jacobr1 3y ago
    And also if you could combine that with some kind of representation software like Lean that can validate proofs, perhaps you can generate some kind of targeted search of the problem space and with a combination of a general high level strategy and maybe brute force of some sub-problems, find novel solutions. My understanding is that is the common human approach: gain some kind of intuition of problem, try a few things and then iteratively refine. Sometimes that works, sometimes you need to find a new starting point. It seems plausible we could automate that workflow, with likely mixed but still useful results.
  - eddtries 3y ago
    I’m not trying to understate the stuff you can do with LLMs or formal proof tooling that could be attached to randomly try things till it reaches a solution to a novel problem, but some solutions you see to lesser problems are sometimes so pie-in-the-sky I think stumbling on one is barely better than a random walk. And I think mathematicians like Tao are far better at narrowing that down. As an aide I can see it’s use, I just don’t believe we’re going to have LLMs & tooling solve these grand problems until compute power is orders of magnitudes better, and even then I’m still sceptical. BUT I’m not a mathematician and this is based on my intuition from pub talks :)

dimask 3y ago

Their current ambition is to be able to solve school math, which is quite far away from solving unsolved conjectures or math olympiads. I really doubt that any of this is within LLM/transformer scope, except maybe in some auxiliary sense to other, much different architectures.

anonylizard 3y ago
Art isn't an easier problem than math. An artbot would have sounded more sci-fi than a mathbot only 2 years ago. Yet it only took the AI world 1.5 years to go from drawing child scribbles to replicating top artists with like 90% similarity (I can barely tell the difference between AI and human drawn art anymore with the new NovelAI model). It won't be long before AI starts to go superhuman in art skills. It won't take long from a school-math model to math olympiad model (I'd say 1 year is enough), and going to unsolved conjectures won't be that long either (2-3 years?). We know from AlphaGo that its possible to make AI systems far superhuman at solving some abstract math problem.
- RandomLensman 3y ago
  AlphaGo solved Go?
  - anonylizard 3y ago
    "Superhuman at solving" != "solved". AlphaGo didn't solve go (Ie, can the first mover guarantee a win?). However, it understood go at a far, far superior level to any human. A mathbot doens't have to solve math in general. It merely has to be better at solving math than any human mathematician to be considered ASI. And it only has to be better than the 'average' human mathematician to be extremely useful in accelerating math research.
    - RandomLensman 3y ago
      What mathematics was AlphaGo solving? What do you mean by solving there?
      - dimask 3y ago
        Proving that the first player has a winning strategy, or the optimal strategy for both players leads to draw.
        falcor84 3y ago
        Just to nitpick, while unlikely, it is theoretically possible that the second player has a winning strategy
        k2enemy 3y ago
        To nitpick a little further, it actually is not possible that the second player has a winning strategy. For that to be the case, P2 would need a winning path of play no matter what P1's first move is. Suppose that P1 passes on their first move (which is a valid move). Then P2 has a winning path of play in which they put down the first stone. But P1 could have made that move and then they would be on the winning path.
        falcor84 3y ago
        Good point, I actually wasn't that they could pass the first move
        godelski 3y ago
        I'm not a game theorist, in RL, nor a big Go player; but I am having a hard time finding this argument convincing. Isn't the whole reason Go is impressive is because the enormous set of possible moves? Like we know that no computer could run ever game in the lifetime of the universe were it to even perform millions of moves a second. So of the I understand this number to be north of 10^500 for possible legal and playable games. So the difference of 1 doesn't seem meaningful. Is there something I'm missing or a more convincing argument? Because even if player 1 always locks out 90% of those possible future moves, that's still an absurdly large search space and it doesn't seem like it is meaningfully different.
- ekianjo 3y ago
  > Art isn't an easier problem than math Not sure. Art is about approximate pattern recognition and if you have a large enough dataset it seems that you can definitely reproduce some of that. For math... it involves consistent reasoning from A to Z - which does not allow for any kind of mistake in the way. In Art you won't feel too bad if the shadows or the lights are a little weird or if a character has 7 fingers instead of 5 on one hand, but this kind of mishaps break everything in Math.
  - anonylizard 3y ago
    Fingers are already mostly solved. The latest models draw hands better than most human artists. I think artists would disagree with your assessment of 'approximate pattern recognition'. Its more like： 1. Given a set of words describing what the user wants. 2. Arrange pixels in a grid 3. That maximizes the user rating On one hand it is tolerant of small errors. On the other hand its an extremely broad problem. Also to get a good user rating, it has to do 99 things right for every 1 thing it draws wrong.
    - larodi 3y ago
      Well this thing about fingers, etc in drawings. Lets put it this way - for us mere mortals the generative images look very much okay. To artists and people who actually draw something, well ... they very often spot inconsistencies in the whole production, including how fingers, arms, overall body posture, etc is presented. So it is exactly what we can expect - good enough on average, but actually a mediocre result of commonality. Thing is the wide audience chews in mediocrity all the time, and nobody seems to have been able to change this for ages...
      - disgruntledphd2 3y ago
        It's a weighted average over the prompts and data. Mediocrity is exactly what we'd expect from such an approach. Now, if we end up seeing mastery, that would be extremely interesting.
        3y ago
- japoco 3y ago
  I think art is much easier for LLM-style AI models to do compared to writing. To make a nice picture you just need to place pixels near each other in a way that looks good, and we all know LLMs are phenomenal at this. Good text on the other hand is not just text that has a good flow and fits the prompt. It must follow a line of thought, and LLMs don’t do that by design, even though we could argue wether they have that capability as an emergent one, but I don't believe that at all.
- jszymborski 3y ago
  > Art isn't an easier problem than math. Someone can tell you when a math problem is solved. Someone else can't tell you when you've successfully art'd with remotely the same degree of confidence. In so far as they can, however, many experts claim that AI cannot, indeed "do an art". Also, it's easier to formulate a hard math question (there are plenty of unsolved problems), but that's (IMHO) harder to do for art. Sure, you may think this is the first time the phrase "Astronaut riding a Llama and holding an avocado" was writ, but those are all well represented concepts in the dataset. For more abstract prompts, there really isn't a way to verify "correctness".
- jack_riminton 3y ago
  NovelAI isn't for drawing it's for writing, did you mean something else?
- dxbydt 3y ago
  > Art isn't an easier problem than math. If I was asked to draw a. the emptiness in your heart b. the lack of furniture in your room c. your empty bank balance d. starvation e. object returned by a python function with no return ... I could just submit an empty sheet of paper, & an artist would argue that my empty sheet of paper represents any/all of the above. Now, if I turn in the same empty paper at a math qualifier and argue that it represents the infinite set of real and complex numbers, ergo the answer to the posed qual problem must be in there, I'll get kicked out of that phd program in a jiffy.
  - civilitty 3y ago
    > I could just submit an empty sheet of paper, & an artist would argue that my empty sheet of paper represents any/all of the above. And they'd be taken about as seriously as the ads taped to a urinal in a Museum of Modern Art washroom.
- godelski 3y ago
  > Art isn't an easier problem than math. In a way it is, in a way it isn't. You have to remember what is easy for machine isn't going to correlate to what is easy for us humans. Look at AI art. Closely. No, closer than that. All the detail is fucked up. Not just the hands, but the tiniest of things. Strokes, lighting, reflections, and consistency, and all that. But can I turn my friend into a convincing werewolf? Yes. Can I turn my cat into a human or Wonder Woman? No. The system isn't a "fancy copier" but it is a compression algorithm and the aforementioned tasks were only possible because lots of work training LoRAs, textual inversions, control nets, and so on (you could seriously improve GANs, VAEs, hell, even Boltzman Machines could probably do pretty well were any of these given the same research investment that diffusion has received. GANs come close but nuances like GANs having a magnitude fewer parameters). But let's look at math, can I consistently add numbers? No. The problem is that in math, all those tiny intricate details matter. Not only that, they matter at every single step. The thing here is that these are still pattern recognition machines. But they aren't generalized machines. You can't really derive out all of math from probability distributions (or at least cleanly, but still not convinced you can). The thing is that for math to work in AI we have to address the elephants in the room: math. Yeah, math. ML people don't like it. But we gotta address the axioms in the room that we're operating under. How do we move on from machines operating on manifolds? How do we make it so data are not distributional? How do we move away from a number of unmentioned axioms remains a large open problem in AI research. One that does not get anywhere serious enough of a conversation, especially within the community. Sure, maybe transformer circuits can learn some addition by learning how to do FFTs and add in the FFT space, but you're not going to get to Abstract Algebra that way. Ideally the AI can solve problems that have no algorithms, pun intended.
bilater 3y ago
Exponential progress. Hard to wrap your mind around. An analogy: If it takes 20 years to create AGI equivalent to the village idiot. It take another couple of hours to go from that to Einstein.
- dimask 3y ago
  I don't disagree, as in that if we achieve school math in a way that is not mere overfitting over a language-based training set, going to more advanced mathematics is definitely conceivable. My problem is that people are talking about solving unsolved conjectures while we are not even in a point where we know how to tackle math at all. Imo we are not currently in the beginning of an exponential curve re solving math with AI, and def not on the path of AGI. I understand that if one believes that we are on the path to AGI soon then we shall have these math-AI advancements quite soon, but I disagree with the premise.

antoinexp 3y ago

Might depend on the terms you have in mind, but current consensus seems to be more like 4-5 years as we speak on https://www.metaculus.com/questions/6728/ai-wins-imo-gold-medal/ https://www.metaculus.com/questions/6728/ai-wins-imo-gold-me...

auntienomen 3y ago
That's 4-5 years for solving Olympiad problems. Those are just very tricky high school math problems. They have solutions and can generally be solved by applying some combination of standard tricks. It's very much the sort of thing an LLM should be good at. Solving Millennium problems is a whole different ballgame. It's not known if these problems are solvable within ZFC axioms. (In one case, the Yang-Mills prize, stating the problem mathematically is part of the challenge.) All of the obvious applications of known tricks have been tried and failed. To solve such problems, one probably has to invent new and surprising mathematical definitions, building a framework in which the problem becomes solvable. This is something that LLMs will be crap at; the process of invention is not represented in any training data we have access to.
- EVa5I7bHFq9mnYK 3y ago
  We can look at it this way: there are ~1000 chess Grandmasters and one World Champion. It took very short time for AI to go from beating an average GM to beating World Champion. There are ~1000 MO winners and 1 (one) Millenial problem solver ...
  - auntienomen 3y ago
    We can. But doing so frames math research as the same sort of activity as math problem solving.. it's not. Many imo champions struggle to do any successful math research. And many successful math researchers (e.g., all of the most recent batch of Fields medallists) never did the Oympiad at all.
    - EVa5I7bHFq9mnYK 3y ago

eddtries 3y ago

I have a couple friends who did the Math tripos at Cambridge (so a pretty high level!) who work in tech and have unanimously said they have 0% expectations of an LLM doing a millennium problem anytime soon

Enginerrrd 3y ago
Yeah, millennium problems almost certainly require truly novel nontrivial ideas to solve. That's a tough thing for AI to do. On the other hand, Terrence Tao had an interesting article on his blog a while back where he was trying to solve a problem and asked chatGPT about it in a high-level strategy sense. ChatGPT suggested several reasonable approaches, one of which turned out to work. That's nowhere near solving a millennium problem, but it is very interesting and suggests fairly sophisticated conceptual understanding of mathematics nevertheless. Current architecture and training methods I don't think are enough to get there. However, with enough compute, I can plausibly envision some sort of meta training of LLMs using an analogy to GANs where one network tries to synthesize new correct ideas and the other shoots them down as not novel, not correct, or not sufficiently interesting. Such an approach I think could perhaps work, but the compute needed would probably be pretty high.
- jacobr1 3y ago
  And also if you could combine that with some kind of representation software like Lean that can validate proofs, perhaps you can generate some kind of targeted search of the problem space and with a combination of a general high level strategy and maybe brute force of some sub-problems, find novel solutions. My understanding is that is the common human approach: gain some kind of intuition of problem, try a few things and then iteratively refine. Sometimes that works, sometimes you need to find a new starting point. It seems plausible we could automate that workflow, with likely mixed but still useful results.
- eddtries 3y ago
  I’m not trying to understate the stuff you can do with LLMs or formal proof tooling that could be attached to randomly try things till it reaches a solution to a novel problem, but some solutions you see to lesser problems are sometimes so pie-in-the-sky I think stumbling on one is barely better than a random walk. And I think mathematicians like Tao are far better at narrowing that down. As an aide I can see it’s use, I just don’t believe we’re going to have LLMs & tooling solve these grand problems until compute power is orders of magnitudes better, and even then I’m still sceptical. BUT I’m not a mathematician and this is based on my intuition from pub talks :)