3 ms·

Personally, I haven't moved the goalposts a millimetre in 30 years, and I won't in future. When a computer does maths - not as a tool wielded by a human mathema

Personally, I haven't moved the goalposts a millimetre in 30 years, and I won't in future. When a computer does maths - not as a tool wielded by a human mathematician, but in its own right discovers/invents and proves significant new theorems, advancing some area of research mathematics - I'll take seriously the idea that we've reached AGI. Maths in and of itself doesn't require any physical resources. It's possible that doing maths in practice requires knowledge of the world to extract some kind of product from (I'm skeptical, but it's possible), but in principle a rack mounted server could demonstrate its mathematical ability to the world with nothing more than the ability to send and receive messages. This hasn't been done so far, not because there are obvious missing prerequsites, or because nobody's tried it, or because it has no value, or because there's a prohibitively high barrier to entry for people to have a go. It hasn't been done because nobody knows how to make a machine be a mathematician, and I've seen little evidence of any progress towards it. That's my goalpost, always has been. Reach it and I'll be overjoyed. And FWIW, I strongly believe it can be reached. I don't see the latest round of ML (or any ML, really) as a step towards it, but I'd love to be proven wrong. When I mention this someone always points at some bit of recent research, such as [1], but it's invariably just a new way for a human mathematician to make use of a computer. If anybody knows of any progress, or serious attempts, towards a true AI mathematician I'm very curious to know. [1] https://www.nature.com/articles/s41586-021-04086-x https://www.nature.com/articles/s41586-021-04086-x

adamsmith143 4y ago
You might be out of the loop a bit. https://dspace.mit.edu/handle/1721.1/132379.2 https://dspace.mit.edu/handle/1721.1/132379.2 Is a well known project for an AI Physicist. There are plenty of other groups working on similar projects >I don't see the latest round of ML (or any ML, really) as a step towards it, but I'd love to be proven wrong. LLM models have been able to do basic math for quite a while now and some have been trained to solve differential equations, calculus problems, etc. Well on their way to more impressive capabilities.
- omnicognate 4y ago
  I said "in its own right discovers/invents and proves significant new theorems, advancing some area of research mathematics". Neither of the things you mention are of this nature, or working towards it. "Finding a symbolic expression that matches data from an unknown function" (Feynman) and "solv[ing] differential equations, calculus problems, etc" are not descriptions of what a research mathematician does.
  - spywaregorilla 4y ago
    Feels to me like your test is really "do something on your own right", which is the hard fluffy sentient part, and then some additional guard rails that it needs to be math for some reason
    - mathteddybear 4y ago
      The reason is simple - in math, it means solving open problems.
      - ummonk 4y ago
        That’s a different thing, since solving an existing open problem doesn’t mean inventing a new theorem.
    - omnicognate 4y ago
      The "do it on your own right" is actually the weaker part of it. It's somewhat ill-defined, and I could imagine some future instance where it's highly debatable whether the AI was working on its own or being used as a tool by a human. There aren't yet any cases where that's in question, though, so it's a hypothetical future debate. In any case, it's certainly not the meat of the test. It has to be maths for a specific reason. I think it's in some sense the purest form of an ability distinctive to human minds and pervasive in how they work. As I mentioned, it's an ability that can be demonstrated in the absence of any particular physical capability, and yet despite it being perhaps the oldest goal of AI it may be the one we have made least progress towards. Anyway that's my goalpost, and it's not moving. AGI, being "general", surely should be capable of this hitherto uniquely human activity. If our attempts so far are not capable of it, then clearly they are not "general". If you know of any evidence that my goalpost has been achieved, please let me know. I'm very eager to see it happen.
gpderetta 4y ago
Most humans are not capable of discovering or prove new significant theorems.
- omnicognate 4y ago
  Most humans aren't capable of playing chess to grandmaster level either, or producing artwork in arbitrary styles, or remembering and distinguishing between millions of faces. I'd settle for a demonstration that a computer has truly independently discovered/invented and proved some significant part of our existing mathematical edifice. This hasn't been achieved yet, either. However, I suspect that once we've figured out how to do this at all, surpassing human capabilities will be inevitable in a relatively short time. So I don't see much value in softening the test unless/until there's some actual candidate available that would pass the softer test. The value in requiring genuinely new maths is that it makes it unlikely that knowledge of the result has been encoded in the algorithm or training set. Certainly, if GPT-3 were to output Euler's formula that wouldn't be at all convincing as a "discovery".
  - ummonk 4y ago
    Not an example of complex mathematical reasoning, but aren't AlphaZero and its cousins evidence that ML can independently rediscover principles that humans have found, as well as discover new principles of its own? For example, LC0 plays for advantages that humans hadn't considered before in Chess.
    - gpderetta 4y ago
      Very good point. Apparently human players stared studying AIs playing go and Chess to learn new techniques.
    - omnicognate 4y ago
      I'm not an expert in those algorithms, so... maybe? If so, maybe someone will successfully apply those ideas to the challenge I've described. I'd love to see it happen.

adamsmith143 4y ago

You might be out of the loop a bit. https://dspace.mit.edu/handle/1721.1/132379.2 https://dspace.mit.edu/handle/1721.1/132379.2 Is a well known project for an AI Physicist. There are plenty of other groups working on similar projects >I don't see the latest round of ML (or any ML, really) as a step towards it, but I'd love to be proven wrong. LLM models have been able to do basic math for quite a while now and some have been trained to solve differential equations, calculus problems, etc. Well on their way to more impressive capabilities.

omnicognate 4y ago
I said "in its own right discovers/invents and proves significant new theorems, advancing some area of research mathematics". Neither of the things you mention are of this nature, or working towards it. "Finding a symbolic expression that matches data from an unknown function" (Feynman) and "solv[ing] differential equations, calculus problems, etc" are not descriptions of what a research mathematician does.
- spywaregorilla 4y ago
  Feels to me like your test is really "do something on your own right", which is the hard fluffy sentient part, and then some additional guard rails that it needs to be math for some reason
  - mathteddybear 4y ago
    The reason is simple - in math, it means solving open problems.
    - ummonk 4y ago
      That’s a different thing, since solving an existing open problem doesn’t mean inventing a new theorem.
  - omnicognate 4y ago
    The "do it on your own right" is actually the weaker part of it. It's somewhat ill-defined, and I could imagine some future instance where it's highly debatable whether the AI was working on its own or being used as a tool by a human. There aren't yet any cases where that's in question, though, so it's a hypothetical future debate. In any case, it's certainly not the meat of the test. It has to be maths for a specific reason. I think it's in some sense the purest form of an ability distinctive to human minds and pervasive in how they work. As I mentioned, it's an ability that can be demonstrated in the absence of any particular physical capability, and yet despite it being perhaps the oldest goal of AI it may be the one we have made least progress towards. Anyway that's my goalpost, and it's not moving. AGI, being "general", surely should be capable of this hitherto uniquely human activity. If our attempts so far are not capable of it, then clearly they are not "general". If you know of any evidence that my goalpost has been achieved, please let me know. I'm very eager to see it happen.

gpderetta 4y ago

Most humans are not capable of discovering or prove new significant theorems.

omnicognate 4y ago
Most humans aren't capable of playing chess to grandmaster level either, or producing artwork in arbitrary styles, or remembering and distinguishing between millions of faces. I'd settle for a demonstration that a computer has truly independently discovered/invented and proved some significant part of our existing mathematical edifice. This hasn't been achieved yet, either. However, I suspect that once we've figured out how to do this at all, surpassing human capabilities will be inevitable in a relatively short time. So I don't see much value in softening the test unless/until there's some actual candidate available that would pass the softer test. The value in requiring genuinely new maths is that it makes it unlikely that knowledge of the result has been encoded in the algorithm or training set. Certainly, if GPT-3 were to output Euler's formula that wouldn't be at all convincing as a "discovery".
- ummonk 4y ago
  Not an example of complex mathematical reasoning, but aren't AlphaZero and its cousins evidence that ML can independently rediscover principles that humans have found, as well as discover new principles of its own? For example, LC0 plays for advantages that humans hadn't considered before in Chess.
  - gpderetta 4y ago
    Very good point. Apparently human players stared studying AIs playing go and Chess to learn new techniques.
  - omnicognate 4y ago
    I'm not an expert in those algorithms, so... maybe? If so, maybe someone will successfully apply those ideas to the challenge I've described. I'd love to see it happen.