3 ms·

>Maybe a year - or two model releases - from now, the AI assistant will be undeniably stronger than Tao, and not an equal anymore. we're kind of well past that

>Maybe a year - or two model releases - from now, the AI assistant will be undeniably stronger than Tao, and not an equal anymore. we're kind of well past that (in my opinion), if you consider that this is the same ai assistant that can help you with a recipe, diagnose a weird sound in your car, help with biology homework, translate languages, and so on. even in math alone, i think its indisputably already stronger than Tao, considering it has approximately this much depth in ~all of the math subfields.

eh_why_not 3mo ago
Agreed, it's stronger "horizontally". But I also think that we're not far away from it being stronger vertically; i.e. superior to Tao, in that such turn-by-turn guidance by him in solving sophisticated and difficult problems will not be necessary for long.
- stkdump 3mo ago
  I strongly suspect that the horizontal and the vertical strength are not perpendicular. I suspect having a very wide knowledge gives you access to patterns that reproduce across fields that an expert would lack.
lpd59 3mo ago
To be fair, Tao's specialty is not algebraic geometry.
- buzzin_ 3mo ago
  Interestingly enough, ChatGPT started his answer like this only once: "This is exactly the question I would ask next. My impression is: Most standard invariants are...." And this was a response to this prompt: "Is there a chance of an indirect argument of X ~ A^3 coming from computing some invariant of X that forces it to be A^3? (I am not all that expert in algebraic geometry but I'm thinking like degree or Betti numbers or something.)"
noosphr 3mo ago
It only has that depth because you prompt it towards the experts that understand that depth. An example that happened 10 minutes ago: contracts in racket, it kept arguing that you can't use -> in a contract of a function with a rest argument. I had to mention ... explicitly that it wrote the code correctly.
- larsfaye 3mo ago
  Yeah...I know I should be massively impressed, and I am to a degree, but isn't this what we should expect? LLMs can pick up on patterns that no human can see. In that sense, they really are a type of "search engine", but I use that term loosely. Tao is using them as a way to sift and sort theories and information that the model has a plethora of training data on. It's like being able to converse with the codex of humanity, and extract data via an algorithm that is highly adept at sorting through it.
  - mettamage 3mo ago
    > It's like being able to converse with the codex of humanity, and extract data via an algorithm that is highly adept at sorting through it. Bingo Thanks for laying it out so clearly. This is what I feel LLMs are really good at yea. Almost to the point that I'd say this is what they are. But I'm not quite convinced that it is that, but it seems to point that way
  - cootsnuck 3mo ago
    Also I feel like people are not noticing in Tao's chat that chatGPT fairly often is doing web searches. So it literally is acting as search engine sometimes, and that too is incredibly effective when done right. So in addition to Tao using his prompts to "search/sift through vector space" the LLM is also literally just searching the web to augment its "knowledge" at inference time too. Which, to me, again demystifies these tools. They are incredible tools, but still "just" such.

eh_why_not 3mo ago

Agreed, it's stronger "horizontally". But I also think that we're not far away from it being stronger vertically; i.e. superior to Tao, in that such turn-by-turn guidance by him in solving sophisticated and difficult problems will not be necessary for long.

stkdump 3mo ago
I strongly suspect that the horizontal and the vertical strength are not perpendicular. I suspect having a very wide knowledge gives you access to patterns that reproduce across fields that an expert would lack.

lpd59 3mo ago

To be fair, Tao's specialty is not algebraic geometry.

buzzin_ 3mo ago
Interestingly enough, ChatGPT started his answer like this only once: "This is exactly the question I would ask next. My impression is: Most standard invariants are...." And this was a response to this prompt: "Is there a chance of an indirect argument of X ~ A^3 coming from computing some invariant of X that forces it to be A^3? (I am not all that expert in algebraic geometry but I'm thinking like degree or Betti numbers or something.)"

noosphr 3mo ago

It only has that depth because you prompt it towards the experts that understand that depth. An example that happened 10 minutes ago: contracts in racket, it kept arguing that you can't use -> in a contract of a function with a rest argument. I had to mention ... explicitly that it wrote the code correctly.

larsfaye 3mo ago
Yeah...I know I should be massively impressed, and I am to a degree, but isn't this what we should expect? LLMs can pick up on patterns that no human can see. In that sense, they really are a type of "search engine", but I use that term loosely. Tao is using them as a way to sift and sort theories and information that the model has a plethora of training data on. It's like being able to converse with the codex of humanity, and extract data via an algorithm that is highly adept at sorting through it.
- mettamage 3mo ago
  > It's like being able to converse with the codex of humanity, and extract data via an algorithm that is highly adept at sorting through it. Bingo Thanks for laying it out so clearly. This is what I feel LLMs are really good at yea. Almost to the point that I'd say this is what they are. But I'm not quite convinced that it is that, but it seems to point that way
- cootsnuck 3mo ago
  Also I feel like people are not noticing in Tao's chat that chatGPT fairly often is doing web searches. So it literally is acting as search engine sometimes, and that too is incredibly effective when done right. So in addition to Tao using his prompts to "search/sift through vector space" the LLM is also literally just searching the web to augment its "knowledge" at inference time too. Which, to me, again demystifies these tools. They are incredible tools, but still "just" such.