4 ms·

Do AIs produce answers whose work is incomprehensible to humans? It seems like you could just have the AI elaborate multiple times until you were satisfied with

Do AIs produce answers whose work is incomprehensible to humans? It seems like you could just have the AI elaborate multiple times until you were satisfied with the explanation and documentation of what went into figuring out the answer. It’s not like the AI is one shotting the answer in a single opaque query anyways.

datsci_est_2015 5mo ago
Like other commenters, I think you’re also underestimating the complexity of esoteric higher level math. Consider the “Magnus Carlsen” of mathematics, who is more capable of understanding mathematics than any other human. But then also realize that that individual has probably devoted their entire career into a specific subdomain of mathematics. Within other deep recesses of mathematics, this Magnus equivalent will be less capable than their peers without years of rewiring their brain to understand the esoteric concepts and properties within that other subdomain. LLMs will be able to dig deeper and broader than any human mathematician, and find results that are completely useless to humans because it would take more than an entire lifetime to “speak the language” of the concepts the LLMs have produced. The only way those results can become useful to humans is if then the LLM itself finds a way for it to be practical to humans once again. So, no, I don’t think this represents the “democratization” of mathematics where mathematicians are no longer necessary because anyone can just prompt the LLM to explain it. The bar for entry level mathematics is lower, for sure, but research level mathematics will continue to be unapproachable for anyone who hasn’t devoted their career to it.
- seanmcdirmid 5mo ago
  I don't get it. LLMs don't have ego, they don't have the ability to say "no, this should be obvious, I'm not going to explain further", they are just token predictors, and given context, they can generate more tokens. If you don't understand how the answer was derived? You just ask more questions and it isn't going to get bored or annoyed, it will just try to answer the questions. Is that what is offending you so much?
  - datsci_est_2015 5mo ago
    No, it doesn’t sound like you get it. It has nothing to do with the properties of LLMs and everything to do with the complexity of mathematics. Have you ever been exposed to concepts that are so complex that you feel like you could devote your entire lifetime to trying to understand it and still fall short? It’s a very humbling experience, especially if you have classmates who pick it up effortlessly. Without a human holding the reins, consider an LLM a rudderless superboat speeding erratically towards the horizon, finding and proving meaningless theorems that not even your most talented classmate could ever begin to understand. My point is the human is a critical piece to the puzzle, but not just any human, a career mathematician.
    - bawolff 5mo ago
      > Have you ever been exposed to concepts that are so complex that you feel like you could devote your entire lifetime to trying to understand it and still fall short? It’s a very humbling experience, especially if you have classmates who pick it up effortlessly. > Without a human holding the reins, consider an LLM a rudderless superboat speeding erratically towards the horizon, finding and proving meaningless theorems that not even your most talented classmate could ever begin to understand. This feels like a little bit of a jump to me. AIs arent actually alive so of course someone is going to have to pose the question. They arent going to just do stuff on their own. And of course mathmaticians are going to need to interpret the results if we are to glean anything beyong if the conjecture is true or false. But you seem to be suggesting that mathematicians will have to micromanage every step. That seems like a bit of a jump which i dont see much evidence for.
      - fc417fc802 5mo ago
        Micromanagement wasn't the message I took from that. Rather the level of human involvement required which (it seems like) the two of you more or less agree on. The meaning I took was how far it's possible to travel from the shore - ie the scope of the state space. The mathematics we're exposed to is all quite shallow compared to what will (presumably) be possible between digital formalization and massive ML models. But the latter probably can't ever be understood by regular biological humans.
- storus 5mo ago
  > “speak the language” of the concepts the LLMs have produced LLMs don't produce concepts, they just predict next tokens; they can't invent new concepts, only synthesize what is in their probability distribution already. They can mix/fuse vast areas of math together that are inaccessible to individual mathematicians, but can't create new concepts not present in their probability distribution.
epgui 5mo ago
It’s easy to imagine this being a problem both in quality and in volume. Verifiable work is less valuable than verified work. And noise is always costly.

datsci_est_2015 5mo ago

Like other commenters, I think you’re also underestimating the complexity of esoteric higher level math. Consider the “Magnus Carlsen” of mathematics, who is more capable of understanding mathematics than any other human. But then also realize that that individual has probably devoted their entire career into a specific subdomain of mathematics. Within other deep recesses of mathematics, this Magnus equivalent will be less capable than their peers without years of rewiring their brain to understand the esoteric concepts and properties within that other subdomain. LLMs will be able to dig deeper and broader than any human mathematician, and find results that are completely useless to humans because it would take more than an entire lifetime to “speak the language” of the concepts the LLMs have produced. The only way those results can become useful to humans is if then the LLM itself finds a way for it to be practical to humans once again. So, no, I don’t think this represents the “democratization” of mathematics where mathematicians are no longer necessary because anyone can just prompt the LLM to explain it. The bar for entry level mathematics is lower, for sure, but research level mathematics will continue to be unapproachable for anyone who hasn’t devoted their career to it.

seanmcdirmid 5mo ago
I don't get it. LLMs don't have ego, they don't have the ability to say "no, this should be obvious, I'm not going to explain further", they are just token predictors, and given context, they can generate more tokens. If you don't understand how the answer was derived? You just ask more questions and it isn't going to get bored or annoyed, it will just try to answer the questions. Is that what is offending you so much?
- datsci_est_2015 5mo ago
  No, it doesn’t sound like you get it. It has nothing to do with the properties of LLMs and everything to do with the complexity of mathematics. Have you ever been exposed to concepts that are so complex that you feel like you could devote your entire lifetime to trying to understand it and still fall short? It’s a very humbling experience, especially if you have classmates who pick it up effortlessly. Without a human holding the reins, consider an LLM a rudderless superboat speeding erratically towards the horizon, finding and proving meaningless theorems that not even your most talented classmate could ever begin to understand. My point is the human is a critical piece to the puzzle, but not just any human, a career mathematician.
  - bawolff 5mo ago
    > Have you ever been exposed to concepts that are so complex that you feel like you could devote your entire lifetime to trying to understand it and still fall short? It’s a very humbling experience, especially if you have classmates who pick it up effortlessly. > Without a human holding the reins, consider an LLM a rudderless superboat speeding erratically towards the horizon, finding and proving meaningless theorems that not even your most talented classmate could ever begin to understand. This feels like a little bit of a jump to me. AIs arent actually alive so of course someone is going to have to pose the question. They arent going to just do stuff on their own. And of course mathmaticians are going to need to interpret the results if we are to glean anything beyong if the conjecture is true or false. But you seem to be suggesting that mathematicians will have to micromanage every step. That seems like a bit of a jump which i dont see much evidence for.
    - fc417fc802 5mo ago
      Micromanagement wasn't the message I took from that. Rather the level of human involvement required which (it seems like) the two of you more or less agree on. The meaning I took was how far it's possible to travel from the shore - ie the scope of the state space. The mathematics we're exposed to is all quite shallow compared to what will (presumably) be possible between digital formalization and massive ML models. But the latter probably can't ever be understood by regular biological humans.
storus 5mo ago
> “speak the language” of the concepts the LLMs have produced LLMs don't produce concepts, they just predict next tokens; they can't invent new concepts, only synthesize what is in their probability distribution already. They can mix/fuse vast areas of math together that are inaccessible to individual mathematicians, but can't create new concepts not present in their probability distribution.

epgui 5mo ago

It’s easy to imagine this being a problem both in quality and in volume. Verifiable work is less valuable than verified work. And noise is always costly.