4 ms·

Is the reason it can show steps for solving an integral because the training set contained webpages or books showing how to do it?

simianwords 8mo ago
if we have steps for understanding any author's english and creative process (generally not specific to an author) would you agree then it is possible for an llm to do it?
- mysterydip 8mo ago
  Repeating a process, yes for sure, even (pseudorandom?) variations on a process. Understanding a process is a different question, and I’m not sure how you would measure that. In school we would have a test with various questions to show you understand the concept of addition, for example. But while my calculator can perfectly add any numbers up to its memory limit, it has no understanding of addition.
  - simianwords 8mo ago
    It’s the degree of generalisability. And LLMs do have understanding. You can ask it how it came up with the process in natural language and it can help - something a calculator can’t do.
    - bigfishrunning 8mo ago
      > And LLMs do have understanding. They absolutely do not. If you "ask it how it came up with the process in natural language" with some input, it will produce an output that follows, because of the statistics encoded in the model. That output may or may not be helpful, but it is likely to be stylistically plausible. An LLM does not think or understand; it is merely a statistical model (that's what the M stands for!)
      - simianwords 8mo ago
        how would you empirically disprove that it doesn't have understanding? i can prove that it does have understanding because it behaves exactly like a human with understanding does. if i ask it to solve an integral and ask it questions about it - it replies exactly as if it has understood. give me a specific example so that we can stress test this argument. for example: what if we come up with a new board game with a completely new set of rules and see if it can reason about it and beat humans (or come close)?
        bigfishrunning 8mo ago
        We don't need to come up with a new board game. How about a board game that has been written about extensively for hundreds of years LLMs can't consistently win at chess https://www.nicowesterdale.com/blog/why-llms-cant-play-chess https://www.nicowesterdale.com/blog/why-llms-cant-play-chess Now, some of the best chess engines in the world are Neural Networks, but general purpose LLMs are consistently bad at chess. As far as "LLM's don't have understanding", that is axiomatically true by the nature of how they're implemented. A bunch of matrix multiplies resulting in a high-dimensional array of tokens does not think; this has been written about extensively. They are really good for generating language that looks plausible; some of that plausable-looking language happens to be true.
        simianwords 8mo ago
        false, chess ELO is pretty good https://maxim-saplin.github.io/llm_chess/ https://maxim-saplin.github.io/llm_chess/ ets not cherry pick and actually see benchmarks please. i would say even ~1000 elo means that it can reason better than the average human.
        bigfishrunning 8mo ago
        If you look at the "workflow" section of that page, they had to add a bunch of scaffolding around telling the model what moves are legal -- an llm can't keep enough context to know how to play chess; only to choose an advantageous move from a given list. But feel free to "cherry pick".
  - netdevphoenix 8mo ago
    > while my calculator can perfectly add any numbers up to its memory limit, it has no understanding of addition. "my calculator can perfectly add any numbers up to its memory limit" This kind of anthropomorphic language is misleading in these conversations. Your calculator isn't an agent so it should not be expected to be capable of any cognition.
- Talanes 8mo ago
  The real sticking point for me is I don't even believe that authors themselves FULLY understand their process. The idea that anybody could achieve such full introspection as to understand and articulate every little thing that influences their output seems astoundingly improbable.

simianwords 8mo ago

if we have steps for understanding any author's english and creative process (generally not specific to an author) would you agree then it is possible for an llm to do it?

mysterydip 8mo ago
Repeating a process, yes for sure, even (pseudorandom?) variations on a process. Understanding a process is a different question, and I’m not sure how you would measure that. In school we would have a test with various questions to show you understand the concept of addition, for example. But while my calculator can perfectly add any numbers up to its memory limit, it has no understanding of addition.
- simianwords 8mo ago
  It’s the degree of generalisability. And LLMs do have understanding. You can ask it how it came up with the process in natural language and it can help - something a calculator can’t do.
  - bigfishrunning 8mo ago
    > And LLMs do have understanding. They absolutely do not. If you "ask it how it came up with the process in natural language" with some input, it will produce an output that follows, because of the statistics encoded in the model. That output may or may not be helpful, but it is likely to be stylistically plausible. An LLM does not think or understand; it is merely a statistical model (that's what the M stands for!)
    - simianwords 8mo ago
      how would you empirically disprove that it doesn't have understanding? i can prove that it does have understanding because it behaves exactly like a human with understanding does. if i ask it to solve an integral and ask it questions about it - it replies exactly as if it has understood. give me a specific example so that we can stress test this argument. for example: what if we come up with a new board game with a completely new set of rules and see if it can reason about it and beat humans (or come close)?
      - bigfishrunning 8mo ago
        We don't need to come up with a new board game. How about a board game that has been written about extensively for hundreds of years LLMs can't consistently win at chess https://www.nicowesterdale.com/blog/why-llms-cant-play-chess https://www.nicowesterdale.com/blog/why-llms-cant-play-chess Now, some of the best chess engines in the world are Neural Networks, but general purpose LLMs are consistently bad at chess. As far as "LLM's don't have understanding", that is axiomatically true by the nature of how they're implemented. A bunch of matrix multiplies resulting in a high-dimensional array of tokens does not think; this has been written about extensively. They are really good for generating language that looks plausible; some of that plausable-looking language happens to be true.
        simianwords 8mo ago
        false, chess ELO is pretty good https://maxim-saplin.github.io/llm_chess/ https://maxim-saplin.github.io/llm_chess/ ets not cherry pick and actually see benchmarks please. i would say even ~1000 elo means that it can reason better than the average human.
        bigfishrunning 8mo ago
        If you look at the "workflow" section of that page, they had to add a bunch of scaffolding around telling the model what moves are legal -- an llm can't keep enough context to know how to play chess; only to choose an advantageous move from a given list. But feel free to "cherry pick".
- netdevphoenix 8mo ago
  > while my calculator can perfectly add any numbers up to its memory limit, it has no understanding of addition. "my calculator can perfectly add any numbers up to its memory limit" This kind of anthropomorphic language is misleading in these conversations. Your calculator isn't an agent so it should not be expected to be capable of any cognition.
Talanes 8mo ago
The real sticking point for me is I don't even believe that authors themselves FULLY understand their process. The idea that anybody could achieve such full introspection as to understand and articulate every little thing that influences their output seems astoundingly improbable.