3 ms·

>> Software : Input Data + Handcrafted Logic = Expected Output Machine Learning : Input Data + Expected Output = Learned Logic Let me stop you right there. No

>> Software : Input Data + Handcrafted Logic = Expected Output Machine Learning : Input Data + Expected Output = Learned Logic Let me stop you right there. No logic is learned in this process. [edit] Also, the LLM is inductive, not deductive. That is, it can only generalize based on observable facts, not universalize based on logical conditions. This also goes to the question of whether a logical statement itself can ever be arrived at by induction, such as whether the absence of life in the observable universe is a problem of our ability to observe or a generally applicable phenomenon. But for the purpose of LLMs we have to conclude that no, it can't find logic by reducing a set of outcomes, regardless of the size of the set. All it can do is find a set of incomprehensible equations that seem to fit the set in every example you throw at it. That's not logic, it's a lens.

famouswaffles 3y ago
You can train a small transformers that learns an algorithm for addition and generalizes perfectly. https://www.alignmentforum.org/posts/N6WM6hs7RQMKDhYjB/a-mechanistic-interpretability-analysis-of-grokking https://www.alignmentforum.org/posts/N6WM6hs7RQMKDhYjB/a-mec... >But for the purpose of LLMs we have to conclude that no, it can't find logic by reducing a set of outcomes, regardless of the size of the set. A ML model is probably not going to converge strictly on a formal logic system practically and there's also the question of if formal logic even underpins the result of the dataset you're feeding it, but that's entirely different from saying it cannot in principle.
vasco 3y ago
I was under the impression that there's a branch of models that use Logic gate networks and in that case the "logic" part seems sensible. The way I think about it is that a machine learning model is a hairball of NAND gates and spaghetti connections that nobody can explain, but work, and in that sense it'd be "logic" in one sense but not in the human logic thinking sense. Like if you pressed "random" on an FPGA millions of times until it replied with sensible outputs.
- noduerme 3y ago
  You can think of a neural network as millions of NAND gates, but those gates are not conforming to the problem if you ask it what 1+1 is. They're conforming to the known answers of 1+1, 1+2, 2+2, etc. They are in other words limited by the size of their memorization of patterns of tokens. (Which is enormous). They have no handle on any underlying logic. They can explain addition to you but they can't perform it without calling out to an external API. In dealing with complicated code, this is a very insidious problem, because they can often write functions that compile and run, but which miss edge cases because there is no actual logic behind them. [edit] I should add that a sufficiently large NN may be able to create an entire computer inside itself that can run code, but even that would not be reliable because it will have arrived at the gating by inference rather than starting from a deterministic root.
  - necovek 3y ago
    While I agree in general, I would venture to say that even humans don't really use logic to say what 1 + 1 is. Best evidence for this is the construction of natural numbers in traditional math* with `successor` function (A. There is a one (1). B. There exists a successor of one (succ(1)) called 2.) — everything else follows from that using formal logic, and that's why everything else is generally out of reach for generative ML models (eg. summation can be defined in multiple ways, one of them being successive application of a successor function the right number of times on one of the operands). * Modern approach is to use set cardinality instead, but the essence is the same: something has to be memorized (axioms), and then logic is applied.
    - noduerme 3y ago
      Hm. Well I agree that humans probably arrive at 1+1 by induction first, but quickly progress to the application of formal logic. Reasoning is essential to human development. Take a 3-year-old who's holding two pieces of candy. Say, "I'm going to take these and give you two pieces back." Then give him one piece back. He's going to start crying. Perhaps he can't explain why, exactly. But if you ask him why it's not fair, he's going to tell you that two is more than one. Ask why is two more than one: It doesn't require succession. It's a fact of the reality we inhabit. We can measure it with our hands, our eyes, its weight. We can hear two sounds, two voices at once and differentiate them (non-blocking, nonlinear processing). AIs don't inhabit any reality grounded in the physical constraints they purport to understand, therefore they can never arrive at the conclusion that 2 > 1. They just appear to "know" it because it's part of the data set. But once you understand that 2 > 1, then 3 must be > 2. If acquisition is hardwired into our brains, then so is the capacity for logically explaining why Y > X.

famouswaffles 3y ago

You can train a small transformers that learns an algorithm for addition and generalizes perfectly. https://www.alignmentforum.org/posts/N6WM6hs7RQMKDhYjB/a-mechanistic-interpretability-analysis-of-grokking https://www.alignmentforum.org/posts/N6WM6hs7RQMKDhYjB/a-mec... >But for the purpose of LLMs we have to conclude that no, it can't find logic by reducing a set of outcomes, regardless of the size of the set. A ML model is probably not going to converge strictly on a formal logic system practically and there's also the question of if formal logic even underpins the result of the dataset you're feeding it, but that's entirely different from saying it cannot in principle.

vasco 3y ago

I was under the impression that there's a branch of models that use Logic gate networks and in that case the "logic" part seems sensible. The way I think about it is that a machine learning model is a hairball of NAND gates and spaghetti connections that nobody can explain, but work, and in that sense it'd be "logic" in one sense but not in the human logic thinking sense. Like if you pressed "random" on an FPGA millions of times until it replied with sensible outputs.

noduerme 3y ago
You can think of a neural network as millions of NAND gates, but those gates are not conforming to the problem if you ask it what 1+1 is. They're conforming to the known answers of 1+1, 1+2, 2+2, etc. They are in other words limited by the size of their memorization of patterns of tokens. (Which is enormous). They have no handle on any underlying logic. They can explain addition to you but they can't perform it without calling out to an external API. In dealing with complicated code, this is a very insidious problem, because they can often write functions that compile and run, but which miss edge cases because there is no actual logic behind them. [edit] I should add that a sufficiently large NN may be able to create an entire computer inside itself that can run code, but even that would not be reliable because it will have arrived at the gating by inference rather than starting from a deterministic root.
- necovek 3y ago
  While I agree in general, I would venture to say that even humans don't really use logic to say what 1 + 1 is. Best evidence for this is the construction of natural numbers in traditional math* with `successor` function (A. There is a one (1). B. There exists a successor of one (succ(1)) called 2.) — everything else follows from that using formal logic, and that's why everything else is generally out of reach for generative ML models (eg. summation can be defined in multiple ways, one of them being successive application of a successor function the right number of times on one of the operands). * Modern approach is to use set cardinality instead, but the essence is the same: something has to be memorized (axioms), and then logic is applied.
  - noduerme 3y ago
    Hm. Well I agree that humans probably arrive at 1+1 by induction first, but quickly progress to the application of formal logic. Reasoning is essential to human development. Take a 3-year-old who's holding two pieces of candy. Say, "I'm going to take these and give you two pieces back." Then give him one piece back. He's going to start crying. Perhaps he can't explain why, exactly. But if you ask him why it's not fair, he's going to tell you that two is more than one. Ask why is two more than one: It doesn't require succession. It's a fact of the reality we inhabit. We can measure it with our hands, our eyes, its weight. We can hear two sounds, two voices at once and differentiate them (non-blocking, nonlinear processing). AIs don't inhabit any reality grounded in the physical constraints they purport to understand, therefore they can never arrive at the conclusion that 2 > 1. They just appear to "know" it because it's part of the data set. But once you understand that 2 > 1, then 3 must be > 2. If acquisition is hardwired into our brains, then so is the capacity for logically explaining why Y > X.