5 ms·

You can think of a neural network as millions of NAND gates, but those gates are not conforming to the problem if you ask it what 1+1 is. They're conforming to

You can think of a neural network as millions of NAND gates, but those gates are not conforming to the problem if you ask it what 1+1 is. They're conforming to the known answers of 1+1, 1+2, 2+2, etc. They are in other words limited by the size of their memorization of patterns of tokens. (Which is enormous). They have no handle on any underlying logic. They can explain addition to you but they can't perform it without calling out to an external API. In dealing with complicated code, this is a very insidious problem, because they can often write functions that compile and run, but which miss edge cases because there is no actual logic behind them. [edit] I should add that a sufficiently large NN may be able to create an entire computer inside itself that can run code, but even that would not be reliable because it will have arrived at the gating by inference rather than starting from a deterministic root.

necovek 3y ago
While I agree in general, I would venture to say that even humans don't really use logic to say what 1 + 1 is. Best evidence for this is the construction of natural numbers in traditional math* with `successor` function (A. There is a one (1). B. There exists a successor of one (succ(1)) called 2.) — everything else follows from that using formal logic, and that's why everything else is generally out of reach for generative ML models (eg. summation can be defined in multiple ways, one of them being successive application of a successor function the right number of times on one of the operands). * Modern approach is to use set cardinality instead, but the essence is the same: something has to be memorized (axioms), and then logic is applied.
- noduerme 3y ago
  Hm. Well I agree that humans probably arrive at 1+1 by induction first, but quickly progress to the application of formal logic. Reasoning is essential to human development. Take a 3-year-old who's holding two pieces of candy. Say, "I'm going to take these and give you two pieces back." Then give him one piece back. He's going to start crying. Perhaps he can't explain why, exactly. But if you ask him why it's not fair, he's going to tell you that two is more than one. Ask why is two more than one: It doesn't require succession. It's a fact of the reality we inhabit. We can measure it with our hands, our eyes, its weight. We can hear two sounds, two voices at once and differentiate them (non-blocking, nonlinear processing). AIs don't inhabit any reality grounded in the physical constraints they purport to understand, therefore they can never arrive at the conclusion that 2 > 1. They just appear to "know" it because it's part of the data set. But once you understand that 2 > 1, then 3 must be > 2. If acquisition is hardwired into our brains, then so is the capacity for logically explaining why Y > X.
- ralegh 3y ago
  Yeah I'd agree, we learn addition/multiplication/etc as processes of smaller problems. If you gave an LLM a prompting framework to do addition I'm sure the results would be better (add the units, add the tens, Food for thought- would a savant use the same process? Or have they somehow recruited more of their brain to where they can memorize much larger problems.
  - noduerme 3y ago
    So first of all, prompting and re-prompting an LLM is basically forcing it to deduce rather than induce; using millions of gates to get from 1+1 to 1+2. That's what our brain does, too (uses millions of gates for dumb stuff), but we designed computers to do that using 4 bits, so it's ironic that we're now trying to write scripts to force something with 60 billion parameters to do the same thing. I think savants usually solve problems in the most deductive way, using reasoning that leads to other reasoning. I went to an elementary school in the 80s where more than half the kids would now be labeled autistic... some got into math programs at colleges by the age of 12. I believe it's all pure reasoning, not like some magical calculator that spat out answers they didn't understand the reasons for. [edit] If you meant: Do savants solve problems by recursively breaking problems into smaller and smaller problems, then yes. But the breaking-apart-of-problems is actually the hard problem, not the solving.

necovek 3y ago

While I agree in general, I would venture to say that even humans don't really use logic to say what 1 + 1 is. Best evidence for this is the construction of natural numbers in traditional math* with `successor` function (A. There is a one (1). B. There exists a successor of one (succ(1)) called 2.) — everything else follows from that using formal logic, and that's why everything else is generally out of reach for generative ML models (eg. summation can be defined in multiple ways, one of them being successive application of a successor function the right number of times on one of the operands). * Modern approach is to use set cardinality instead, but the essence is the same: something has to be memorized (axioms), and then logic is applied.

noduerme 3y ago
Hm. Well I agree that humans probably arrive at 1+1 by induction first, but quickly progress to the application of formal logic. Reasoning is essential to human development. Take a 3-year-old who's holding two pieces of candy. Say, "I'm going to take these and give you two pieces back." Then give him one piece back. He's going to start crying. Perhaps he can't explain why, exactly. But if you ask him why it's not fair, he's going to tell you that two is more than one. Ask why is two more than one: It doesn't require succession. It's a fact of the reality we inhabit. We can measure it with our hands, our eyes, its weight. We can hear two sounds, two voices at once and differentiate them (non-blocking, nonlinear processing). AIs don't inhabit any reality grounded in the physical constraints they purport to understand, therefore they can never arrive at the conclusion that 2 > 1. They just appear to "know" it because it's part of the data set. But once you understand that 2 > 1, then 3 must be > 2. If acquisition is hardwired into our brains, then so is the capacity for logically explaining why Y > X.
ralegh 3y ago
Yeah I'd agree, we learn addition/multiplication/etc as processes of smaller problems. If you gave an LLM a prompting framework to do addition I'm sure the results would be better (add the units, add the tens, Food for thought- would a savant use the same process? Or have they somehow recruited more of their brain to where they can memorize much larger problems.
- noduerme 3y ago
  So first of all, prompting and re-prompting an LLM is basically forcing it to deduce rather than induce; using millions of gates to get from 1+1 to 1+2. That's what our brain does, too (uses millions of gates for dumb stuff), but we designed computers to do that using 4 bits, so it's ironic that we're now trying to write scripts to force something with 60 billion parameters to do the same thing. I think savants usually solve problems in the most deductive way, using reasoning that leads to other reasoning. I went to an elementary school in the 80s where more than half the kids would now be labeled autistic... some got into math programs at colleges by the age of 12. I believe it's all pure reasoning, not like some magical calculator that spat out answers they didn't understand the reasons for. [edit] If you meant: Do savants solve problems by recursively breaking problems into smaller and smaller problems, then yes. But the breaking-apart-of-problems is actually the hard problem, not the solving.