4 ms·

> We use language to describe symbols, not even very precisely, and we can convert imprecise language to more precise symbols in our brain, manipulate them as s

> We use language to describe symbols, not even very precisely, and we can convert imprecise language to more precise symbols in our brain, manipulate them as symbols, and only then turn them back into language. I don't think that's true for all people. I know that some people manipulate words in their heads, others images, I manipulate sounds and images. Language is just a noisy medium through which we communicate the internal state of our brain or its outputs to other people / humans and ourselves. > can learn to do math that ChatGTP never could without an API. GPT4 does just fine in some cases and extrapolates just fine in others, e.g. ask it whether there are more wheels or doors, and try to investigate the definitions of either and see how well it adds the numbers. > When Newton invented calculus he wasn’t predictively generating words by token; There are very few people in history up to Newton so I don't think it's fair to hold up what is essentially a new field up to him. > he performed logical manipulation of symbols in his brain first. We don't know "how" he did that. We don't know that his brain manipulated symbols everything he did. We simply know that Calculus can be derived from a set of axioms following logical inference. What you are expressing is largely true for many primates, and according to some, our brains are "just linearly scaled primate brains". > That’s why LLMs can’t tell you where they got a specific piece of their own output from, while a human can. I don't think that is correct. The human might provide a justification for something but that doesn't mean it is the true reason they reached a conclusion. The only way this happens is if you apply logical operators, at which point we are doing math again. It turns out that our brains have decided long before we are even aware of the decision, such decisions may be guided by external stimulation, or even by internal stimulation since our neural networks don't have well defined components and boundaries thus neighbouring neurons can affect or even trigger circuits, and our own forward predictive models back propagate information to other circuits. > If fundamentally human thinking was just “LLM” we would have never generated the words to train ourselves on in the first place! I don't think that's true. Language has evolved over thousands of years in many different ways by > 80 bn humans each of whom having 80bn neurons and trillions of synapses. Yet, we have found that models can learn to communicate with each other and derive their own languages. I highly recommend you read Eagleman's "The Brain: The Story of You". It covers nearly everything I spoke of here and is very easy to read / listen to. https://pubmed.ncbi.nlm.nih.gov/19915731/ https://pubmed.ncbi.nlm.nih.gov/19915731/

joshuahedlund 4y ago
> I know that some people manipulate words in their heads, others images, I manipulate sounds and images. Language is just a noisy medium through which we communicate the internal state of our brain or its outputs to other people / humans and ourselves. We are in agreement here. I think you are only strengthening my argument that language is too imprecise and restrictive for LLMs to be fundamentally equivalent to human thinking. > GPT4 does just fine in some cases and extrapolates just fine in others "just fine" is not very persuasive here. A nine-year-old can do much better than "just fine" after learning some very simple rules with very minimal examples. And I conjecture that if you removed a lot of the mathematical examples from GTP's training corpus, to be more equivalent to what a nine-year-old has seen, it would do even worse. And it's fundamentally because it cannot break out of its linear language limits and understand numbers as abstract symbols that can be manipulated before converting back into language. > I don't think that is correct. The human might provide a justification for something but that doesn't mean it is the true reason they reached a conclusion. Sometimes, yes. What I meant here is that a human can (sometimes - when it is acting intelligently) specifically repeat information they learned and specifically recall the source of that information. This is how our entire scientific process works; we can go back and look up exactly how we derived any piece of our collective knowledge, verify or repeat it if necessary, or build on it further. You're proving my point by the citations you are giving me! (thank you for them) As an intelligent human, you do not "hallucinate" sources, you can provide real ones and provide them directly. And you can do this because - to go back to my original argument - your intelligence is fundamentally different than an LLM. (That's not an argument that AI is impossible, only that we work differently somehow than what we've seen so far.)
- PartiallyTyped 4y ago
  The fact that GPT understands how to use tools suggests that not only does it understand the meaning of numbers, it also understands its own limitations. By all means, the argument around numeracy is bogus, as lots of people have numeracy issues but they know how to use a calculator. The fact that so many people seem stuck up over the inability to write perfect math when it can do in context learning of a novel programming language, do addition over groups where addition is not defined as a+b but as a+b+c where c is a constant is incredible. If we held humans to the same standard we hold GPT3.5+ models, the vast majority pf humans would fail. The fact that it needs as much data as it does is simply an architectural issue and not inherent to the model itself. As for hallucinations; I will point to whole thing that religion is, a mass psychosis, Eagleman's book goes into great detail on how we hallucinate our reality.

joshuahedlund 4y ago

> I know that some people manipulate words in their heads, others images, I manipulate sounds and images. Language is just a noisy medium through which we communicate the internal state of our brain or its outputs to other people / humans and ourselves. We are in agreement here. I think you are only strengthening my argument that language is too imprecise and restrictive for LLMs to be fundamentally equivalent to human thinking. > GPT4 does just fine in some cases and extrapolates just fine in others "just fine" is not very persuasive here. A nine-year-old can do much better than "just fine" after learning some very simple rules with very minimal examples. And I conjecture that if you removed a lot of the mathematical examples from GTP's training corpus, to be more equivalent to what a nine-year-old has seen, it would do even worse. And it's fundamentally because it cannot break out of its linear language limits and understand numbers as abstract symbols that can be manipulated before converting back into language. > I don't think that is correct. The human might provide a justification for something but that doesn't mean it is the true reason they reached a conclusion. Sometimes, yes. What I meant here is that a human can (sometimes - when it is acting intelligently) specifically repeat information they learned and specifically recall the source of that information. This is how our entire scientific process works; we can go back and look up exactly how we derived any piece of our collective knowledge, verify or repeat it if necessary, or build on it further. You're proving my point by the citations you are giving me! (thank you for them) As an intelligent human, you do not "hallucinate" sources, you can provide real ones and provide them directly. And you can do this because - to go back to my original argument - your intelligence is fundamentally different than an LLM. (That's not an argument that AI is impossible, only that we work differently somehow than what we've seen so far.)

PartiallyTyped 4y ago
The fact that GPT understands how to use tools suggests that not only does it understand the meaning of numbers, it also understands its own limitations. By all means, the argument around numeracy is bogus, as lots of people have numeracy issues but they know how to use a calculator. The fact that so many people seem stuck up over the inability to write perfect math when it can do in context learning of a novel programming language, do addition over groups where addition is not defined as a+b but as a+b+c where c is a constant is incredible. If we held humans to the same standard we hold GPT3.5+ models, the vast majority pf humans would fail. The fact that it needs as much data as it does is simply an architectural issue and not inherent to the model itself. As for hallucinations; I will point to whole thing that religion is, a mass psychosis, Eagleman's book goes into great detail on how we hallucinate our reality.