3 ms·

> So ChatGPT at least has learned addition. Or, OpenAI added a calculator function. Furthermore, I would add: - ChatGPT hasn't "learned" anything outside of w

> So ChatGPT at least has learned addition. Or, OpenAI added a calculator function. Furthermore, I would add: - ChatGPT hasn't "learned" anything outside of what exists in it's model. - If it is an AI-based response, it's still derived from token-based inferencing. - One run of asking ChatGPT something is not enough to prove much of anything. > I'm sure if someone analyzed the network carefully enough they could probably find the digit add/carry neurons. They will find tokens for 'dig' and 'it', as well as 'add', 'car' and 'ry'. They will not find internalized understanding of the concept of math.

famouswaffles 4y ago
>Or, OpenAI added a calculator function. Furthermore, I would add: No lol. The test doesn't have to be mental arithmetic. and accuracy mistakes creep in at large numbers for multiplication. That's not how calculators work i'm sure you know >ChatGPT hasn't "learned" anything outside of what exists in it's model. I'm sorry...and you have? >If it is an AI-based response, it's still derived from token-based inferencing. Um...Ok? Lol >One run of asking ChatGPT something is not enough to prove much of anything. You can run this on gpt-4 as much as you like. the results are the same. It knows addition. >They will find tokens for 'dig' and 'it', as well as 'add', 'car' and 'ry'. They will not find internalized understanding of the concept of math. That's...not how that works lol, You don't probe neurons and see tokens. https://clementneo.com/posts/2023/02/11/we-found-an-neuron https://clementneo.com/posts/2023/02/11/we-found-an-neuron Weights don't store the data they train on like that. They are essentially configuration settings.
- deleted 4y ago
  [deleted]
lhl 4y ago
> Or, OpenAI added a calculator function. I just asked my local LLaMA 30Bq4 "What is 23214 + 34243?" and it gave me 57457 so there's probably some innate math abilities in larger LLMs. That being said, I'm much more impressed that you can literally just tell one of these new LLMs to use a calculator for math calculations (or to do web searches, or whatever) in English and it will understand and actually do so. That's a lot more impressive (and useful) to me.
flangola7 4y ago
>They will find tokens for 'dig' and 'it', as well as 'add', 'car' and 'ry'. Neurons are not tokens. Technical jargon isn't fungible. >They will not find internalized understanding of the concept of math. I'm not sure how you figure that. The AlphaGo Zero model is able to learn and reach massively superhuman ability on any board game thrown at, including new ones not in its training set. I don't see how someone or something goes about mopping the floor with grandmasters (and the top champions of every other game) without having some kind of internal understanding of what it is you're playing. If there isn't an internal understanding then what is it doing? Sure we can say it is just predicting tokens, but obviously it using more than just random chance to predict them or else the output would be gibberish. What is inside those dozens of neural layers may not be a familiar form like logic gates assembled into add and carry circuits, but clearly some type of decision structure exists.
- ttpphd 4y ago
  I argue against the notion that a large language model has an "internal understanding". It has parameter weights. For some philosophy on the matter, here is some Wittgenstein as quoted in https://doi.org/10.1016/j.langsci.2011.04.028 https://doi.org/10.1016/j.langsci.2011.04.028 >“Try not to think of understanding as a ‘mental process’ at all – For that is the expression which confuses you… In the sense in which there are processes (including mental processes) which are characteristic of understanding, understanding is not itself a mental process… Thus what I wanted to say was: when he suddenly knew how to go on, when he Understood the principle [of, e.g., the number series to be completed], Then possibly he had a special experience – and if he is asked: ‘What was it? What took place when you suddenly grasped the principle?’ perhaps he will describe it… – but for us it is the circumstances under which he had such an experience that justify him is saying in such a case that he understands." (Wittgenstein, 1968, paragraphs 154–155)

famouswaffles 4y ago

>Or, OpenAI added a calculator function. Furthermore, I would add: No lol. The test doesn't have to be mental arithmetic. and accuracy mistakes creep in at large numbers for multiplication. That's not how calculators work i'm sure you know >ChatGPT hasn't "learned" anything outside of what exists in it's model. I'm sorry...and you have? >If it is an AI-based response, it's still derived from token-based inferencing. Um...Ok? Lol >One run of asking ChatGPT something is not enough to prove much of anything. You can run this on gpt-4 as much as you like. the results are the same. It knows addition. >They will find tokens for 'dig' and 'it', as well as 'add', 'car' and 'ry'. They will not find internalized understanding of the concept of math. That's...not how that works lol, You don't probe neurons and see tokens. https://clementneo.com/posts/2023/02/11/we-found-an-neuron https://clementneo.com/posts/2023/02/11/we-found-an-neuron Weights don't store the data they train on like that. They are essentially configuration settings.

deleted 4y ago
[deleted]

lhl 4y ago

> Or, OpenAI added a calculator function. I just asked my local LLaMA 30Bq4 "What is 23214 + 34243?" and it gave me 57457 so there's probably some innate math abilities in larger LLMs. That being said, I'm much more impressed that you can literally just tell one of these new LLMs to use a calculator for math calculations (or to do web searches, or whatever) in English and it will understand and actually do so. That's a lot more impressive (and useful) to me.

flangola7 4y ago

>They will find tokens for 'dig' and 'it', as well as 'add', 'car' and 'ry'. Neurons are not tokens. Technical jargon isn't fungible. >They will not find internalized understanding of the concept of math. I'm not sure how you figure that. The AlphaGo Zero model is able to learn and reach massively superhuman ability on any board game thrown at, including new ones not in its training set. I don't see how someone or something goes about mopping the floor with grandmasters (and the top champions of every other game) without having some kind of internal understanding of what it is you're playing. If there isn't an internal understanding then what is it doing? Sure we can say it is just predicting tokens, but obviously it using more than just random chance to predict them or else the output would be gibberish. What is inside those dozens of neural layers may not be a familiar form like logic gates assembled into add and carry circuits, but clearly some type of decision structure exists.

ttpphd 4y ago
I argue against the notion that a large language model has an "internal understanding". It has parameter weights. For some philosophy on the matter, here is some Wittgenstein as quoted in https://doi.org/10.1016/j.langsci.2011.04.028 https://doi.org/10.1016/j.langsci.2011.04.028 >“Try not to think of understanding as a ‘mental process’ at all – For that is the expression which confuses you… In the sense in which there are processes (including mental processes) which are characteristic of understanding, understanding is not itself a mental process… Thus what I wanted to say was: when he suddenly knew how to go on, when he Understood the principle [of, e.g., the number series to be completed], Then possibly he had a special experience – and if he is asked: ‘What was it? What took place when you suddenly grasped the principle?’ perhaps he will describe it… – but for us it is the circumstances under which he had such an experience that justify him is saying in such a case that he understands." (Wittgenstein, 1968, paragraphs 154–155)