4 ms·

You might want to do some research into the words “science” and “pedantry”, since the former is just meticulous application of the latter. Calling a scientist p

You might want to do some research into the words “science” and “pedantry”, since the former is just meticulous application of the latter. Calling a scientist pedantic is a compliment. That aside, you’ve been arguing that these models understand things and citing these papers as evidence. They are not. They are evidence that the ability of these models to generate text based off of their existing training set can easily be finetuned in a number of ways to add training sets after the initial zero-shot learning. That’s all these models do, they generate text based upon some training set. If we define understanding as the ability to extrapolate beyond what one has been told, they are expressly not doing that. Your papers explain this quite well. Edit: to see more concrete examples of this, look into the unfortunately named “hallucination” ability of LLMs. Once you realize that they only know what they were told and are unable to logically extrapolate the point becomes clearer. I hope that helps.

kilgnad 4y ago
> You might want to do some research into the words “science” and “pedantry”, since the former is just meticulous application of the latter. Calling a scientist pedantic is a compliment. Not only am I extremely well versed in the definition and philosophy of the word science (likely much more well-versed than you), but you are completely and utterly wrong about the compliment part. A scientist is a human, if I call a scientist pedantic during a normal discussion then the scientist will take it as an insult. Do you think a scientist has conditioned his mind into a sort of emotionless robot who can needlessly branch off onto a debate about the definition of the word "science" and "pedantry" when the topic is actuality "machine learning"? No. A scientist can both be pedantic and stupid, being a scientist does not preclude one from being human. >That aside, you’ve been arguing that these models understand things and citing these papers as evidence. They are not. They are evidence that the ability of these models to generate text based off of their existing training set can easily be finetuned in a number of ways to add training sets after the initial zero-shot learning. I posted two papers. You're conveniently ignoring the first and naively mistaken about the second. Part of "understanding" is the ability to formulate new theorems from previously known facts, in order to do this one must "understand" how these facts compose to form new statements. This is what's happening in the fine tuning. It is a demonstration of understanding... that it knows how disparate knowledge composes to form new knowledge. The very definition of understanding. >That’s all these models do, they generate text based upon some training set. If we define understanding as the ability to extrapolate beyond what one has been told, they are expressly not doing that. Your papers explain this quite well. Of course. You cannot extrapolate anything beyond what you Observe as well. Can you literally form new knowledge out of thin air? No. You have three things: Existing knowledge, knowledge through observation, and knowledge through composition of existing knowledge. Without introducing new knowledge, LLMs can be coerced to compose existing knowledge to form new knowledge. Additionally, in the ICL step they can be introduced to new knowledge and form additional compositions there. This has been demonstrated repeatedly. >Edit: to see more concrete examples of this, look into the unfortunately named “hallucination” ability of LLMs. Once you realize that they only know what they were told and are unable to logically extrapolate the point becomes clearer. I hope that helps. It's obvious chatGPT makes stuff up. Every one who has worked with LLMs in depth is fully aware of this. It's an obvious thing, you don't even have to "look it up" everyone knows about it. This claim is made DESPITE the fact that LLMs hallucinate. It's obvious these models are imperfect and it's obvious they have huge deficiencies. But when it doesn't hallucinate, when the answer is Novel, creative, correct and unmistakably not existing in any training set, then we know the model understood the query you gave it.
- naasking 4y ago
  > You have three things: Existing knowledge, knowledge through observation, and knowledge through composition of existing knowledge. I mostly agree with your position but have a quibble with this characterization. Knowledge can also be generated from randomization and enumeration. For instance, we could enumerate all Turing machines that might satisfy some property, or we could randomly permute some Turing machine as in genetic algorithms to find some new behaviours. You might be tempted to categorize these under "composition", but I think they have different properties from composition, which is typically understood to be a finite deterministic map. Enumeration is potentially unbounded, and random mutation is non-determinstic.
  - kilgnad 4y ago
    It's trivial to add randomness too the model. chatGPT, I believe already has it. Simply add a new input neuron with a random seed or add a tiny bit of noise to some weights or seed the tokens yourself in your query. "<Query Seed: 4334> hello chatGPT, how are you?" In this case chatGPT can deliberately randomize the response through understanding the intent of what you mean by "Query Seed". In the brain, if such randomness existed, it would largely be modelled as a similar mechanism. A seed value (or multiple seeds at different places) is either inserted near the input step or happens at a branching logic step. Additionally, the query to the human brain can also be seeded. True randomness requires that this seed number comes from quantum properties of particles that expresses itself in a sort of macro level random number. There is no other known source of true randomness in nature, though we can get perceptually identical results through just seeding with timestamps. Either way it's trivial to add and not critical to what we mean by the word "understanding" because it's both easy to add and we aren't even sure if we have such randomness is in our brains. If it existed and if a human had this mechanism removed from his brain we would still say that this human is capable of "understanding" things.
    - naasking 4y ago
      I'm not talking about ChatGPT or anything like that, I'm just taking slight issue with your characterizing of our sources of knowledge. I think there are more sources than you laid out, as I explained.

kilgnad 4y ago

> You might want to do some research into the words “science” and “pedantry”, since the former is just meticulous application of the latter. Calling a scientist pedantic is a compliment. Not only am I extremely well versed in the definition and philosophy of the word science (likely much more well-versed than you), but you are completely and utterly wrong about the compliment part. A scientist is a human, if I call a scientist pedantic during a normal discussion then the scientist will take it as an insult. Do you think a scientist has conditioned his mind into a sort of emotionless robot who can needlessly branch off onto a debate about the definition of the word "science" and "pedantry" when the topic is actuality "machine learning"? No. A scientist can both be pedantic and stupid, being a scientist does not preclude one from being human. >That aside, you’ve been arguing that these models understand things and citing these papers as evidence. They are not. They are evidence that the ability of these models to generate text based off of their existing training set can easily be finetuned in a number of ways to add training sets after the initial zero-shot learning. I posted two papers. You're conveniently ignoring the first and naively mistaken about the second. Part of "understanding" is the ability to formulate new theorems from previously known facts, in order to do this one must "understand" how these facts compose to form new statements. This is what's happening in the fine tuning. It is a demonstration of understanding... that it knows how disparate knowledge composes to form new knowledge. The very definition of understanding. >That’s all these models do, they generate text based upon some training set. If we define understanding as the ability to extrapolate beyond what one has been told, they are expressly not doing that. Your papers explain this quite well. Of course. You cannot extrapolate anything beyond what you Observe as well. Can you literally form new knowledge out of thin air? No. You have three things: Existing knowledge, knowledge through observation, and knowledge through composition of existing knowledge. Without introducing new knowledge, LLMs can be coerced to compose existing knowledge to form new knowledge. Additionally, in the ICL step they can be introduced to new knowledge and form additional compositions there. This has been demonstrated repeatedly. >Edit: to see more concrete examples of this, look into the unfortunately named “hallucination” ability of LLMs. Once you realize that they only know what they were told and are unable to logically extrapolate the point becomes clearer. I hope that helps. It's obvious chatGPT makes stuff up. Every one who has worked with LLMs in depth is fully aware of this. It's an obvious thing, you don't even have to "look it up" everyone knows about it. This claim is made DESPITE the fact that LLMs hallucinate. It's obvious these models are imperfect and it's obvious they have huge deficiencies. But when it doesn't hallucinate, when the answer is Novel, creative, correct and unmistakably not existing in any training set, then we know the model understood the query you gave it.

naasking 4y ago
> You have three things: Existing knowledge, knowledge through observation, and knowledge through composition of existing knowledge. I mostly agree with your position but have a quibble with this characterization. Knowledge can also be generated from randomization and enumeration. For instance, we could enumerate all Turing machines that might satisfy some property, or we could randomly permute some Turing machine as in genetic algorithms to find some new behaviours. You might be tempted to categorize these under "composition", but I think they have different properties from composition, which is typically understood to be a finite deterministic map. Enumeration is potentially unbounded, and random mutation is non-determinstic.
- kilgnad 4y ago
  It's trivial to add randomness too the model. chatGPT, I believe already has it. Simply add a new input neuron with a random seed or add a tiny bit of noise to some weights or seed the tokens yourself in your query. "<Query Seed: 4334> hello chatGPT, how are you?" In this case chatGPT can deliberately randomize the response through understanding the intent of what you mean by "Query Seed". In the brain, if such randomness existed, it would largely be modelled as a similar mechanism. A seed value (or multiple seeds at different places) is either inserted near the input step or happens at a branching logic step. Additionally, the query to the human brain can also be seeded. True randomness requires that this seed number comes from quantum properties of particles that expresses itself in a sort of macro level random number. There is no other known source of true randomness in nature, though we can get perceptually identical results through just seeding with timestamps. Either way it's trivial to add and not critical to what we mean by the word "understanding" because it's both easy to add and we aren't even sure if we have such randomness is in our brains. If it existed and if a human had this mechanism removed from his brain we would still say that this human is capable of "understanding" things.
  - naasking 4y ago
    I'm not talking about ChatGPT or anything like that, I'm just taking slight issue with your characterizing of our sources of knowledge. I think there are more sources than you laid out, as I explained.