2 ms·

I don't think this tone is at all justified. If you think otherwise, I do ask that you point out where I went too far in a comment that I feared was overburdene

I don't think this tone is at all justified. If you think otherwise, I do ask that you point out where I went too far in a comment that I feared was overburdened by caveats and admissions of my own human flaws. "This sentence has five words" is going to appear far more often than "This sentence has four words". This is the entire premise of LLMs working at all, stochastic parrots or otherwise.

dwa3592 3mo ago
you are right, i was more curt than i should have been. apologies. but you helped prove my point: >>"This sentence has five words" is going to appear far more often than "This sentence has four words". it's not about this at all. your point is about data quality. you need to take a step back. the point is that if you trained a language model just on this data set which has sentences akin to "this sentence has two words" - the model is going to learn that. this shows that the language modeling itself doesn't truly provide an understanding of the real world. you can train a language model with the most advanced technology on shitty data and the model will start providing shitty outputs confidently - the model will never say "hey there is something wrong with the data i am trained on". thats what 'understanding of the world' meant in that paper.
- ToValueFunfetti 3mo ago
  If someone substituted all of your sensory inputs for something else for your entire life, how would you notice? If you wore contacts from birth that made the sky red and earbuds that censored when people said it was blue, on what basis would you realize that was wrong? I don't see what this says about the architecture of your brain, and I don't think it's the point being made in the paper. That the training data must statistically connect to reality in order for the model to model reality doesn't seem that important.
  - dwa3592 3mo ago
    >>If someone substituted all of your sensory inputs for something else for your entire life, how would you notice? I don't know how or if I will notice. That's the biology, chemistry and physics of the brain that I don't know. I hope someone is looking into it. But this does not mean in any way that LLMs are similar to our brains!!!!!! WE DON'T KNOW HOW OUR BRAINS WORK. So going back to language modeling - language models were stochastic in nature when that paper was written, they still are albeit we are trying to make them as deterministic as possible.
    - ToValueFunfetti 3mo ago
      I'm quite sure that LLMs are not one-to-one with our brains. It would at least be surprising if instrumental convergence was that powerful, and, even in that case, they're still not getting hungry or sleepy or having dreams, etc. At best they'd match up with a smaller region of our brains, and I'd take 9:1 odds against a close match. But the paper argues a much stronger point than "LLMs are not similar to human brains": it claims LLMs cannot engage with meaning because they model human language instead of sensory input or human thought. I also don't think human brains are particularly deterministic.

dwa3592 3mo ago

you are right, i was more curt than i should have been. apologies. but you helped prove my point: >>"This sentence has five words" is going to appear far more often than "This sentence has four words". it's not about this at all. your point is about data quality. you need to take a step back. the point is that if you trained a language model just on this data set which has sentences akin to "this sentence has two words" - the model is going to learn that. this shows that the language modeling itself doesn't truly provide an understanding of the real world. you can train a language model with the most advanced technology on shitty data and the model will start providing shitty outputs confidently - the model will never say "hey there is something wrong with the data i am trained on". thats what 'understanding of the world' meant in that paper.

ToValueFunfetti 3mo ago
If someone substituted all of your sensory inputs for something else for your entire life, how would you notice? If you wore contacts from birth that made the sky red and earbuds that censored when people said it was blue, on what basis would you realize that was wrong? I don't see what this says about the architecture of your brain, and I don't think it's the point being made in the paper. That the training data must statistically connect to reality in order for the model to model reality doesn't seem that important.
- dwa3592 3mo ago
  >>If someone substituted all of your sensory inputs for something else for your entire life, how would you notice? I don't know how or if I will notice. That's the biology, chemistry and physics of the brain that I don't know. I hope someone is looking into it. But this does not mean in any way that LLMs are similar to our brains!!!!!! WE DON'T KNOW HOW OUR BRAINS WORK. So going back to language modeling - language models were stochastic in nature when that paper was written, they still are albeit we are trying to make them as deterministic as possible.
  - ToValueFunfetti 3mo ago
    I'm quite sure that LLMs are not one-to-one with our brains. It would at least be surprising if instrumental convergence was that powerful, and, even in that case, they're still not getting hungry or sleepy or having dreams, etc. At best they'd match up with a smaller region of our brains, and I'd take 9:1 odds against a close match. But the paper argues a much stronger point than "LLMs are not similar to human brains": it claims LLMs cannot engage with meaning because they model human language instead of sensory input or human thought. I also don't think human brains are particularly deterministic.