5 ms·

No, only knowledge that was on the internet and specifically on sites like Reddit that were crawled to train the model. This is definitely not all human knowle

No, only knowledge that was on the internet and specifically on sites like Reddit that were crawled to train the model. This is definitely not all human knowledge!

aoeusnth1 3y ago
We don’t know how many books and academic papers were in the dataset.
- qbasic_forever 3y ago
  I can say with absolute certainty they did not include all books with all human knowledge. For example I suspect chat GPT has zero knowledge of how to speak native American languages that have effectively died with no remaining speakers and no complete written history of their exact usage.
  - hammyhavoc 3y ago
    It also has massive gaps when it comes to problems when porting video games, concepts in game audio like procedural mixing, and even basic stuff like not generating a terribad NGINX config despite being fed the documentation. Anything niche, it's appalling. Anything you can otherwise Google? Sure.
csomar 3y ago
Strong disagree. Will be writing a blog post how GPT-4 understands languages that don't even exist (at least not formally).
- qbasic_forever 3y ago
  It's confabulating (or hallucinating) meaning. GPT has no understanding of anything. Given an input prompt (tell me what this made up language sentence means for example), it's reaching into its training set and stochastically parroting a response. There is no cognition or understanding of language.
  - stevenhuang 3y ago
    > it's reaching into its training set and stochastically parroting a response Yes, this is how humans work too. Also I hope the irony of trotting out this oft-repeated quip much like a stochastic parrot would isn't lost on you :P
    - qbasic_forever 3y ago
      No, humans don't work like your simplistic view of a neural network. I as a human can apply logic and deduction to figure out how something works despite never having experienced or learned about it directly for example. GPT cannot do this and can only guess or make up responses when faced with a problem that requires such reasoning.
      - stevenhuang 3y ago
        GPT cannot do this? Not even a bit? Well... that's definitely an opinion. A reasonable person would grant it _some_ level of reasoning ability, however flawed. To dismiss it all as 'pattern matching' rather shows some confused ideas about how cognition works, as if pattern matching plays no role in human cognition or intelligence. I'll understand difference in opinion if we're talking about more nebulous aspects like consciousness or qualia...
        qbasic_forever 3y ago
        > Well... that's definitely an opinion. A reasonable person would grant it _some_ level of reasoning ability, however flawed. No this is not an opinion, this is an objective fact about how deep learning and neural network models work period. You are confabulating capabilities onto them which they do not have. There's not 'some level of reasoning' in a neural network, there's _no reasoning_. You're being tricked by plausible sounding responses from something trained on an enormous corpus of internet BS (reddit posts, etc.). There is no intelligence or reasoning or logic inside GPT. Your human emotions (which GPT does not have) are clouding your judgement and making you think there is intelligence there which does not actually exist--you want it to be there so badly you'll invent reasons to confirm your views. If you asked GPT directly if it were intelligent or sentient it would not agree with you either, because it was not trained to do so.
RoyGBivCap 3y ago
[dead]

No, only knowledge that was on the internet and specifically on sites like Reddit that were crawled to train the model. This is definitely not all human knowledge!

aoeusnth1 3y ago

We don’t know how many books and academic papers were in the dataset.

qbasic_forever 3y ago
I can say with absolute certainty they did not include all books with all human knowledge. For example I suspect chat GPT has zero knowledge of how to speak native American languages that have effectively died with no remaining speakers and no complete written history of their exact usage.
- hammyhavoc 3y ago
  It also has massive gaps when it comes to problems when porting video games, concepts in game audio like procedural mixing, and even basic stuff like not generating a terribad NGINX config despite being fed the documentation. Anything niche, it's appalling. Anything you can otherwise Google? Sure.

csomar 3y ago

Strong disagree. Will be writing a blog post how GPT-4 understands languages that don't even exist (at least not formally).

qbasic_forever 3y ago
It's confabulating (or hallucinating) meaning. GPT has no understanding of anything. Given an input prompt (tell me what this made up language sentence means for example), it's reaching into its training set and stochastically parroting a response. There is no cognition or understanding of language.
- stevenhuang 3y ago
  > it's reaching into its training set and stochastically parroting a response Yes, this is how humans work too. Also I hope the irony of trotting out this oft-repeated quip much like a stochastic parrot would isn't lost on you :P
  - qbasic_forever 3y ago
    No, humans don't work like your simplistic view of a neural network. I as a human can apply logic and deduction to figure out how something works despite never having experienced or learned about it directly for example. GPT cannot do this and can only guess or make up responses when faced with a problem that requires such reasoning.
    - stevenhuang 3y ago
      GPT cannot do this? Not even a bit? Well... that's definitely an opinion. A reasonable person would grant it _some_ level of reasoning ability, however flawed. To dismiss it all as 'pattern matching' rather shows some confused ideas about how cognition works, as if pattern matching plays no role in human cognition or intelligence. I'll understand difference in opinion if we're talking about more nebulous aspects like consciousness or qualia...
      - qbasic_forever 3y ago
        > Well... that's definitely an opinion. A reasonable person would grant it _some_ level of reasoning ability, however flawed. No this is not an opinion, this is an objective fact about how deep learning and neural network models work period. You are confabulating capabilities onto them which they do not have. There's not 'some level of reasoning' in a neural network, there's _no reasoning_. You're being tricked by plausible sounding responses from something trained on an enormous corpus of internet BS (reddit posts, etc.). There is no intelligence or reasoning or logic inside GPT. Your human emotions (which GPT does not have) are clouding your judgement and making you think there is intelligence there which does not actually exist--you want it to be there so badly you'll invent reasons to confirm your views. If you asked GPT directly if it were intelligent or sentient it would not agree with you either, because it was not trained to do so.

RoyGBivCap 3y ago

[dead]