4 ms·

> The “statistical parrot” assertion is pretty thoroughly disproven by this point errr... all NNs are just optimisations of an associative probability objectiv

> The “statistical parrot” assertion is pretty thoroughly disproven by this point errr... all NNs are just optimisations of an associative probability objective: P(Y|X), they are by definition "statistical parrots". There isn't anything to prove or disprove. People offering prompts as evidence are people who fundamentally do not understand the basics. NNs aren't strange empirical objects, they're specified by mathematical rules whose properties are known ahead of time. Any property of a trained NN is derivative of a property of a formula P(Answer|Prompt, TrainingData) This is an associative statistical relation, which by definition, selects elements of TrainingData by-association with the Prompt. If the basis by which you understand LLMs is putting prompts into ChatGPT you're severely underqualified for drawing any conclusions about LLMs, and radically subject to confirmation bias.

kristiandupont 3y ago
Well sure, but I think that same thing can be said about the human brain. Obviously at a whole different level of sophistication and all, but the two really do seem related, at least to me. >People offering prompts as evidence are people who fundamentally do not understand the basics This I agree with, but I also don't see anyone doing that in this thread?
williamcotton 3y ago
You’re disregarding the emergent phenomena, which are not at all understood. There was a distinct and unpredicted jump in what can loosely be described as “cognitive abilities” between GPTs 2, 3, and 4, especially after some supervised techniques like RLHF.
- mjburgess 3y ago
  There is no "emergent phenomena" the pattern described is just the same as when you add +b to an ax+b model of linear data. ie., it's just fitting capacity. The "emergent boundary" is just an empirical measure of the necessary fitting capacity of these models on "everything ever digitised in english" given any particular functional requirement. All the language around this area is not scientific, nor are these practices. This is superstitious neophyte engineers, hopped up on scifi, by giddy VCs who love to be told they're funding captin picard.
  - williamcotton 3y ago
    Well hurry up and get your paper published because if you've cracked the code on emergent abilities the world is looking for answers!
    - deleted 3y ago
      [deleted]
    - mjburgess 3y ago
      The triviality of these observations doesn't rise to the level of getting a paper published. You can resolve all of this hype by reading the intro chapters of any applied stats textbooks. I've been to many academic conferences, and the fresh PhDs who pump out this BS are not, err, very credible seeming people. Yes, they're young and naive, and really desperate to make their career impactful -- etc. But they're also not really the kinds of smart sceptics you'd hope for. Many are, though they tend to exit at masters level and go make money. The academic publishing environment today is far far far away from epistemically hygienic, indeed, in these areas i could only imagine how insane credible sceptical empirical types would be. Could you imagine being surrounded by this desperate need to call 'useless correlations' "hallucinations", and to call "model fitting", 'emergence' ? I find it disquieting at a removed distance from it -- I'd be quite mentally ill within it. I don't do this cultish game playing. And I wrote exactly the same here on AI, crytop and the reset of it many years ago when it would be downvoted; and the same today now it's upvoted. This is the degree i'm inclined to make public contributions on these matters. I haven't the temperament to handle doe-eyed PhDs repeating universal function fitting theorems and turing-equivalence talking points without understanding a single iota of applied statistics, scientific epistemology, basic methdological scepticism etc. Unless I am hired to do so, which occasionally I am. But thankfully in actual corp environments the people I meet are far more credible and sceptical than those writing hype papers
      - williamcotton 3y ago
        I’m pretty familiar with statistics and a range of machine learning techniques from before and after the NN revolution. I’ve applied the techniques plenty of times in production environments. Whatever sort of internal structures that are being formed during the training process is somewhat evident when looking at the structure of a CNN… edge detect kernels emerge, etc. Whatever sort of internal structures formed during the training of transformer based architectures are basically unknown at this point. There’s your thesis, have at it! You won’t be ignored for your discoveries. Hell, send me a copy and I’ll make sure anyone who matters at UT or Baylor sees a copy!
berndi 3y ago
You’re confused about what “statistical parrot” means and you don’t seem to understand the difference between an optimization objective and the resulting model. The term “parrot” is used to imply inference by something akin to a look-up table, specifically it is used to indicate poor out-of-sample performance and a lack of a proper world model. The optimization objective is irrelevant when determining the generalization performance of a model and when judging whether it can reason beyond looking up answers in a table. As the user above noted, it is now quite well established that GPT-4 has impressive out-of-sample performance which can be explained by it possessing an actual model of the world and not being a “parrot”.
- intended 3y ago
  That out of sample performance is a mirage. Yes it’s impressive. Yes it’s got amazing zero shot performance in domains. But there’s a pattern of failure in production which describe a limit, that shouldn’t exist if the emergent properties were stable. You can build this right now and test it. Build a sequence of agents to work on a domain you are not an expert in. Let them loose. See what happens. Do the same thing on a domain you have expertise in. Assume the number of errors you find, the number of modifications you have to make are stable for other domains.
  - ethbr1 3y ago
    I'd phrase characterizing the reliability of out-of-sample performance a priori as impossible, but not necessarily automatically failing. There may be a subtle correlation between properties needed to answer a specific out-of-sample request and in-sample features. Unfortunately, prior to training/testing and without recognizing that correlation in the data set, I believe it's impossible to guarantee the model will include it. (Corrections welcome)
    - intended 3y ago
      In essence: “You cant know in advance how far the model can approximate semantic patterns” So claiming that out-of-sample performance is a mirage, would be a bridge too far?

kristiandupont 3y ago

Well sure, but I think that same thing can be said about the human brain. Obviously at a whole different level of sophistication and all, but the two really do seem related, at least to me. >People offering prompts as evidence are people who fundamentally do not understand the basics This I agree with, but I also don't see anyone doing that in this thread?

williamcotton 3y ago

You’re disregarding the emergent phenomena, which are not at all understood. There was a distinct and unpredicted jump in what can loosely be described as “cognitive abilities” between GPTs 2, 3, and 4, especially after some supervised techniques like RLHF.

mjburgess 3y ago
There is no "emergent phenomena" the pattern described is just the same as when you add +b to an ax+b model of linear data. ie., it's just fitting capacity. The "emergent boundary" is just an empirical measure of the necessary fitting capacity of these models on "everything ever digitised in english" given any particular functional requirement. All the language around this area is not scientific, nor are these practices. This is superstitious neophyte engineers, hopped up on scifi, by giddy VCs who love to be told they're funding captin picard.
- williamcotton 3y ago
  Well hurry up and get your paper published because if you've cracked the code on emergent abilities the world is looking for answers!
  - deleted 3y ago
    [deleted]
  - mjburgess 3y ago
    The triviality of these observations doesn't rise to the level of getting a paper published. You can resolve all of this hype by reading the intro chapters of any applied stats textbooks. I've been to many academic conferences, and the fresh PhDs who pump out this BS are not, err, very credible seeming people. Yes, they're young and naive, and really desperate to make their career impactful -- etc. But they're also not really the kinds of smart sceptics you'd hope for. Many are, though they tend to exit at masters level and go make money. The academic publishing environment today is far far far away from epistemically hygienic, indeed, in these areas i could only imagine how insane credible sceptical empirical types would be. Could you imagine being surrounded by this desperate need to call 'useless correlations' "hallucinations", and to call "model fitting", 'emergence' ? I find it disquieting at a removed distance from it -- I'd be quite mentally ill within it. I don't do this cultish game playing. And I wrote exactly the same here on AI, crytop and the reset of it many years ago when it would be downvoted; and the same today now it's upvoted. This is the degree i'm inclined to make public contributions on these matters. I haven't the temperament to handle doe-eyed PhDs repeating universal function fitting theorems and turing-equivalence talking points without understanding a single iota of applied statistics, scientific epistemology, basic methdological scepticism etc. Unless I am hired to do so, which occasionally I am. But thankfully in actual corp environments the people I meet are far more credible and sceptical than those writing hype papers
    - williamcotton 3y ago
      I’m pretty familiar with statistics and a range of machine learning techniques from before and after the NN revolution. I’ve applied the techniques plenty of times in production environments. Whatever sort of internal structures that are being formed during the training process is somewhat evident when looking at the structure of a CNN… edge detect kernels emerge, etc. Whatever sort of internal structures formed during the training of transformer based architectures are basically unknown at this point. There’s your thesis, have at it! You won’t be ignored for your discoveries. Hell, send me a copy and I’ll make sure anyone who matters at UT or Baylor sees a copy!

berndi 3y ago

You’re confused about what “statistical parrot” means and you don’t seem to understand the difference between an optimization objective and the resulting model. The term “parrot” is used to imply inference by something akin to a look-up table, specifically it is used to indicate poor out-of-sample performance and a lack of a proper world model. The optimization objective is irrelevant when determining the generalization performance of a model and when judging whether it can reason beyond looking up answers in a table. As the user above noted, it is now quite well established that GPT-4 has impressive out-of-sample performance which can be explained by it possessing an actual model of the world and not being a “parrot”.

intended 3y ago
That out of sample performance is a mirage. Yes it’s impressive. Yes it’s got amazing zero shot performance in domains. But there’s a pattern of failure in production which describe a limit, that shouldn’t exist if the emergent properties were stable. You can build this right now and test it. Build a sequence of agents to work on a domain you are not an expert in. Let them loose. See what happens. Do the same thing on a domain you have expertise in. Assume the number of errors you find, the number of modifications you have to make are stable for other domains.
- ethbr1 3y ago
  I'd phrase characterizing the reliability of out-of-sample performance a priori as impossible, but not necessarily automatically failing. There may be a subtle correlation between properties needed to answer a specific out-of-sample request and in-sample features. Unfortunately, prior to training/testing and without recognizing that correlation in the data set, I believe it's impossible to guarantee the model will include it. (Corrections welcome)
  - intended 3y ago
    In essence: “You cant know in advance how far the model can approximate semantic patterns” So claiming that out-of-sample performance is a mirage, would be a bridge too far?