3 ms·

You’re disregarding the emergent phenomena, which are not at all understood. There was a distinct and unpredicted jump in what can loosely be described as “cog

You’re disregarding the emergent phenomena, which are not at all understood. There was a distinct and unpredicted jump in what can loosely be described as “cognitive abilities” between GPTs 2, 3, and 4, especially after some supervised techniques like RLHF.

mjburgess 3y ago
There is no "emergent phenomena" the pattern described is just the same as when you add +b to an ax+b model of linear data. ie., it's just fitting capacity. The "emergent boundary" is just an empirical measure of the necessary fitting capacity of these models on "everything ever digitised in english" given any particular functional requirement. All the language around this area is not scientific, nor are these practices. This is superstitious neophyte engineers, hopped up on scifi, by giddy VCs who love to be told they're funding captin picard.
- williamcotton 3y ago
  Well hurry up and get your paper published because if you've cracked the code on emergent abilities the world is looking for answers!
  - deleted 3y ago
    [deleted]
  - mjburgess 3y ago
    The triviality of these observations doesn't rise to the level of getting a paper published. You can resolve all of this hype by reading the intro chapters of any applied stats textbooks. I've been to many academic conferences, and the fresh PhDs who pump out this BS are not, err, very credible seeming people. Yes, they're young and naive, and really desperate to make their career impactful -- etc. But they're also not really the kinds of smart sceptics you'd hope for. Many are, though they tend to exit at masters level and go make money. The academic publishing environment today is far far far away from epistemically hygienic, indeed, in these areas i could only imagine how insane credible sceptical empirical types would be. Could you imagine being surrounded by this desperate need to call 'useless correlations' "hallucinations", and to call "model fitting", 'emergence' ? I find it disquieting at a removed distance from it -- I'd be quite mentally ill within it. I don't do this cultish game playing. And I wrote exactly the same here on AI, crytop and the reset of it many years ago when it would be downvoted; and the same today now it's upvoted. This is the degree i'm inclined to make public contributions on these matters. I haven't the temperament to handle doe-eyed PhDs repeating universal function fitting theorems and turing-equivalence talking points without understanding a single iota of applied statistics, scientific epistemology, basic methdological scepticism etc. Unless I am hired to do so, which occasionally I am. But thankfully in actual corp environments the people I meet are far more credible and sceptical than those writing hype papers
    - williamcotton 3y ago
      I’m pretty familiar with statistics and a range of machine learning techniques from before and after the NN revolution. I’ve applied the techniques plenty of times in production environments. Whatever sort of internal structures that are being formed during the training process is somewhat evident when looking at the structure of a CNN… edge detect kernels emerge, etc. Whatever sort of internal structures formed during the training of transformer based architectures are basically unknown at this point. There’s your thesis, have at it! You won’t be ignored for your discoveries. Hell, send me a copy and I’ll make sure anyone who matters at UT or Baylor sees a copy!
      - mjburgess 3y ago
        Ah good. Well, then if you're interested in a half-empirical sincere attempt to characterise "why certain weights obtain certain values under optimisation" then i'm much more inclined to be, say, more humble on these matters. The reason CNN weights obtain 'recursive-hierarchical representations' of pixel-pattern geometry in the training data follow from the recusrive-heirachical relationship of their weight matrices and from the geometry of the 'pixel space' from which the training data is drawn. This is certainly interesting; and there's something magical feeling about 'principles of least action' at work. Indeed, many new physicists have a kind of schizophrenic reaction to discovering action principles -- since it imparts to nature a strange apparent conspiracy. Of course, the job of any good physicist is to be sceptical of this conspiracy, and to get to the heart of how 'accounting tricks' performed by moving objects over time create this illusion. Likewise this is the job of any good ML researcher; yet they do the oppoiste. Rather than get to the heart of this apparent conspiracy, they call it 'emergence' -- this offends my sense of what the virtues of a scientist ought be. In any case, on the matter of the LLMs obtaining 'useful' weights for any given task here the job of the researcher is circumstantial, empirical, and sceptical: go and find those 'accounting tricks' within the training data that give rise to this apparent conspiracy of the system to acquire a useful state. There is no emergence: there is just a set of weights which compress the structure of a target space. At some point this set is large enough, and 'lies across the space like a mental chain does a gate'. Emergence is an ontological relation between parts and wholes whereby wholes arent reducible to their parts because of ontologically-relevant interaction properties between their parts which aren't intrinsic properties of them. The fluidity of water emerges out of hydrogen bonding which does not occur when you isolate H20 alone. There is no such relationship here. This ontologising of the formal, this language which gives a causal-physical semantics to purely formal properties of abstract models -- this is pseudoscience. It's done as part of a computational-idealist worldview in vogue because it's a helpful language for VC investment were-changing-the-world hype. The formal properties of NNs cannot be described in these terms, because they do not have ontological relationship -- they have formal (mathematical, statistica, etc.) ones.
  - intended 3y ago
    This paper was published, and the distinct jump was found to be a measurement artifact. https://hai.stanford.edu/news/ais-ostensible-emergent-abilities-are-mirage https://hai.stanford.edu/news/ais-ostensible-emergent-abilit...
    - williamcotton 3y ago
      Sweet, now complete and present a meta-analysis of all such papers about the topic and maybe I’ll find it more convincing than a cherry-picked publication that supports your preexisting position on the topic!
      - intended 3y ago
        I cherry picked a Stanford article and professor… those Stanford liars! Curse them! and silly me! You are absolutely right, research should have a meta analysis ready of papers that came out… 6 months ago. Absolutely ridiculous. I am sure you have read this paper, and have an army of meta analyses and counter claims for this nascent sub field.
      - scotty79 3y ago
        Metanalysis of bunch of weak papers is worth nothing when confronted with one strong study. I hoped covid drug research has tought us all that.
- kristiandupont 3y ago
  >There is no "emergent phenomena" Yes there is, that's all there is.

mjburgess 3y ago

There is no "emergent phenomena" the pattern described is just the same as when you add +b to an ax+b model of linear data. ie., it's just fitting capacity. The "emergent boundary" is just an empirical measure of the necessary fitting capacity of these models on "everything ever digitised in english" given any particular functional requirement. All the language around this area is not scientific, nor are these practices. This is superstitious neophyte engineers, hopped up on scifi, by giddy VCs who love to be told they're funding captin picard.

williamcotton 3y ago
Well hurry up and get your paper published because if you've cracked the code on emergent abilities the world is looking for answers!
- deleted 3y ago
  [deleted]
- mjburgess 3y ago
  The triviality of these observations doesn't rise to the level of getting a paper published. You can resolve all of this hype by reading the intro chapters of any applied stats textbooks. I've been to many academic conferences, and the fresh PhDs who pump out this BS are not, err, very credible seeming people. Yes, they're young and naive, and really desperate to make their career impactful -- etc. But they're also not really the kinds of smart sceptics you'd hope for. Many are, though they tend to exit at masters level and go make money. The academic publishing environment today is far far far away from epistemically hygienic, indeed, in these areas i could only imagine how insane credible sceptical empirical types would be. Could you imagine being surrounded by this desperate need to call 'useless correlations' "hallucinations", and to call "model fitting", 'emergence' ? I find it disquieting at a removed distance from it -- I'd be quite mentally ill within it. I don't do this cultish game playing. And I wrote exactly the same here on AI, crytop and the reset of it many years ago when it would be downvoted; and the same today now it's upvoted. This is the degree i'm inclined to make public contributions on these matters. I haven't the temperament to handle doe-eyed PhDs repeating universal function fitting theorems and turing-equivalence talking points without understanding a single iota of applied statistics, scientific epistemology, basic methdological scepticism etc. Unless I am hired to do so, which occasionally I am. But thankfully in actual corp environments the people I meet are far more credible and sceptical than those writing hype papers
  - williamcotton 3y ago
    I’m pretty familiar with statistics and a range of machine learning techniques from before and after the NN revolution. I’ve applied the techniques plenty of times in production environments. Whatever sort of internal structures that are being formed during the training process is somewhat evident when looking at the structure of a CNN… edge detect kernels emerge, etc. Whatever sort of internal structures formed during the training of transformer based architectures are basically unknown at this point. There’s your thesis, have at it! You won’t be ignored for your discoveries. Hell, send me a copy and I’ll make sure anyone who matters at UT or Baylor sees a copy!
    - mjburgess 3y ago
      Ah good. Well, then if you're interested in a half-empirical sincere attempt to characterise "why certain weights obtain certain values under optimisation" then i'm much more inclined to be, say, more humble on these matters. The reason CNN weights obtain 'recursive-hierarchical representations' of pixel-pattern geometry in the training data follow from the recusrive-heirachical relationship of their weight matrices and from the geometry of the 'pixel space' from which the training data is drawn. This is certainly interesting; and there's something magical feeling about 'principles of least action' at work. Indeed, many new physicists have a kind of schizophrenic reaction to discovering action principles -- since it imparts to nature a strange apparent conspiracy. Of course, the job of any good physicist is to be sceptical of this conspiracy, and to get to the heart of how 'accounting tricks' performed by moving objects over time create this illusion. Likewise this is the job of any good ML researcher; yet they do the oppoiste. Rather than get to the heart of this apparent conspiracy, they call it 'emergence' -- this offends my sense of what the virtues of a scientist ought be. In any case, on the matter of the LLMs obtaining 'useful' weights for any given task here the job of the researcher is circumstantial, empirical, and sceptical: go and find those 'accounting tricks' within the training data that give rise to this apparent conspiracy of the system to acquire a useful state. There is no emergence: there is just a set of weights which compress the structure of a target space. At some point this set is large enough, and 'lies across the space like a mental chain does a gate'. Emergence is an ontological relation between parts and wholes whereby wholes arent reducible to their parts because of ontologically-relevant interaction properties between their parts which aren't intrinsic properties of them. The fluidity of water emerges out of hydrogen bonding which does not occur when you isolate H20 alone. There is no such relationship here. This ontologising of the formal, this language which gives a causal-physical semantics to purely formal properties of abstract models -- this is pseudoscience. It's done as part of a computational-idealist worldview in vogue because it's a helpful language for VC investment were-changing-the-world hype. The formal properties of NNs cannot be described in these terms, because they do not have ontological relationship -- they have formal (mathematical, statistica, etc.) ones.
- intended 3y ago
  This paper was published, and the distinct jump was found to be a measurement artifact. https://hai.stanford.edu/news/ais-ostensible-emergent-abilities-are-mirage https://hai.stanford.edu/news/ais-ostensible-emergent-abilit...
  - williamcotton 3y ago
    Sweet, now complete and present a meta-analysis of all such papers about the topic and maybe I’ll find it more convincing than a cherry-picked publication that supports your preexisting position on the topic!
    - intended 3y ago
      I cherry picked a Stanford article and professor… those Stanford liars! Curse them! and silly me! You are absolutely right, research should have a meta analysis ready of papers that came out… 6 months ago. Absolutely ridiculous. I am sure you have read this paper, and have an army of meta analyses and counter claims for this nascent sub field.
    - scotty79 3y ago
      Metanalysis of bunch of weak papers is worth nothing when confronted with one strong study. I hoped covid drug research has tought us all that.
kristiandupont 3y ago
>There is no "emergent phenomena" Yes there is, that's all there is.