3 ms·

Well said. How did the authors and reviewers miss this?

• mjburgess 4y ago
  The transition from useless to useful ML models no doubt often seems magical to researchers. But it follows just from the distribution of the training data and from the degree of its compression by the function approximation algorithm they're using. What's "magical" is not their system, but rather that the vast library of text they use for training has useful properties which can be approximated. What researchers are observing is more like the illusion of a "phase transition" in the quality of approximations. This illusion arises because we have discontinuous standards for the approximations. Ie., when assessing free text prediction by LLMs there's very very many ways for them to generate an acceptable answer. For mathematics, there's only one acceptable way. If we applied the same standard/goal to both, no such apparent "quality transition" would occur. LLMs would be exposed as equally good, or equally bad, at prediction regardless of scale.
  • xpe 4y ago
    Interesting arguments. It seems plausible and insightful. IMO, your analysis here deserves a longer write-up. Is it something you are working on?
    • mjburgess 4y ago
      Any person with a "scientific attitude" in this field would find it incredibly easy to observer that the training target for natural language is, f(Q) = {A1..An} -- n being very very large and the target for mathematics is, g(Q) = A1 And the model they're using approximates with, m(Q) = A_guess So it's incredibly easy to model f with m because A_guess just has to be close to one of A1...n; and its very hard to model mathematics because it has to be only A1. The reason articles like this are written isn't because people dont know this; it's because they just do not have a sceptical attitude. And that's a problem I can't fix. If they'd approached this issue with the goal of "finding nothing surprising about this behaviour", ie., trying to make it maximally consistent with existing (basic, trivial, widely-taught) theory, they'd reach this conclusion in under 5min. The problem is their goal is always to find something surprising so they can release some PR about it. There's nothing I can write to fix this problem, it's edemic in the whole field. It makes ML/AI research much more like psychology than physics. Alas!
• baandang 4y ago
  From Introduction To The Theory Of Complex Systems: • Complex systems can exhibit a rich phase structure and have a huge variety of macrostates that often cannot be inferred from the properties of the elements. This is sometimes referred to as emergence. This is the term and as common a term in complex systems as there is. "scale-dependent capabilities" implies inference from elements. I think some people just don't like the very idea even though it is not unlike the concept of stochastic process. I would think the same reasons to not like the concept of emergence applies to stochastic process. Murray Gell-Mann though couldn't even raise complex systems above arguing if emergence is magical thinking so it is probably a lost cause. Such an interesting field that always ends up as this conversation.
  • naasking 4y ago
    > "scale-dependent capabilities" implies inference from elements. Don't confuse the post-hoc explanation with an a priori inference. We can post-hoc explain water's emergent liquidity property using modern quantum theories, but that doesn't mean we could have inferred it if given Schrodinger's equation and the atomic structure of hydrogen and oxygen. "Scale-dependent capability" is a post-hoc explanation that of course looks obvious in hindsight, just like liquidity and pressure looks obvious in hindsight once you understand electromagnetism and atomic theory.