5 ms·
The transition from useless to useful ML models no doubt often seems magical to researchers. But it follows just from the distribution of the training data and
by mjburgess 4y ago
The transition from useless to useful ML models no doubt often seems magical to researchers. But it follows just from the distribution of the training data and from the degree of its compression by the function approximation algorithm they're using.
What's "magical" is not their system, but rather that the vast library of text they use for training has useful properties which can be approximated.
What researchers are observing is more like the illusion of a "phase transition" in the quality of approximations. This illusion arises because we have discontinuous standards for the approximations.
Ie., when assessing free text prediction by LLMs there's very very many ways for them to generate an acceptable answer. For mathematics, there's only one acceptable way.
If we applied the same standard/goal to both, no such apparent "quality transition" would occur. LLMs would be exposed as equally good, or equally bad, at prediction regardless of scale.
- xpe 4y agoInteresting arguments. It seems plausible and insightful. IMO, your analysis here deserves a longer write-up. Is it something you are working on?
- mjburgess 4y agoAny person with a "scientific attitude" in this field would find it incredibly easy to observer that the training target for natural language is, f(Q) = {A1..An} -- n being very very large and the target for mathematics is, g(Q) = A1 And the model they're using approximates with, m(Q) = A_guess So it's incredibly easy to model f with m because A_guess just has to be close to one of A1...n; and its very hard to model mathematics because it has to be only A1. The reason articles like this are written isn't because people dont know this; it's because they just do not have a sceptical attitude. And that's a problem I can't fix. If they'd approached this issue with the goal of "finding nothing surprising about this behaviour", ie., trying to make it maximally consistent with existing (basic, trivial, widely-taught) theory, they'd reach this conclusion in under 5min. The problem is their goal is always to find something surprising so they can release some PR about it. There's nothing I can write to fix this problem, it's edemic in the whole field. It makes ML/AI research much more like psychology than physics. Alas!