3 ms·
Seems intuitively sound; a larger model would have the ability to differentiate among a larger variety of concepts, which translates to a larger vocabulary and
by cadamsdotcom 6mo ago
Seems intuitively sound; a larger model would have the ability to differentiate among a larger variety of concepts, which translates to a larger vocabulary and greater ability to use expressive tools such as imagery, metaphor etc etc.
I could go on, but brevity is virtuous.
- yorwba 6mo agoNone of this has anything to do with the paper, which is concerned with theoretical computer science and constructs artificial "languages" that have a small representation as a(n idealized theoretical) transformer but whose smallest representation in some other formalisms is much larger. In other words, its conception of succinctness is almost diametrically opposite of the way you appear to have understood it. They looked for small models that produce gigantic but meaningless outputs, not large models that produce short, meaningful text.
- cadamsdotcom 6mo agoHad another read - you’re absolutely right, thanks for the kind correction and explanation.