4 ms·
Wasn't there some paper recently that showed over training models, well beyond when it is normally halted, led them to create internal generalised models of a s
by MattPalmer1086 2y ago
Wasn't there some paper recently that showed over training models, well beyond when it is normally halted, led them to create internal generalised models of a subject, e.g. arithmetic?
Essentially, the model internalised the core concepts of arithmetic. In that sense, the "reasoning" is pre baked into the model by training. Inference just plays things back through that space.
EDIT: as I recall, this is because understanding the concepts provides better compression than remembering lots of examples. It just takes a lot more training before it discovers them.
- quantadev 2y agoI don't like the analogy of "compression" that much, because for example if you train a model to predict linear data points, ideally it will only end up knowing two numbers in it's model weights when it's done training: "m" and "b" in "Y=Mx+b". Once it's successfully captured "m" and "b" it has "knowledge" with which it can predict infinite numbers of points correctly, and hopefully it didn't "compress" any of the examples but discarded all of them.
- MattPalmer1086 2y agoYeah, it's not compression in the sense of compressing data. Kind of compression in that it takes less resource to encode general rules than to remember the answer for everything. The paper said was that the most efficient bits of the network were those that encoded rules rather than remembered data. Somehow those bits gradually took over from the less efficient parts. I'll have to dig around, can't seem to find it right now.
- quantadev 2y agoI agree with that.