4 ms·
An overfit LLM would probably regurgitate common phrases from the training dataset and struggle to output phrases which didn't occur in that dataset. I'm not r
by fancyfredbot 2y ago
An overfit LLM would probably regurgitate common phrases from the training dataset and struggle to output phrases which didn't occur in that dataset.
I'm not really sure what that looks like when your training data is so huge. Might not be very noticeable?
- seydor 2y agoalso, an LLM is not learning phrases per se but the relationships between tokens