2 ms·
So is it analogous to how a CNN starts with fragments of images and further up the chain assembles these into objects?
by lawlessone 3y ago
So is it analogous to how a CNN starts with fragments of images and further up the chain assembles these into objects?
- ma2rten 3y agoYes, I think that is a reasonable way to think about it, in my opinion. However, with the language modeling objective it predicts the next token and because of the residual connections each intermediate layer is in the same space. So, maybe it would be more accurate to say that it is an increasingly accurate representation of the next token.