3 ms·
Isn’t “sentence prediction” roughly the same as multi token prediction of sufficient length? In the end are we just talking about a change to hyper parameters o
by blackeyeblitzar 2y ago
Isn’t “sentence prediction” roughly the same as multi token prediction of sufficient length? In the end are we just talking about a change to hyper parameters or maybe a new hyper parameter that controls the granularity of “prediction length”?
- mdp2021 2y ago> multi token prediction of sufficient length Is multi token prediction the same as predicting the embedding of a complex token (the articulation of those input tokens in a sentence)?
- blackeyeblitzar 2y agoTo be honest I don’t know. Maybe the only way to know is to build and measure all these variations.