4 ms·
Seems like we should just use gradual annealing of tokens to more fine grained single character tokens over the course of training then
by pizza 2y ago
Seems like we should just use gradual annealing of tokens to more fine grained single character tokens over the course of training then
- kevmo314 2y agoI believe that's similar to the idea behind https://github.com/facebookresearch/blt https://github.com/facebookresearch/blt