4 ms·

Is that true? I thought there was a desire to move towards byte level work rather than tokens, and that the benefits of tokens was more that you are reducing th

by IanCal 2y ago

Is that true? I thought there was a desire to move towards byte level work rather than tokens, and that the benefits of tokens was more that you are reducing the context size for the same input.

fngjdflmdflg 2y ago
>there was a desire to move towards byte level work rather than tokens Yeah, latest work on this is from Meta a last month.[0] It showed good results. [0] https://ai.meta.com/research/publications/byte-latent-transformer-patches-scale-better-than-tokens/ https://ai.meta.com/research/publications/byte-latent-transf... (https://news.ycombinator.com/item?id=42415122 https://news.ycombinator.com/item?id=42415122)