3 ms·
> That is how they can also handle input in audio or visual forms because there is an audio or visual tokenizer. This is incorrect. They get translated into th
by viraptor 2y ago
> That is how they can also handle input in audio or visual forms because there is an audio or visual tokenizer.
This is incorrect. They get translated into the shared latent space, but they're not tokenized in any way resembling the text part.
- pseudosavant 2y agoThey are almost certainly tokenized in most LLM multi-modal models. https://en.wikipedia.org/wiki/Large_language_model#Multimodality https://en.wikipedia.org/wiki/Large_language_model#Multimoda...
- viraptor 2y agoAh, an overloaded "tokenizer" meaning. "split into tokens" vs "turned into a single embedding matching a token" I've never heard it used that way before, but it makes sense kinda.