3 ms·
Technically true, but when those tokens are 1:1 mapped to text, I think we can simplify this down without losing anything important. Of course, once you start
by int_19h 3y ago
Technically true, but when those tokens are 1:1 mapped to text, I think we can simplify this down without losing anything important.
Of course, once you start using tokens for other things - as multimodal LMs already do - that changes. But this current crop of model still has visual modality in its infancy IMO, and gauging the overall performance of model as a whole based on that is very questionable.
- dragonwriter 3y ago> Technically true, but when those tokens are 1:1 mapped to text I don't know what GPT-4V does particularly, but my understanding is that multimodal models very often have an expanded token space with special tokens related to image handling, so, literally, there is not a 1:1 relationship of tokens to text.