5 ms·
I'm curious when we'll start to see open access multimodal models being released. The advancement in text only models has been amazing, but a lot of the 'emerg
by kromem 3y ago
I'm curious when we'll start to see open access multimodal models being released.
The advancement in text only models has been amazing, but a lot of the 'emergent' behavior in GPT-4 may be because of multimodal training and not just MoE or parameter sizes.
I'll be curious to see if multimodal smaller models see similar leaps.
- ijustlovemath 3y agoI've heard that Google actually got the jump on OpenAI in this regard (just from people in a FAANG), and they're playing a bit of catch-up. OpenAI still has a distinct advantage on the language side, though. This is all hearsay, of course.
- minimaxir 3y agoLLaVA is open, although not the leap you are expecting: https://llava-vl.github.io/ https://llava-vl.github.io/ Meta also released a (non-commercial) multimodal model among 6 modalities: https://ai.meta.com/blog/imagebind-six-modalities-binding-ai/ https://ai.meta.com/blog/imagebind-six-modalities-binding-ai...
- coder543 3y agoCogVLM is very good in my (brief) testing: https://github.com/THUDM/CogVLM https://github.com/THUDM/CogVLM The model weights seem to be under a non-commercial license, not true open source, but it is "open access" as you requested. It would be nice if someone trained a CogVLM-compatible model from scratch under an open source license.
- beoberha 3y agoWhen I tried last, it couldn’t be run on M-series Macs.