4 ms·
This is the same approach as Mini-GPT4 of training a linear layer using frozen image encoder and text decoder. the difference is that they use CLIP instead of B
by ipsum2 4y ago
This is the same approach as Mini-GPT4 of training a linear layer using frozen image encoder and text decoder. the difference is that they use CLIP instead of BLIP, and LLaMA instead of Vicuna (which was trained on LLaMA. Interesting that they both came out at the same time.
- seydor 4y agoIt says they used Vicuna
- orpheansodality 4y agothe paper says llama: > We develop a large multimodal model (LMM), by connecting the open-set visual encoder of CLIP [36] with the language decoder LLaMA, and fine-tuning them end-to-end on our generated instructional vision-language data