3 ms·
> EDIT: Another way to think of this model is to say “can we exfiltrate multi-model training data from open weights / closed training models?” And I think there
by triyambakam 2y ago
> EDIT: Another way to think of this model is to say “can we exfiltrate multi-model training data from open weights / closed training models?” And I think there the answer is yes.
Could you elaborate on this? Do you mean to say that this work reveals some of LLaMa's training data, or that this can be used to generate training data for new models?
- vessenes 2y agoSince they're using LLama with Llava, they're getting lots of Llama's baked-in understanding of the world. And I believe that Llama had multimodal training data as part of its pipeline, whether or not we get access to the heads that parse that data in the open weights. So, I'm saying that Llama has some ability to describe / complete captions of photos, and very probably that ability comes from some multimodal training done at FB, and that using it in this way to train something new that beats benchmarks is a form of pulling out the value of that training data. p.s. In the original comment, I meant to type 'multimodal' and didn't, or got autocorrected and didn't notice - apologies