3 ms·
It’s dead wrong, I suggest reading the original. _No one_ thinks models trained on more data would be about the same, since everything flows out of that premise
by refulgentis 3y ago
It’s dead wrong, I suggest reading the original. _No one_ thinks models trained on more data would be about the same, since everything flows out of that premise…then throws Current Thing on top…it’s very unhelpful.
The “patches” he refers to are LoRA and are treated as deus ex machina. They’re not, ex. playing with Stable Diffusion we can see they’re additive but they’re not nearly as good as training the original model on the data.
(Disclaimer: Googler)
- haldujai 3y ago> they’re not nearly as good as training the original model on the data. How are you defining "nearly as good", can you be more specific? It's obvious full fine-tuning > [PEFT tuning] but to my understanding the gap isn't that significant, as reported in various papers. (specifically with respects to language models, I'm not familiar with diffusion models).