4 ms·
It is not typically possible to blend models like that, since the training process is (lateral) order insensitive, as far as the model goes.
by eoerl 4y ago
It is not typically possible to blend models like that, since the training process is (lateral) order insensitive, as far as the model goes.
- ShamelessC 4y agoIs that still the case when all models have a common ancestor (i.e. finetuned) and haven’t yet overfit on new data?
- liuliu 4y agoI thought so too until found that there are quite a bit of literatures nowadays about "merging" weights, for example, this one: https://arxiv.org/pdf/1811.10515.pdf https://arxiv.org/pdf/1811.10515.pdf and also the OpenCLIP paper.