3 ms·
Related is this one https://arxiv.org/abs/1912.02757 https://arxiv.org/abs/1912.02757 which as I understand finds that there's little to be gained moving around
by version_five 4y ago
Related is this one https://arxiv.org/abs/1912.02757 https://arxiv.org/abs/1912.02757 which as I understand finds that there's little to be gained moving around locally in weight space, vs finding another mode. So it's not surprising that a model positioned between two modes would function sort of like an ensemble model
- xpe 4y agoWhich of course raises the question of quantifying when averaging models (or, more generally, staying within a convex hull) is likely to remain in the same low cost (high performing) basin.
- LChoshen 4y agoAll related, but why do any of it changes things about fusing models of "different" training, which is more interesting imo