3 ms·

Git Re-Basin: Merging Models Modulo Permutation Symmetries

• caprock 4y ago
  > We show that NN loss landscapes contain effectively only a single basin(!) provided sufficient width. Even better, we develop practical algos to navigate these basins... Supporting twitter thread: https://twitter.com/SamuelAinsworth/status/1569719494645526529 https://twitter.com/SamuelAinsworth/status/15697194946455265... Paper: https://arxiv.org/abs/2209.04836 https://arxiv.org/abs/2209.04836