This paper tries something like that https://arxiv.org/pdf/2202.13914.pdf https://arxiv.org/pdf/2202.13914.pdf The gain isn't that significant. We don't under

This paper tries something like that https://arxiv.org/pdf/2202.13914.pdf https://arxiv.org/pdf/2202.13914.pdf The gain isn't that significant. We don't understand what these low-rank updates represent, and they might not correspond to "skills" that humans have.