3 ms·
Can you point to any literature on this by any chance? I would be really interested to see some in depth analysis.
by anonymousDan 2y ago
Can you point to any literature on this by any chance? I would be really interested to see some in depth analysis.
- bradfox2 2y agoI don't have the arxiv link bookmarked, but there was a paper written on pretraining with Lora. It involved merging adapters back every n steps with good results.