3 ms·
> With 5 finetunes you need to host 5 copies or load and unload them. If you use LoRA, which many do when fine-tuning nowadays, you don't need five full copies
by yberreby 3y ago
> With 5 finetunes you need to host 5 copies or load and unload them.
If you use LoRA, which many do when fine-tuning nowadays, you don't need five full copies. You only need to store adapters, which can be in the tens of MBs range for a given finetune.
- sanxiyn 3y agoYou can also batch requests using different LoRAs. See "S-LoRA: Serving Thousands of Concurrent LoRA Adapters". https://arxiv.org/abs/2311.03285 https://arxiv.org/abs/2311.03285