3 ms·

Could lora fine tunes be used instead of completely different models? I wonder if that would save space.

• amilios 3y ago
  Yeah that would save disk space! In terms of inference, you'd still need to hold multiple models in memory though, and I don't think we're that close to that (yet) on personal devices. You could imagine a system that dynamically unloads and reloads the models as you need them in this process, but that unloading and reloading would be pretty slow probably.
  • ilaksh 3y ago
    https://github.com/predibase/lorax https://github.com/predibase/lorax does this, it's not that slow, since LoRAs aren't usually very big.
  • Kubuxu 3y ago
    With a fast NVME loading a model is only 2-3s.