4 ms·

Can a model be fine-tuned “online”? If cost wasn’t an issue, could I fine-tune a model in real time, while also using it for inference?

by nico 4y ago

joshka 4y ago
Stats from https://replicate.com/blog/replicate-alpaca https://replicate.com/blog/replicate-alpaca suggest 90 minutes on 4 x A100/80GB. This is 3.5 hours on A100/40GB - I'm guessing LoRA is probably parallizable.
all2 4y ago
Stats from TFA say 3 hours to fine tune on an A100 processor.
- all2 4y ago
  I wonder if one could "fine tune" specific layer values... That might be faster than updating every weight in every layer.
  - nico 4y ago
    Or “multiplex” fine-tuning with inference, ie. do fine-tuning for 100ms, inference for 100ms, then tuning again… etc Btw, is there a way to combine two or more models? So for example, if I create 5 copies of a model, then fine-tune each copy with a different dataset -> can the 5 datasets be merged together somehow to create a model that has the learning of the 5?
  - nl 4y ago
    LoRA is an alternative to traditional fine tuning (which is usually done on specific layers as you mentioned). To quote the LoRA paper[1]: > We hypothesize that the change in weights during model adaptation also has a low “intrinsic rank”, leading to our proposed Low-Rank Adaptation (LoRA) approach. LoRA allows us to train some dense layers in a neural network indirectly by optimizing rank decomposition matrices of the dense layers’ change during adaptation instead, while keeping the pre-trained weights frozen It's truly revolutionary: It basically lets you create a very small "diff" which you apply yo an existing model and it is suddenly fine tuned. These diff models are very small (5M for example). [1] https://arxiv.org/abs/2106.09685 https://arxiv.org/abs/2106.09685
nl 4y ago
Yes and no. The training process is modifying the network weights. These are usually written to copies of the file instead of overwriting it (because what if the loss is actually worse after an epoch of training?) But there's nothing stopping inference from occurring on a model that is being trained.

Can a model be fine-tuned “online”? If cost wasn’t an issue, could I fine-tune a model in real time, while also using it for inference?

by nico 4y ago

Can a model be fine-tuned “online”? If cost wasn’t an issue, could I fine-tune a model in real time, while also using it for inference?

joshka 4y ago

Stats from https://replicate.com/blog/replicate-alpaca https://replicate.com/blog/replicate-alpaca suggest 90 minutes on 4 x A100/80GB. This is 3.5 hours on A100/40GB - I'm guessing LoRA is probably parallizable.

all2 4y ago

Stats from TFA say 3 hours to fine tune on an A100 processor.

all2 4y ago
I wonder if one could "fine tune" specific layer values... That might be faster than updating every weight in every layer.
- nico 4y ago
  Or “multiplex” fine-tuning with inference, ie. do fine-tuning for 100ms, inference for 100ms, then tuning again… etc Btw, is there a way to combine two or more models? So for example, if I create 5 copies of a model, then fine-tune each copy with a different dataset -> can the 5 datasets be merged together somehow to create a model that has the learning of the 5?
- nl 4y ago
  LoRA is an alternative to traditional fine tuning (which is usually done on specific layers as you mentioned). To quote the LoRA paper[1]: > We hypothesize that the change in weights during model adaptation also has a low “intrinsic rank”, leading to our proposed Low-Rank Adaptation (LoRA) approach. LoRA allows us to train some dense layers in a neural network indirectly by optimizing rank decomposition matrices of the dense layers’ change during adaptation instead, while keeping the pre-trained weights frozen It's truly revolutionary: It basically lets you create a very small "diff" which you apply yo an existing model and it is suddenly fine tuned. These diff models are very small (5M for example). [1] https://arxiv.org/abs/2106.09685 https://arxiv.org/abs/2106.09685

nl 4y ago

Yes and no. The training process is modifying the network weights. These are usually written to copies of the file instead of overwriting it (because what if the loss is actually worse after an epoch of training?) But there's nothing stopping inference from occurring on a model that is being trained.