2 ms·

Yeah, good idea! I'm not sure if you would be successful mixing LoRA + functional tokens. If you could, that would be great. Then You could ship very light LoRA

Yeah, good idea! I'm not sure if you would be successful mixing LoRA + functional tokens. If you could, that would be great. Then You could ship very light LoRA packs with repositories. Their LoRA training was I think against their finetuned model, not Gemma-2B directly. But, seems worth playing with -- could be super useful.

AlexChen-NexaAI 3y ago
Yes, you can do that! We have experimented about this idea. The lora training is reported in the paper already, and it works well. I think it would be very interesting.