3 ms·
I haven't done this with sentence transformers but I imagine it's possible since they can be loaded as regular transformers. Check out https://github.com/huggi
by batch12 3y ago
I haven't done this with sentence transformers but I imagine it's possible since they can be loaded as regular transformers.
Check out https://github.com/huggingface/peft https://github.com/huggingface/peft -- they've packaged it up nicely- and read up on LoRA (https://arxiv.org/pdf/2106.09685.pdf https://arxiv.org/pdf/2106.09685.pdf) That should get you started.
- leobg 3y agoThank you. Peft and adapters seem to be two different things though, no? AFAIK there are other libraries for adapters (forgot the name). Is peft what you were talking about when you said adapters in your original comment?
- batch12 3y agoI was of the understanding that Lora was one flavor of adapters, but I am still learning so I may be wrong. I yet gotten too deep into other transformer adapters yet (still reading).