3 ms·
I appreciate that. As it happens, I was thinking more for LLMs, but that is a useful link and made it into my bookmarks!
by SubiculumCode 3y ago
I appreciate that. As it happens, I was thinking more for LLMs, but that is a useful link and made it into my bookmarks!
- hobofan 3y agoFor LLMs I think most people create their own LoRA and rarely share them as they are the result of finetuning on proprietary data that they don't want to leak. Separate to that, the open-source model space is so fragmented and LoRAs are not transferable between models, so few people go through the effort of publishing + documenting their LoRAs for such a small audience.
- Tostino 3y agoThem being very tailored to the main model is true, and it takes a lot of work for the lora author to target multiple base models, so they generally just target one. I plan on releasing my next iteration of my model (trained on generating knowledge graphs, among other things) as both a lora, and a pre-merged model for multiple base models (Mistral 7b, Llama2 13b, Phi2 2.7b, Yi 34b, and possibly Mixtral) Will be interesting to compare the results between them, considering last time I just did Llama2 13b. Can't wait to see the improvements in the base models since then.
- hyperliner 3y ago[dead]
- selfhoster11 3y agoYou want to check out TheBloke's Discord server in that case. There is a lot happening there, and sadly not on the indexable open web.