4 ms·
The results are pretty good; I wish they'd just publish the models so we can run the inference locally (not too many people have access to 8xA100 to train thems
by gigel82 4y ago
The results are pretty good; I wish they'd just publish the models so we can run the inference locally (not too many people have access to 8xA100 to train themselves, though I appreciate including the training data and instructions too).
- jerrygenser 4y agoLlama may not be licensed for people to share since you need to apply to get one from Facebook for non commercial use. I think it's more of a license issue Hopefully similar work can be done with LoRA so the fine-tuning is not as expensive
- sahil_chaudhary 4y agoYeah the license issue is the only reason why I didn't release the weights yet.
- doctoboggan 4y agoIf a couple of us get together and throw in some money we could train it on Lambda Labs hardware like the OP suggests. I would volunteer to do it myself but I don’t know enough about training models to guarantee I am not wasting money with a stupid mistake.
- menzoic 4y agoAnyone with a few hundred bucks to spare can do it by renting GPUs from a cloud provider. It only cost Stanford $600 to create Alpaca from LLAMA. $100 to generate instructions with GPT-3 and $500 to rent cloud GPUs. The license restriction is due to the use of GPT-3 output to train a model.
- MacsHeadroom 4y agoMore like $50 or even $5 or less for the cloud GPUs. Alpaca-7B's compute costs were close to $50 and that was before the 100x cost savings of using LoRA. A 4bit LoRA fine tune of this project would cost less than $5 to train even up to 30B/33B.
- nickthegreek 4y agoI’d love to see a crowdsourcing platform to donate to specific fine-tuning projects. I would gladly throw some money at someone to do the labor and release the models to the public.
- sahil_chaudhary 4y agoGiven how many people are asking for the weights, I'm working to make them available now. So you'll be able to run them locally as well.