4 ms·
Looking at what they're doing here probably not as much as you think. As you note, with the plethora of open/open-ish LLMs today and LoRA + PEFT you can fine t
by kkielhofner 4y ago
Looking at what they're doing here probably not as much as you think.
As you note, with the plethora of open/open-ish LLMs today and LoRA + PEFT you can fine tune with low VRAM and pretty quickly so even a single A100 or whatever cloud GPUs are just fine. I've even seen people pull it off in reasonable time on super cheap T4s, A10s, etc.
I doubt anyone reading a blog post is attempting to train a "true" multi-billion param LLM from scratch.
- ftxbro 4y ago[flagged]
- kkielhofner 4y agoOuch. Ok, I'll try again because I read the article and watched the demo video from Mosaic: - MosaicML claims to have some magic that efficiently autoscales with awareness of specific instance per hour pricing. - MosaicML claims to have auto-optimize magic that makes training 3-7x faster. - The incentives seem to be aligned because their value prop is likely "we'll markup 10% but save you 50%" (or whatever). It's too bad Replit didn't provide the costs for this post in the same way Stanford, etc have done with Alpaca, Vicuna, etc. They probably didn't for from-scratch training because the answer to your question is almost certainly "it depends" or "if you have to ask you can't afford it" (for now).
- ftxbro 4y agoThanks I understand better now. Sorry my other comment was rude. Probably I'm just frustrated that I will never be able to make a sentient bot because it will cost literally a billion dollars, and I just have to sit on sidelines and watch the billionaires play with their sentient bots.