3 ms·
I don’t know about out of the box, but I’ve been having a great time training my own domain-specific models from scratch and utilizing them locally. It does see
by zxexz 4y ago
I don’t know about out of the box, but I’ve been having a great time training my own domain-specific models from scratch and utilizing them locally. It does seem that the more domain specific a task is, far fewer params are needed.
Having fun right now trying to build a model in GDELT, not much luck so far, but I’ve pushed less than 5% of the data through so far.
I’ve also been experimenting on fine-tuning Llama on my personal data archives, which seems promising but it’s pretty expensive to do so. Hoping someone will release a ~13B param model of Llama that they’ve trained with transfer learning from the 65B llama model and other data. FWIW even the 7B llama model running through llama.cpp after being quantized performs (subjectively, but substantially) better than GPT2.
This has been one of my most expensive, but also most rewarding, hobbies thus far.
- worldsayshi 4y agoWhat is the ballpark cost of this hobby, if I may ask? Do you train exclusively in the cloud?
- zxexz 4y agoReally depends. I’ve pretty much almost burned through all the cloud credits I’ve been able to find at this point. Probably spent 20 grand over a year in credits? Never more than a couple hundred of my own money. Still trying to find a better solution. I have some 2080 TIs that I use locally. I tend to prototype architectures locally then rent out some A100s for a few hours to see how it goes, continue training when I can spare the cash. I do a lot of CPU and sharded training too. I really wish there were better options for hobbyists to play around with this stuff.
- swyx 4y agogot a quick guide as to available cloud credits for GPUs?
- LoganDark 4y agoTraining domain-specific models from scratch is both very difficult and also not possible in some cases. Sometimes we want to generate some niche content where there isn't enough training data to create a domain-specific model. We would have to somehow find a model that can produce the niche content through understanding what we're asking rather than through actually being trained on it. Producing "understanding" is not something you can do at home unless you have a bunch of A100s and have studied machine learning for years. -Emily
- zxexz 4y agoTotally agree there. Even people that have years of ML under their belt may have a hard time there. And the cost (and scarcity) of those GPUs really are a major barrier to entry.