3 ms·
Faster, cheaper fine-tuning and training If I could train a useful model, on my own data, in a reasonable time I would want to have a CI-training pipeline to
by nico 3y ago
Faster, cheaper fine-tuning and training
If I could train a useful model, on my own data, in a reasonable time
I would want to have a CI-training pipeline to always have my models up to date
- makestuff 3y agoYeah I remember in undergrad I was working on using transformation learning to train an object detector. Basically you only needed 100ish images to get the model to detect that new object really well. I'm not sure what the analogous term is for a similar process on LLMs, but that will be huge when there is a service for it.
- visarga 3y agoLLMs can do that without any examples (zero shot) or with one or a few demonstrations in the prompt, if you can describe the task in the limited context window. If you want for example to train the model to learn to use a very large API, or access the knowledge in a whole book, it might need fine-tuning.
- nico 3y agoCould I just train a very small LLM with an English dictionary + Python + large API documentation + large Python code base? Then do some chat fine tuning (like what HF did with StarCoder to get ChatCoder) And get a lightweight LLM that knows the docs and code for the thing I need it for After that, maybe incrementally fine tune the model as part of your CI/CD process
- toss1 3y agoHow similar were the object to other objects? E.g., were you trying to distinguish an object vs nothing, a bicycle vs a fish, a bird vs a squirrel, or two different species of songbird at a feeder? How much would the training requirements increase or decrease moving up or down that scale?
- ilaksh 3y agoThe PaLM 2 stuff released yesterday has fine tuning for their newest large models as a core feature.
- jiggawatts 3y agoOpen-source LLM projects have largely solved this using Low-Rank Adaptation of Large Language Models (LoRA): https://arxiv.org/abs/2106.09685 https://arxiv.org/abs/2106.09685 Apparently an RTX 4090 running overnight is sufficient to produce a fine-tuned model that can spit out new Harry Potter stories, or whatever...
- nico 3y agoHow much would they cost aprox per training? What would the quality of the model be, compared to what Karpathy uses on his video “let’s build gpt from scratch”? In that video, he builds a decoder-only transformer model, that learns Shakespeare from 1MB of data and trains in 15min
- jiggawatts 3y agoThese just fine-tune existing models, so the cost is single digit dollars. Whatever it costs in electricity to run a desktop GPU for a day or a few days.
- hooande 3y agoJust use a machine learning method other than an LLM. If you're going to go to the effort of fine tuning and training, pay people to collect and label data like we used to do before Feb 2023. If your main concern is question answering or summarization or code completion there are plenty of ways to do that now. If you really require the advanced emergent properties of LLMs, you'll have to work with a company that can afford to train a transformer on the Entire Internet.