4 ms·
Can anyone who's worked with fine-tuning models this large weigh in on how much fine-tuning data is typically required to be effective? Can 100k tokens (as ment
by lukev 3y ago
Can anyone who's worked with fine-tuning models this large weigh in on how much fine-tuning data is typically required to be effective? Can 100k tokens (as mentioned in the docs) really influence the behavior of the base model that much, or was that just a toy example?
- danielbln 3y agoJust look at instruct fine tuning that is being done to completion models to turn them into assistant models. A few thousand examples are enough to alter the behavior of the model and what and how it outputs things significantly and thoroughly. Mechanisms like LoRA (a very efficient fine-tuning mechanism that has a accuracy penalty) change only a few layers at the top to alter the model considerably.
- minimaxir 3y agoFrom the newly-updated fine-tuning guide: > To fine-tune a model, you are required to provide at least 10 examples. We typically see clear improvements from fine-tuning on 50 to 100 training examples with gpt-3.5-turbo but the right number varies greatly based on the exact use case. > We recommend starting with 50 well-crafted demonstrations and seeing if the model shows signs of improvement after fine-tuning. In some cases that may be sufficient, but even if the model is not yet production quality, clear improvements are a good sign that providing more data will continue to improve the model. No improvement suggests that you may need to rethink how to set up the task for the model or restructure the data before scaling beyond a limited example set.
- htrp 3y agoSignificantly. You start to see improvements at 1000 tokens.
- nunodonato 3y agoI did a lot of fine-tuning for an AI assistant I was building. I noticed a good improvement when I crossed the 200-300 samples (Q&A pairs)
- robertnishihara 3y agoWe've run experiments on datasets ranging from 5K - 100K examples, which gave fantastic results [1]. Some examples - https://huggingface.co/datasets/b-mc2/sql-create-context https://huggingface.co/datasets/b-mc2/sql-create-context - https://huggingface.co/datasets/GEM/viggo https://huggingface.co/datasets/GEM/viggo On the other hand, 8K examples was not enough to learn to solve grade school math problems [2], so it is very problem dependent. [1] https://www.anyscale.com/blog/fine-tuning-llama-2-a-comprehensive-case-study-for-tailoring-models-to-unique-applications https://www.anyscale.com/blog/fine-tuning-llama-2-a-comprehe... [2] https://huggingface.co/datasets/gsm8k https://huggingface.co/datasets/gsm8k