3 ms·
What would you expect from fine tuning? What would the input training material be, and what would the expected differences in output be?
by MuffinFlavored 2y ago
What would you expect from fine tuning? What would the input training material be, and what would the expected differences in output be?
- magicalhippo 2y agoIn several cases I've been wanting better prompt adherence. Llama 3.2 Vision is very strictly trained to output a summary at the end which I find difficult to get it stop doing for example. Another one is that when given a math problem and asked to generate some code that computes the result, most models outputs code fine but insists on doing calculations themselves even if the prompt explicitly say they shouldn't. As expected, sometimes these intermediate calculations are incorrect and hence I don't want the LLM to do that when the produced code would handle it perfectly. If the input prompt contains "four times five" I want the model to generate "4 * 5" rather than "20", consistently. I've been curious to see if I could tune them to adhere better to the kind of prompts I would be giving. For LLama 3.2 Vision I've also been curios if I can get it to focus on different details when asked to describe certain images. In many cases it is great but sometimes misses some key aspects. As for the input training material, that's what I'm trying to figure out what I need. I feel a lot of the guides are like that "how to draw an owl" meme[1], leaving out some crucial aspects of the whole process. Obviously I need input prompts and expected answers, but how many, how much variation on each example, and do I need to include data it was already trained on to avoid overfitting or something like that? None of the guides I've found so far touch on these aspects. [1]: https://knowyourmeme.com/memes/how-to-draw-an-owl https://knowyourmeme.com/memes/how-to-draw-an-owl