4 ms·
Something I've learned as I've dived a bit into fine-tuning is that it's really hard. I even have what I think is good data, and a lot of it. But, as a non-ML e
by phillipcarter 3y ago
Something I've learned as I've dived a bit into fine-tuning is that it's really hard. I even have what I think is good data, and a lot of it. But, as a non-ML engineer who just wants to use these tools as good APIs (and someone with the ability to ensure I have a coherent data pipeline), I woulnd't know how to proceed if I wasn't already working with an ML engineer who's been doing this sort of stuff for a while.
Specifically, I fine-tuned gpt-3.5 and llama-2-7b on some real-world usage data, and I can't tell a single difference between the quality of these outputs compared to "base" gpt-3.5 with the same requests made to it. Moreover, if I attempt to remove a lot of the "static" sections of a prompt (after all, there's 2k+ lines of data it was fine-tuned on where this is all present), both models just go completely off the rails.
I'd love to get into a world where I can fine-tune a bunch of different models. But it's so, so much harder than just calling OpenAI's API and getting really good results with that and some prompting work. If you're able to help crack that nut then there's a lot of people like me who would pay money to have their problems solved.
- adw 3y ago2k lines is (probably) not a lot of data.
- phillipcarter 3y agoThat may be the issue. I can get about 6k lines of data today and in time, could have a bunch more. Each line represents a full request (user input, model output, full prompt, etc.) so I would intuitively think that 2k of these is good enough. And it's actually good enough for llama-2-7b, insofar as it transforms this model from "terrible with my prompt in production today" to "this could be workable, actually". But my bar is to be better than gpt-3.5 and require less text in my prompt, but no fine-tuning I've done so far has achieved that.
- kiratp 3y agoIt’s not reasonably possible (currently?) to get the same performance from a 7 billion parameter model as a 175 billion parameter model with just an additional 6000 lines of finetuning data.
- phillipcarter 3y agoIt gets very similar performance, so maybe it is possible. But also when I fine-tune 3.5, its response quality is indistinguishable from when I use the base model. All of this is to say: this shit is way harder than it needs to be. I'm not an ML engineer but I do know my data and how to get it. Why is it still so hard to specialize a model?
- kiratp 3y agoBecause we don’t have a clear understanding of how the things work yet.
- jph00 3y agoYeah it really needs to be much easier. Hopefully we can help with that.
- alew1 3y agoOne thought: If you want to be able to remove the static part, you could consider fine tuning without the static part. If you fine tune with, you’re teaching the model that the desired behavior occurs only in the presence of the static part (hence the going off the rails).