4 ms·
You definitely can't in the general case (for example, your 7B model is never going to be able to help much with coding, fine tuning or no). It can make sense
by smallnamespace 3y ago
You definitely can't in the general case (for example, your 7B model is never going to be able to help much with coding, fine tuning or no).
It can make sense if you have a particularly simple use case.
- qeternity 3y agoBy definition you wouldn’t fine tune a 7B model to be generally as good at GPT4. You would just be trying to overfit some small amount of functionality in a narrow domain.
- smallnamespace 3y agoYes but from the context of this discussion, we’re trying to figure out the “sweet spot” model size where it’s worth attempting fine tuning. My guess is it’s only worthwhile for matching simple tasks with small models, and any sufficiently complicated task it’s better to do few/zero shot instead.