4 ms·
By definition you wouldn’t fine tune a 7B model to be generally as good at GPT4. You would just be trying to overfit some small amount of functionality in a nar
by qeternity 3y ago
By definition you wouldn’t fine tune a 7B model to be generally as good at GPT4. You would just be trying to overfit some small amount of functionality in a narrow domain.
- smallnamespace 3y agoYes but from the context of this discussion, we’re trying to figure out the “sweet spot” model size where it’s worth attempting fine tuning. My guess is it’s only worthwhile for matching simple tasks with small models, and any sufficiently complicated task it’s better to do few/zero shot instead.