4 ms·
LLM fine tuning tends to destroy the model's capabilities if you aren't very careful. It's not as easy or effective as with image generation.
by qcnguy 1y ago
LLM fine tuning tends to destroy the model's capabilities if you aren't very careful. It's not as easy or effective as with image generation.
- israrkhan 1y agodo you have a suggestion or a way to measure if model capabilities are getting destroyed? how do one measure it objectively?
- RALaBarge 1y agoAsk it a series of the same questions after you train that you posed before training started. Is the quality lower?
- israrkhan 1y agoThat series of questions will measure only a particular area. I am concerned about destorying model capabilities in some other area that that I do not pay attention to, and have no way of knowing.
- simonh 1y agoIsn’t that a general problem with LLMs? The only way to know how good it is at something is to test it.
- mensetmanusman 1y agoThese are now questions at the cutting edge of academic research. It might be computationally unknowable until checked.
- nxobject 1y agoMy very cursory understanding -- at least from Unsloth's recommendations -- is that you have to work very hard to preserve reasoning/instruct capabilities [1]: for example to "preserve" Qwen3's reasoning capabilities (however that's operationalized), they suggest a fine-tuning corpus that's 75% chain of thought to 25% non-reasoning. Is that a significant issue for orgs/projects that currently rely on fine-tuning? [1] https://docs.unsloth.ai/basics/qwen3-how-to-run-and-fine-tune#notebook-guide https://docs.unsloth.ai/basics/qwen3-how-to-run-and-fine-tun...