4 ms·
the mechanism is the same (that's why it impacts all weights) but the target of gradient descent is not the same. in finetuning they aren't saying "go down the
by cowl 2y ago
the mechanism is the same (that's why it impacts all weights) but the target of gradient descent is not the same. in finetuning they aren't saying "go down the mountain" anymore, but "go down toward this plateau" ofcourse this changes the gradient.
is it "learning" in a certain sense? sure, the same way like all indoctrinations are sold as "teaching". the model "learned" to be rapresentative but forgot what 1940 nazi soldier was like...
and fine tuning is not a scalable approach because it has human feeback in the loop. could they fix this error with more fine tuning? yes and they tried abut then the users simply asked "give me a picture a viking warriors" and the problems was again there, you can't fine-tune everything even if we assume the purpose is always noble.
- phreeza 2y agoI think all this ideological stuff is completely unrelated to the issue we started talking about. You can fine tune a large model on whatever data you want, and so you can also fine tune it on the most recent user inputs. You can do unsupervised fine-tuning btw, no need for human in the loop. It all depends on what you want to achieve.