3 ms·
fine-tuning original weights solves that, and any sane person would fine-tune for their task anyways to get better results
by slt2021 4y ago
fine-tuning original weights solves that, and any sane person would fine-tune for their task anyways to get better results
- textninja 4y agoSince fine-tuning is often done by freezing all but the top layers I wonder if it would still be possible to take a set of inputs and outputs and mathematically demonstrate that a model is derivative of ChatGPT. There may well be too much entropy to unpack, but I’m sure there will be researchers exploring this, if only to identify AI-generated material. Of course, since the model is so large and general purpose already, I can’t assume the same fine-tuning techniques are used as for vastly smaller models, so maybe layers aren’t frozen at all.