3 ms·
> Fine-tuning is good for treating it how to act, but not great for reciting/recalling data. What underlying process makes it this way? Is it because the promp
by gtirloni 3y ago
> Fine-tuning is good for treating it how to act, but not great for reciting/recalling data.
What underlying process makes it this way? Is it because the prompt has heavier weight?
- bluepoint 3y agoI just read the paper about LORA. The main idea is that you write the weights of each neural network as W = W0 + B A Where W0 is the trained model’s weights, which are kept fixed, and A and B are matrices but with a much much lower rank than the originals (say r = 4). It has been shown (as mentioned in the lora paper that training for specific tasks results in low rank corrections, so this is what it is all about. I think that doing LoRa can be done locally. [1] https://github.com/microsoft/LoRA https://github.com/microsoft/LoRA
- SparkyMcUnicorn 3y agoI think your question is asking about the fundamentals of how an LLM works, which I'm not really qualified to answer. But I do have a general understanding of it all. Fine-tuning is like having the model take a class on a certain subject. By the end of the class, it's going to have a general understanding on how to do that thing, but it's probably going to struggle when trying to quote the textbooks verbatim. A good use-case for fine-tuning is teaching it a response style or format. If you fine-tune a model to only respond in JSON, then you no longer need to include formatting instructions in your prompt to get a JSON output.