3 ms·
What if we fine tune a model like LLaMA on all published research papers? Would be able to product new knowledge?
by marcopicentini 3y ago
What if we fine tune a model like LLaMA on all published research papers? Would be able to product new knowledge?
- capableweb 3y agoDepends on what you mean with "new knowledge". A lot of inventions are "just" novel combinations of things we already knew.
- 6gvONxR4sf7o 3y agoThe value of research isn’t just in the ideas. It’s in the grounding of ideas in fact. It might be able to suggest interesting experiments, but those experiments still need to be carried out.
- zbyforgotp 3y agoFinetuning is not useful for teaching new facts, the current solution for that is using RAG: https://zzbbyy.substack.com/p/why-you-need-rag-not-finetuning https://zzbbyy.substack.com/p/why-you-need-rag-not-finetunin...