3 ms·
To get the outcome you want, RAG (retrieval augmented generation) would be the way to go, not fine-tuning. Fine-tuning doesn't make the model memorize specific
by felix089 2y ago
To get the outcome you want, RAG (retrieval augmented generation) would be the way to go, not fine-tuning. Fine-tuning doesn't make the model memorize specific content like a book. It teaches new behaviors or styles. RAG allows the model to access and reference the book during inference. Our platform focuses on fine-tuning with structured datasets, so data needs to be in a specific format.
This is a very common topic, so I wrote a blog post that explains the difference between fine-tuning and RAG if you're interested: https://finetunedb.com/blog/fine-tuning-vs-rag https://finetunedb.com/blog/fine-tuning-vs-rag
- thomashop 2y agoThese days, I'd say the easiest and most effective approach is to put the whole book in the context of one of the longer context models.
- felix089 2y agoAgreed, for this use case probably the easiest way to go.
- KaoruAoiShiho 2y agoNot really, for something like gemini the accuracy and performance is very poor.
- farouqaldori 2y agoThe magic behind NotebookLM can't be replicated only with fine-tuning. It's all about the workflow, from the chunking strategy, to retrieval etc. For a defined specific use-case it's certainly possible to beat their performance, but things get harder when you try to create a general solution. To answer your question, the format of the data depends entirely on the use-case and how many examples you have. The more examples you have, the more flexible you can be.
- hodanli 2y agoI dont think it is a big deal but you can use your own image or give credit to openai presentation on YouTube.