2 ms·

Assuming you would want to fine-tune over a codebase or set of documents, I would argue vector databases and fine-tuning are completely different tools. I woul

Assuming you would want to fine-tune over a codebase or set of documents, I would argue vector databases and fine-tuning are completely different tools. I would strongly recommend against fine-tuning over a set of documents as this is a very lossy information system retrieval system. LLMs are not well suited for information retrieval like databases and search engines. The applications of fine-tuning that we are seeing have a lot of success is making completion models like LLaMA or original GPT3 become prompt-able. In essence, prompt-tuning or instruction-tuning. That is, giving it the ability to respond with a user prompt, llm output chat interface. Vector databases, for now, are a great way to store mappings of embeddings of documents with the documents themselves for relevant-document information retrieval. I would highly recommend skimming this RLHF paper for how demonstration data was used to make a model prompt-able [1]. Keep in mind RLHF is another concept all together and we might be seeing a revolution where it might become optional (thanks to LIMA)! 1: https://huyenchip.com/2023/05/02/rlhf.html https://huyenchip.com/2023/05/02/rlhf.html

a_bonobo 3y ago
Great reply, here's an example from my own work: I want the user to be able to ask technical questions about a set of documents, then the user should retrieve a summary-answer from those documents along with a source. I first need to finetune GPT4 so it better understands the niche-specific technical questions, the words used, etc. I could ask the finetuned model questions, but it won't really know from where it got the information. Without finetuning the summarised answer will suffer, or it will pull out the wrong papers. Then I need to use a vector database to store the technical papers for the model to access; now I can ask questions, get a decent answer, and will have access to the sources.
- heliophobicdude 3y ago
  Ah! That makes sense! That's a neat strategy!
- forgingahead 3y ago
  Thanks (to both you and the parent) for sharing these details. So is it fair to say the following: 1. Fine-tuning bakes the knowledge into the model, but getting the "source" of an answer to a specific question becomes cagey and it is unclear if the answer is accurate or just a hallucination. 2. Therefore vector databases, which can provide context to the LLM before it answers, can solve this "citation" problem, BUT: 3. We then have limits because of the context window of the LLM to begin with. Is that a fair understanding, or have I totally gotten this incorrect? Edit: Or, are you saying that you both fine-tune AND also use a vector database which stores the embeddings of the dataset used to fine-tune the model?

a_bonobo 3y ago

Great reply, here's an example from my own work: I want the user to be able to ask technical questions about a set of documents, then the user should retrieve a summary-answer from those documents along with a source. I first need to finetune GPT4 so it better understands the niche-specific technical questions, the words used, etc. I could ask the finetuned model questions, but it won't really know from where it got the information. Without finetuning the summarised answer will suffer, or it will pull out the wrong papers. Then I need to use a vector database to store the technical papers for the model to access; now I can ask questions, get a decent answer, and will have access to the sources.

heliophobicdude 3y ago
Ah! That makes sense! That's a neat strategy!
forgingahead 3y ago
Thanks (to both you and the parent) for sharing these details. So is it fair to say the following: 1. Fine-tuning bakes the knowledge into the model, but getting the "source" of an answer to a specific question becomes cagey and it is unclear if the answer is accurate or just a hallucination. 2. Therefore vector databases, which can provide context to the LLM before it answers, can solve this "citation" problem, BUT: 3. We then have limits because of the context window of the LLM to begin with. Is that a fair understanding, or have I totally gotten this incorrect? Edit: Or, are you saying that you both fine-tune AND also use a vector database which stores the embeddings of the dataset used to fine-tune the model?