38 ms·
I still don't have a really good answer to this question: If you want to be able to do Q&A against an existing corpus of documentation, can fine-tuning an LLM
by simonw 4y ago
I still don't have a really good answer to this question:
If you want to be able to do Q&A against an existing corpus of documentation, can fine-tuning an LLM on that documentation get good results, or is that a waste of time compared to the trick where you search for relevant content and paste that into a prompt along with your question?
I see many people get excited about fine-tuning because they want to solve this problem.
The best answer I've seen so far is in https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb https://github.com/openai/openai-cookbook/blob/main/examples...
> Although fine-tuning can feel like the more natural option—training on data is how GPT learned all of its other knowledge, after all—we generally do not recommend it as a way to teach the model knowledge. Fine-tuning is better suited to teaching specialized tasks or styles, and is less reliable for factual recall. [...] In contrast, message inputs are like short-term memory. When you insert knowledge into a message, it’s like taking an exam with open notes. With notes in hand, the model is more likely to arrive at correct answers.
- deleted 4y ago[deleted]
- lumost 4y agoWith search, you don’t know if the model/search engine will retrieve the right context. With fine tuning you don’t know if it will forget important info, or learn incorrect things.
- kordlessagain 4y agoWith search, you can "tune" the query by using related references to the documents. Ideally, we include documents that are relevant to the topics at hand and eliminate documents that aren't. Using prompt chaining, or an iterative process of labeling and extracting features, we may be able to increase the efficiency of the final prompt. Obviously this increases costs, but so does repeated fine-tuning.
- furyofantares 4y agoAs long as the search part of search + prompt is good, the prompt part will emit accurate results or will say it couldn't find the answer. You can also cite the sources this way. It seems pretty expensive because you may be pasting a lot of context into each query. If you allow the user to have follow up queries and you want to retain context of the conversation that seems expensive too. But it does seem like it should give the best results for Q&A as long as question is directly answered in your data somewhere.
- spenczar5 4y agoI have similar questions for code assistance. Github Copilot seems to be the most effective code assistant currently. It seems to use many heuristics for figuring out relevant snippets to include in prompts, like computing Jaccard similarity of windows of the last 20 opened files. It also tries some tree-sitter cleverness for some languages, but when I snoop on the HTTP traffic it seems to almost always just give up and only include the rest of a file as context. I have wondered whether a model fine-tuned on my own code would do much better, and be simpler. But perhaps building embeddings and searching them (like in the article you linked) would be superior. Code assistants need to be super low latency though which maybe complicates things too.
- thejosh 4y agoThis is why Tabnine got e super excited, until I ran across the issue where they think their results are better than what the IDE gives you, which is incredibly annoying. https://github.com/codota/tabnine-intellij/issues/18 https://github.com/codota/tabnine-intellij/issues/18 . I would be happy to pay, but it seems they are convinced their way is best. I honestly think that if you could have all your private code indexed and accessible, this would be a game changer as it has way better context.
- spenczar5 4y agoI think so too. I’m thinking about writing a little server you would run as a sidecar process. It would tokenize your code and build a local index in a vector DB. Client plugins in IDEs would ask that server for similar code snippets, with a short deadline for response, perhaps 1ms. Then the plugins could assemble a prompt to send to a configurable LLM backend, and do editor-specific code insertion or suggestion or whatever. I use emacs, so I know how I want that to work for me, but it must be different for others. My open questions though are: - does the tokenizer for the index need to be the same as the one used by the LLM? - is running a local LLM feasible, or should I just assume prompts must sent over the network? This would affect latency and privacy a lot. - is fine tuning helpful? Or necessary, even, to ensure the LLM only provides source code, and only in the target language?
- 4y ago
- thejosh 4y agoLangchain wrote about LLMs and SQL, and although it's about SQL, it's still a great read, especially the references. https://blog.langchain.dev/llms-and-sql/ https://blog.langchain.dev/llms-and-sql/ I'm also super keen for the 32k API limit for OpenAI, that's going to be great.
- amluto 4y agoAn answer from OpenAI will be very biased. OpenAI will happily take your money to query their models with long prompts. They will also happily take your money to compute embeddings and thus help you search (and lock you in!). But, as far as I know, OpenAI will not help you fine-tune, they will not run a fine-tuned model for you, and they would probably prefer that you use their models over open models that can be fine tuned. (None of this is to say that fine tuning is better. I’m just saying that OpenAI has a strong commercial bias.)
- spenczar5 4y agoWhat about their fine tuning API? https://platform.openai.com/docs/guides/fine-tuning https://platform.openai.com/docs/guides/fine-tuning
- amluto 4y agoInteresting. But not supported for their chat models. I like how the fine-tuning page talks about how fine tuning is supposedly better, though.
- sysstemlord 4y agoI saw a video tutorial about it and if i understood correctly, the API will automatically include whatever you "tuned" in the prompt, making it just as costly i guess. But i could be wrong as it was late and i was getting sleepy.
- lukev 4y agoThe search+prompt approach has another benefit, which is that allows a chat interface to "cite it's source", something you really can't do with just parameters (fine-tuned or not.) Although a lot of people are working on hallucination reduction, I think we're a long way off from that in the general case. So having the ability to point to a real piece of data, outside the model, is important for applications where accuracy matters.
- vharuck 4y agoI've done that with the rough Q&A tool I made. I include links to websites or documents not so much for verification, but because I work with government info and the devil's in the details. The Q&A is more like a smart search engine, with the summarized answer being a nice bonus.
- crosen99 4y agoIt's amazing how much misinformation and vague information there is on this topic. I tried getting to the bottom of this in the following post in the OpenAI forum: https://community.openai.com/t/fine-tuning-myths-openai-documentation/ https://community.openai.com/t/fine-tuning-myths-openai-docu... Bottom line is that fine-tuning does not seem to be a feasible option for adding new knowledge to a model for question answering.
- bitL 4y agoMy research shows otherwise. Tuning via transformer adapters pretty much added new knowledge to QA models or could be used for adversarial QA training. You can throw away learned adapters anytime and retrain from scratch with new information if your adapters become stale. Fine-tuning is cheap and small (e.g. 60kB data in an adapter). You can customize it in production for each individual customer as well by swapping adapters at the time of inference. Embeddings for very short-term facts and adapters for medium-long-term info seems like the best combination.
- pumanoir 4y agoYou mean like what's described in this blog post, correct? https://adapterhub.ml/blog/2022/03/adapter-transformers-v3-unifying-efficient-fine-tuning/ https://adapterhub.ml/blog/2022/03/adapter-transformers-v3-u...
- bitL 4y agoYes, those adapters. There are many types now, most recently LLaMA adapter: https://arxiv.org/abs/2303.16199 https://arxiv.org/abs/2303.16199
- crosen99 4y agoCould you link to your research and/or describe the models, libraries, data and tests you used for this?
- tmaly 4y agoHave you tried fine-tuning via adapter, if so what has been your experience and was the the total cost?
- kaycebasques 4y ago> the trick where you search for relevant content and paste that into a prompt Supabase Clippy was the first docs site to ship this experience to production as far as I can tell: https://supabase.com/blog/chatgpt-supabase-docs https://supabase.com/blog/chatgpt-supabase-docs I believe they called it "context injection" and I have been following suit in my own writing on the topic. I am prototyping experiences like Supabase Clippy and am also very interested in fine-tuning for docs Q&A. But my main question is: what exactly would the fine-tuning inputs and outputs look like for docs Q&A? Edit: For Q&A the question is the input and the answer is the desired output? Is that right? A more general comment about fine-tuning for docs from my blog: > AI is all about prediction. Given this temperature, this wind, this day of the year, what is the chance of rain? Temperature, wind, and date are your inputs. Chance of rain is your desired output. Now, try to apply this same type of thinking towards documentation. What are your inputs? What’s your output? The page title and code block could be your inputs. Whether or not the code builds could be your output. Or maybe the code block should be the output? This is why I keep saying that applying fine-tuning to docs is tricky. What are the inputs and outputs? https://technicalwriting.tools/posts/ten-principles-response/ https://technicalwriting.tools/posts/ten-principles-response... (I am an AI n00b and have not looked deeply into how fine-tuning works but it's high on my list to experiment with OpenAI's fine-tuning API. Please LMK if I am getting any fundamentals wrong.)
- RockyMcNuts 4y agoWhich OpenAI fine-tuning API? I don't think fine-tuning is currently available for GPT-4 https://help.openai.com/en/articles/7127982-can-i-fine-tune-on-gpt-4 https://help.openai.com/en/articles/7127982-can-i-fine-tune-...
- deleted 4y ago[deleted]
- dragonwriter 4y ago> If you want to be able to do Q&A against an existing corpus of documentation, can fine-tuning an LLM on that documentation get good results, or is that a waste of time compared to the trick where you search for relevant content and paste that into a prompt along with your question? If you want exact detailed recall, then using a framework that provides search and recall (by embeddings or otherwise) is probably always going to beat fine tuning, but also remember, it doesn’t have to be either-or. I mean, if you want a person to handle Q&A on a corpus, is it better for them to have studied it, or have direct access to the corpus with an appropriate index? The answer is clearly that its better if they’ve trained and have access to the corpus, and while LLMs aren’t the same as people, I think the answer for them is the same here.