A team where I work recently rolled out a doc-answer LLM and context was an issue we ran into. Retrieved doc chunks didn't have nearly enough context to answer

A team where I work recently rolled out a doc-answer LLM and context was an issue we ran into. Retrieved doc chunks didn't have nearly enough context to answer some of the broader questions well. Another issue I've run into with doc-answer LLMs is that they don't handle synonyms well. If I don't know the terminology for the tool, say llama-index [0], I can't ask around the concept to see if something like what I'm describing exists. A part of me thinks a lang-chain with the LLM in it might be useful. Something like 1. User makes vague query "hey, llama-index, how do I create a moving chunk answer thing with llama-index?" 2. Initial context comes back to the LLM, and the LLM determines there is not straight forward answer to the question. 2a. The LLM might ask followup questions "when you say X, what do you mean?" to clarify terms it doesn't have ready answers for. 2b. The LLM says "hm, let me think about that. I'll email you when I have a good answer." 2c. The LLM reads the docs and relevant materials and attempts to solve the problem. 3. Email the user with a potential answer to the question. 4. Stashes the solution text in the docs if the user OKs the plan. Updates an embedding table to include words/terms used that the docs didn't contain. This last step is the most important. Some kind of method to capture common questions and answers, synonyms, etc. would ensure that the model has access to (potentially) increasingly robust information.