3 ms·

Do you have any pointers on how to get GPT-4 to do citations? Is a prompt like “quote back the passage you are citing” so you can locate and highlight the origi

• pwmiller_ai 3y ago
  For RAG applications, starting simply with a reference to the most relevant chunk(s) is helpful in building transparency. A lot of our contract review task is a data extraction one (e.g. extract this type of insurance language and compare against your policy). As such, it's much easier to pinpoint the exact text from the source doc for citation. In our applications, currently, we are doing citations as a postprocessing task as opposed to as part of the prompt itself. Finding that feeding too many instructions to the LLM results in worse responses. We're not fine-tuning today. Instead, "tuning for precision" is done through prompt chains. A simple example would be returning "I don't know" early on if the document isn't a contract or doesn't have clear insurance requirements in it. We've had success with various guardrail prompts. The citation work we did at Google used model internals to highlight text (path integrated gradients). It's also easier to finetune for precision when you have control over the model itself.