Never rely on the AI's own "memory". GPT-4 can remember a lot of deep technical things correctly from its training, but it can also write non-factual statements

Never rely on the AI's own "memory". GPT-4 can remember a lot of deep technical things correctly from its training, but it can also write non-factual statements that look perfectly reasonable. We only use LLMs to retrieve and extract information from reliable sources (mostly papers and patents). If you provide extensive background information in the prompt, it will stick to that and avoid making up facts, and it has the added benefit of including verifiable references for each statement in the answer. Again, we treat the AI mostly as an "automatic reader", it's very good at that, it's not hard to keep it factually grounded and that alone can save hours of research. But when asking it to do deep reasoning or original writing it can be a bit more tricky to keep it controlled.