2 ms·
> As the authors of the article discuss, it has no model of reality to understand that citations are not just a form of words and symbols put together. Which
by vineyardmike 4y ago
> As the authors of the article discuss, it has no model of reality to understand that citations are not just a form of words and symbols put together.
Which is the exact same issue that plagues it for the answer it’s generating a citation for.
I believe that it can be trained to improve this, and I believe it can be nearly solvable but I don’t believe it’s a solvable problem.
It can be trained to get facts mostly right using old-school academic tests and adversarial/supervised training, like an elementary school child. Today, it (supposedly?) punts math and some other logix to a separate function, and it just compiled the prompt. That can work for validating citations and “known facts” in many cases.
But again, it can’t “think” and understand when it returns something illogical (regardless of fact correctness).
- 317070 4y agoThe problem is mostly solved though. ChatGPT doesn't have the feature to cite its sources, but some other models do https://www.deepmind.com/blog/building-safer-dialogue-agents https://www.deepmind.com/blog/building-safer-dialogue-agents
- tsimionescu 4y agoI honestly doubt very much that a reinforcement learning based approach can scale to the breadth of facts that such a model would need to know about. Is there any reason that using a rewards-based approach to cover one set of facts would in any way generalize to a completely different area of expertise (say, rewards for true facts in mathematics generalizing to questions about literary criticism, or new world cultures), or would you have to include (manually curated) rewards for all areas?