4 ms·
What's surprising about this is that you can get the bullshit machine to produce correct externally validate citations. It's not particularly hard either—it's o
by Spivak 1mo ago
What's surprising about this is that you can get the bullshit machine to produce correct externally validate citations. It's not particularly hard either—it's one of the first things you build when you give an LLM access to a body of documents/search. So for a large public service to whiff like this is certainly a stain on their credibility.
- delichon 1mo agoOn the other hands it's a boost to their credibility that they make their mistakes easier to evaluate than their competition does. It would be worse if they had a similar error rate without openly providing references. Kudos to Perplexity for including more empirical attack surface.
- tempfile 1mo ago> you can get the bullshit machine to produce correct externally validate citations How?
- MarkusQ 1mo agoBy training it on a whole bunch of examples with valid external citations, to the point where it's able to hallucinate something that's valid. Of course, it won't always work, which is the point of TFA.
- tempfile 1mo agoI don't think this is an answer. I'm basically claiming there is an uncontrollable error probability, and I think you agree with that. The person I'm replying to implies it can always be reduced (maybe even to zero).
- MarkusQ 28d agoAh, yes, I see. The distinction between a working clock and one that's right twice a day.
- pessimizer 1mo agoI assume by forcing them to validate their citations with a deterministic tool.
- tempfile 1mo agoWhat deterministic tool will validate that a citation faithfully represents the cited claim?