5 ms·
Relevant to this is that Perplexity lies to the user when specifically asked about this. When the user asks if there is a robots.txt file for the domain, it lie
by runako 1y ago
Relevant to this is that Perplexity lies to the user when specifically asked about this. When the user asks if there is a robots.txt file for the domain, it lies and says there is not.
If an LLM will not (cannot?) tell the truth about basic things, why do people assume it is a good summarizer of more complex facts?
- charcircuit 1y agoThe article did not test if the issue was specific to robots.txt or if it can not find other files. There is a difference between doing a poor summarization of data, and failing to even be able to get the data to summarize in the first place.
- runako 1y ago> specific to robots.txt > poor summarization of data I'm not really addressing the issue raised in the article. I am noting that the LLM, when asked, is either lying to the user or making a statement that it does not know to be true (that there is no robots.txt). This is way beyond poor summarization.
- charcircuit 1y agoI would say it's orthogonal to it. LLMs being unable to judge their capabilities is a separate issue to summarization quality.
- runako 1y agoI'm not critiquing its ability to judge its own capability, I am pointing out that it is providing false information to the user.