3 ms·
I just don't know about this. I also find it answers great when I'm not familiar with a topic. However, when I am familiar with a topic I find all sorts of inco
by yoyohello13 3y ago
I just don't know about this. I also find it answers great when I'm not familiar with a topic. However, when I am familiar with a topic I find all sorts of inconsistencies or wrong facts. I'm concerned the same inconsistencies are there in the topics I'm not familiar with, I just don't know enough about the subject to spot them.
- sunsetMurk 3y agoconceptually, I get it.
- jprete 3y agoI would expect the distribution of inconsistencies and errors to be the same for both sets of topics, in the absence of any other information.
- dmux 3y agoI've noticed the same thing, specifically when asking about programming topics. It reminds me of the Gell-Mann Amnesia effect, but at least you're aware of the inaccuracies. https://en.wikipedia.org/wiki/Michael_Crichton#GellMannAmnesiaEffect https://en.wikipedia.org/wiki/Michael_Crichton#GellMannAmnes...
- sundarurfriend 3y agoI'm normally worried about that kind of Gell-Mann amnesia effect when I'm reading articles or watching YouTube videos that dive into a topic I'm unfamiliar with. But when it comes to LLMs, I'm conscious of the fact that I'm asking deep, specific questions when it comes to programming and software, about things on the fringes that the LLM can't have read much about. In contrast, with things outside my area of expertise, I'm asking very superficial question that any practitioner of the field could answer, and so likely has many many sources for the LLM to pull information from. With fields outside my area of expertise, I'm most often asking questions that are comparable to "what is a variable" or at most "how does A* search work". That gives me confidence that the quality of answers is likely to be much better with these questions.