3 ms·
Maybe you should fact check your AI outputs more if you think it only hallucinates in niche topics
by malfist 1y ago
Maybe you should fact check your AI outputs more if you think it only hallucinates in niche topics
- simianwords 1y agoThe accuracy is high enough that I don't have to fact check too often.
- collingreen 1y agoWithout some exploratory fact checking how do you estimate how high the accuracy is and how often you should be fact checking to maintain a good understanding?
- simianwords 1y agoI did initial tests so that I don't have to do it anymore.
- malfist 1y agoIf there's one thing that's constant it's that these systems change.
- jibal 1y agoEveryone else has done tests that indicate that you do.
- glenstein 1y agoAnd this is why you can't use personal anecdotes to settle questions of software performance. Comment sections are never good at being accountable for how vibes-driven they are when selecting which anecdotes to prefer.
- platevoltage 1y agoI totally get that you meant this in a nuanced way, but at face value it sort of reads like... Joe Rogan has high enough accuracy that I don't have to fact check too often. Newsmax has high enough accuracy that I don't have to fact check too often, etc. If you accept the output as accurate, why would fact checking even cross your mind?
- gspetr 1y agoNot a fan of that analogy. There is no expectation (from a reasonable observer's POV) of a podcast host to be an expert at a very broad range of topics from science to business to art. But there is one from LLMs, even just from the fact that AI companies diligently post various benchmarks including trivia on those topics.
- simianwords 1y agoDo you question everything your dad says?
- platevoltage 1y agoIf it's about classic American cars, no. Anything else, usually.
- mvdtnz 1y agoIf you're not fact checking it how could you possibly know that?