4 ms·
The answer by humans is a) publicly accessible b) hallucination-free (although it still may not be correct) c) subject to a voting process which gives a good si
by lordgrenville 2y ago
The answer by humans is a) publicly accessible b) hallucination-free (although it still may not be correct) c) subject to a voting process which gives a good signal of how much we should trust it.
Which makes me think, maybe a good move for Stack Overflow (which does not allow the submission of LLM-generated answers, wisely imo) would be to add an AI agent that would suggest an answer for each question, that people could vote on. That way you can elicit human and machine answers, and still have the verification process.
- scotty79 2y agoIt's a good idea but probably not easy to implement. SO answers are usually quite neat, like an email. Solving a problem with ChatGPT is more like ... chat. It's hard to turn it into something googlable and Google is how SO gets most of its traffic and utility.
- TeMPOraL 2y agoLLMs are much better experience on the "Q side". Sure, there's the occasional hallucination here and there, but QnA sites are not all StackOverflow. Most of them are just content farms for SEO and advertising purposes - meaning, the veracity of the content doesn't matter, as long as it's driving clicks. At this moment, this makes LLMs much more trustworthy.
- intended 2y agoWhy a vote? Voting != Verification.
- Davidzheng 2y agoI don't think human mistakes are distinguishable from hallucinations.
- Y_Y 2y agoLet's train a discriminator and see!
- david-gpu 2y agoAs a user, why would I care whether an answer is "incorrect" or "hallucinated"? Neither one is going to solve the problem I have at hand. It sounds like a distinction without a difference.
- lordgrenville 2y agoOne relevant difference is that a better-quality human answer is correlated with certain "tells": correct formatting and grammar, longer answers, higher reputation. An incorrect LLM answer looks (from the outside) exactly the same as a correct answer.
- mikepurvis 2y agoObviously there are exceptions but human-wrong answers tend to be more subtly wrong whereas hallucinated answers are just baffling and nonsensical.
- alex7734 2y ago- AI answers are much easier and faster to produce thus it's going to produce a lot more wrong answers by sheer volume. - AI answers are grammatically correct and verbose so it looks like the poster put effort into it which deceives people into thinking the answers is more trustworthy than it is. Barring trolls, humans (for the most part) only answer if they think they're right, and the more effort put into the answer the more likely they don't get it wrong.