4 ms·
Don’t use GPT for fact finding. Use it for synthesis when you already have the facts.
by babelfish 3y ago
Don’t use GPT for fact finding. Use it for synthesis when you already have the facts.
- _cenw 3y agoOr use it for fact discovery if you're not well-versed in a field. But always check if it hallucinated reasonable sounding gibberish.
- ttul 3y agoOr use it to retrieve alleged facts from its own output, pass those alleged facts to another tool (like a legal search engine) for verification, and then get GPT to edit its output accordingly…
- whatever1 3y agoThe thing is that if you are not an expert in the field you cannot tell gibberish from legit facts. Especially if the writing style and grammar are top-notch. I know it is a bad bias, but we typically associate good and clear writing with legitimacy. Here we have chat GPT, that can do exactly that, but spit out complete bs.
- hosh 3y agoOne thing I like about phind.com is that it ties the specific assertions to the specific web page it made it from. That allows me to check into the sources. However, like all generative AI, it’s good at forming narratives, and not many people are aware how powerfully influential narrative frames are because people rarely step back to examine the frame itself.
- ayewo 3y agoYep. People rarely step back to examine how a narrative was framed. It is a cognitive bias [1] built into our brains. It takes a lot of mental effort to spot, which is why we don’t do it often. 1: https://en.wikipedia.org/wiki/Framing_effect_(psychology) https://en.wikipedia.org/wiki/Framing_effect_(psychology)
- raincole 3y agoIf you pay for ChatGPT Pro you can ask it to link to real (not hallucinated) web pages from Bing's index. I'm not promoting it. Just in case you're not aware of this feature.
- seba_dos1 3y agoEnjoy it while it lasts - soon the Bing's index will be full of hallucinated GPT content itself.
- piaste 3y ago> The thing is that if you are not an expert in the field you cannot tell gibberish from legit facts. Some answers are more easily verifiable than other. If I ask about an explanation of quantum mechanics, I might check if the names of particles and equations are correct (which they probably will be), but verifying the overall reasoning is basically the same as knowing the subject in the first place. But if I ask 'tell me the 5 most important neutrino experiments performed at the LHC and their outcomes', I can relatively easily check the results by finding the papers, their citation numbers, and reading the abstracts - I don't need to understand every detail. Maybe it will have missed one that should have been on the list, much like a manual search could have, but I won't fall for an outright hallucination. And if I ask about, oh I don't know, let's say to find some precedents for a case I'm working on, it's straightforward to take the case names that the LLM spat out, look for them in a legal database, and see if the judge actually ruled what the LLM claimed he had.
- croes 3y agoDon't take something as fact without at least checking sources. Couldn't be too hard for court decisions.
- arlcode 3y agoYes he is guaranteed to have access to some case library software. Why he didn't verify the existence of these cases spit out by a tool he used for the first time is beyond me. It would have taken him a few minutes at most to discover the fake ones and skimming the bogus references (assuming some legal competence)
- rhino369 3y agoHow do you do that? Is there a tutorial?
- babelfish 3y agoThree years of law school?