4 ms·
>>You won't go to a doctor expecting 100% accuracy. The way LLMs work at the moment is equivalent to going to a doctor with a set of symptoms, and the doctor t
by gambiting 1y ago
>>You won't go to a doctor expecting 100% accuracy.
The way LLMs work at the moment is equivalent to going to a doctor with a set of symptoms, and the doctor telling you "ah yes, of course, it's illness X and you need to take medicine Y to cure it" and then you check and neither X nor Y exists. That's not "accuracy" that's just straight up fraud?
I wouldn't have any problem with Google's AI saying "I don't know" or "I don't have enough sources to provide a good answer, but my best approximation is this". But I literally battle misinformation produced by Google's AI search every single day, because it's telling people actual made up facts that don't exist.
- simianwords 1y agoI have a system prompt that gives me probability estimates to everything the LLM claims. Its a simple fix for your problem.
- gambiting 1y agoMy problem is people coming to communities I'm a part of with information they got "from Google" and that information being 100% wrong. Not sure how your prompt helps with that, I need Google to fix their system first.
- _shantaram 1y agoHow does that work if the LLM is the one generating the probabilities too?
- simianwords 1y agoaccording to you, if a human makes a prediction with some probability estimate it is useless because the estimate itself is inaccurate (hence probability "estimate"). in reality nothing needs to be 100% accurate for it to be useful including the estimate of probability itself.
- gambiting 1y agoIt's weird to make an assumption about OPs position and argue with that instead of what they actually wrote. Also, why make it so personal? I think it was a fair question to ask - you didn't answer how it works - just got weirdly defensive about it.
- simianwords 1y agohey that was not my intention, it was to bring to light that we assign probability estimates ourselves to our own predictions despite the estimates being not 100% accurate.
- otabdeveloper4 1y agoHate to break it to you, but the "probability estimates" it spits out are also complete bullshit.
- simianwords 1y agonope! you can also self assign probability estimates to your own predictions. if you follow it you will end up being more accurate in the long run even if your accuracy of probability estimates are themselves not accurate.
- jazzyjackson 1y agoLLMs are unable to introspect and don't know what they don't know. Watson the Jeopardy-bot had a confidence interval, but Watson was not an LLM