3 ms·

No, that doesn't mean you should never ask your friend things again if they make that mistake. But, if 30% of all their references are made up then you might st

No, that doesn't mean you should never ask your friend things again if they make that mistake. But, if 30% of all their references are made up then you might start to question everything your friend says. And looking up references to every claim you're reading is not a productive use of time.

crazygringo 2y ago
If my friend has a million times more knowledge than the average human being, then I'm willing to put up with a 30% error rate on references. And I'm talking about references when doing deep academic research. Looking them up is absolutely a productive use of time -- I'm asking for the references so I can read them. I'm not asking for them for fun. Remember, it's hundreds of times easier to verify information than it is to find it in the first place. That's the basic principle of what makes LLM's so incredibly valuable.
- yaro330 2y ago
  But how can you be sure that the info is correct if it made up the reference? Where did it pull the info? What good is a friend that's just bullshiting their way through every conversation hoping you wouldn't notice? A third of the time is an insane number, if 30% of code that I wrote contained non existent headers I would be fired long ago.
  - hnfong 2y ago
    A person who's bullshitting their way doesn't get a 70% accuracy. For yes/no questions they'll get 50%. For open ended questions they'll be lucky to get 1%. You're really underestimating the difficulty of getting 70% accuracy for general open-ended questions. And while you might think you're better than 70%, I'm pretty sure if you didn't run your code through compilers and linters, and testing for at least a couple times, your code doesn't get anywhere near 70% correct.
  - t4TLLLSZ185x 2y ago
    Because he reads the reference document…
bobsmooth 2y ago
"you might start to question everything your friend says" That's exactly what the OP is saying. Verify everything.
- pixl97 2y ago
  Maybe I'm getting old, but sometimes it feels like everybody is young now and has only lived in a world where they can look up anything at a moments notice and now things they are infallible. Having lived a decent chunk of my life pre-internet, or at least fast and available internet, looking back at those days you realize just how often people were wrong about things. Old wives tales, made up statistics, imagined scenarios, people really do seem to confabulate a lot of information.

crazygringo 2y ago

If my friend has a million times more knowledge than the average human being, then I'm willing to put up with a 30% error rate on references. And I'm talking about references when doing deep academic research. Looking them up is absolutely a productive use of time -- I'm asking for the references so I can read them. I'm not asking for them for fun. Remember, it's hundreds of times easier to verify information than it is to find it in the first place. That's the basic principle of what makes LLM's so incredibly valuable.

yaro330 2y ago
But how can you be sure that the info is correct if it made up the reference? Where did it pull the info? What good is a friend that's just bullshiting their way through every conversation hoping you wouldn't notice? A third of the time is an insane number, if 30% of code that I wrote contained non existent headers I would be fired long ago.
- hnfong 2y ago
  A person who's bullshitting their way doesn't get a 70% accuracy. For yes/no questions they'll get 50%. For open ended questions they'll be lucky to get 1%. You're really underestimating the difficulty of getting 70% accuracy for general open-ended questions. And while you might think you're better than 70%, I'm pretty sure if you didn't run your code through compilers and linters, and testing for at least a couple times, your code doesn't get anywhere near 70% correct.
- t4TLLLSZ185x 2y ago
  Because he reads the reference document…

bobsmooth 2y ago

"you might start to question everything your friend says" That's exactly what the OP is saying. Verify everything.

pixl97 2y ago
Maybe I'm getting old, but sometimes it feels like everybody is young now and has only lived in a world where they can look up anything at a moments notice and now things they are infallible. Having lived a decent chunk of my life pre-internet, or at least fast and available internet, looking back at those days you realize just how often people were wrong about things. Old wives tales, made up statistics, imagined scenarios, people really do seem to confabulate a lot of information.