3 ms·

This is an insightful comment, but I feel like you omit the fact that LLMs often give out verifiably false information that can hurt the user or other people.

This is an insightful comment, but I feel like you omit the fact that LLMs often give out verifiably false information that can hurt the user or other people. It is true that this also happens on the Internet, but! When I encounter an article about a topic and it is clearly LLM generated, I can expect it doesn't contain much valuable information, only rehashes of what is already out there. On the other hand, when it is clearly written by a human, I can expect to learn something new, even though the author has some bias.

menaerus 7mo ago
It's wrong to assume incompetence, which is what you did to a comment which displays much deeper chain of thought about the subject. A more proper way of doing it would be to reflect over your own opinions and critically assess them, as the comment points that out. To be more specific, what makes you think that the person you're replying to is not aware that LLMs can give false information, and is not taking that into account?
lovelearning 7mo ago
You're right that LLMs do spit out false information or wrong knowledge. I've experienced them too. But a redeeming quality is that we can ask the same LLM to fact check its own answer step by step in real time with little effort. They often identify their own hallucinations and reduce the probability of retaining that mistake in the rest of the conversation. This isn't easy with human sources. The effort to fact check without LLMs or ask the sources to fact check themselves are both higher. So it's often not done at all. We also often ignore subtle but very common biases in human media sources [1], which create other types of errors like omissions and euphemisms which have been no less harmful than LLM hallucinations. The case of the Iraqi WMDs of Iraq and the NYT's dispersal of that disinfo, for example [2]. Regarding valuable information and rehashing, we probably shouldn't equate between all the things LLMs can do, and AI-generated articles. The quality of the latter may be entirely due to the lack of interest, attention, and cost concerns of whoever generated the article. Anecdotally, I have often found valuable knowledge and obscure connections by using deep research tools with careful prompts. Lastly, if you're frequently finding something new from human-written sources, and LLMs are being trained on most of those same sources, isn't it logical that the latter will also likely output that same information? This is why I feel human and AI sources are probably best used as complementary tools. Neither set of sources are perfect but each set has its strengths. By using both, we can get closer to an objective truth than using only one of them. [1]: https://gipplab.uni-goettingen.de/wp-content/uploads/2022/04/181122_mediabiasprocess-800x602.png https://gipplab.uni-goettingen.de/wp-content/uploads/2022/04... [2]: https://www.theguardian.com/media/2004/may/26/pressandpublishing.usnews https://www.theguardian.com/media/2004/may/26/pressandpublis...
- dalke 7mo ago
  The effort to fact check with LLMs is also high. Here's one from a few days ago. Someone used AI to generate an image in the style of a Charles Schulz Peanuts cartoon. Someone else observed that there were 5 fingers on the characters, and quoted as Google AI as saying “Charlie Brown, along with other Peanuts characters, is generally depicted with four fingers on each hand (three fingers and one thumb) ...” Yet if you go to the Wikipedia entry at https://en.wikipedia.org/wiki/Peanuts https://en.wikipedia.org/wiki/Peanuts you'll see the kids have 5 fingers. Or take a look at the actual cartoons. Or read the TVTropes entry https://tvtropes.org/pmwiki/pmwiki.php/Main/FourFingeredHands https://tvtropes.org/pmwiki/pmwiki.php/Main/FourFingeredHand... under "Comic Strips". Fact checking this with human sources is easy and not ambiguous. While LLMs are being trained that many cartoon characters only have a thumb and three fingers - it is a trope for a reason - so isn't it logical for LLMs to give the wrong answer for a comic where the human characters are actually drawn with 5 fingers? My experience with LLMs is they keep getting things wrong, when details matter. Do you ask the LLM to fact check everything? (In which case, why isn't that part of the standard prompt?) Or do you only ask to fact check things where you are unsure about the answer? (In which case, is it the algorithm telling you what you want to hear?) When do you stop the fact checking?
  - lovelearning 7mo ago
    > When do you stop the fact checking? Exactly the same calculus as fact checking anything else from any other source. What are the social/economic/ethical consequences to me if the answer is wrong or inaccurate or incomplete? How much time do I have to check? How thorough should I be? I imagine this calculus isn't really that different for most people. Or is it? As for your example, I believe it. But I also feel it's a rather outlier example involving image comprehension of an obscure factoid. That isn't typical of how I use LLMs which is mostly as text-based question answering engines and not what I had in mind when writing the comment. I guess LLMs for image comprehension need a much higher level of skepticism.
    - dalke 7mo ago
      Well, in my case going to a Peanuts comic and looking at hands was pretty easy, and didn't involve any questions about negative environment or labor consequences, the massive hammering of web sites to gather data, centralization of power, and the like. Like, "!w Peanuts" in my search bar, look at the image, and count fingers. "a rather outlier example" You wrote that you use AI to find "obscure connections" - aren't those all by definition outliers? "mostly as text-based question" I just now asked Google AI "how many fingers are on charlie brown's hand?" It replied "In the Peanuts comic strip, Charlie Brown and the rest of the gang are traditionally drawn with four fingers (or three fingers and a thumb) on each hand." No image comprehension, exactly as you had in mind. And completely false. And that's from a training corpus which almost certainly includes statements that the kids are drawn with 5 fingers, since I confirmed that info on TVTropes and Reddit comments, like https://www.reddit.com/r/pics/comments/swod8/charlie_brown_has_4_fingers_and_a_thumb/ https://www.reddit.com/r/pics/comments/swod8/charlie_brown_h... .

menaerus 7mo ago

It's wrong to assume incompetence, which is what you did to a comment which displays much deeper chain of thought about the subject. A more proper way of doing it would be to reflect over your own opinions and critically assess them, as the comment points that out. To be more specific, what makes you think that the person you're replying to is not aware that LLMs can give false information, and is not taking that into account?

lovelearning 7mo ago

You're right that LLMs do spit out false information or wrong knowledge. I've experienced them too. But a redeeming quality is that we can ask the same LLM to fact check its own answer step by step in real time with little effort. They often identify their own hallucinations and reduce the probability of retaining that mistake in the rest of the conversation. This isn't easy with human sources. The effort to fact check without LLMs or ask the sources to fact check themselves are both higher. So it's often not done at all. We also often ignore subtle but very common biases in human media sources [1], which create other types of errors like omissions and euphemisms which have been no less harmful than LLM hallucinations. The case of the Iraqi WMDs of Iraq and the NYT's dispersal of that disinfo, for example [2]. Regarding valuable information and rehashing, we probably shouldn't equate between all the things LLMs can do, and AI-generated articles. The quality of the latter may be entirely due to the lack of interest, attention, and cost concerns of whoever generated the article. Anecdotally, I have often found valuable knowledge and obscure connections by using deep research tools with careful prompts. Lastly, if you're frequently finding something new from human-written sources, and LLMs are being trained on most of those same sources, isn't it logical that the latter will also likely output that same information? This is why I feel human and AI sources are probably best used as complementary tools. Neither set of sources are perfect but each set has its strengths. By using both, we can get closer to an objective truth than using only one of them. [1]: https://gipplab.uni-goettingen.de/wp-content/uploads/2022/04/181122_mediabiasprocess-800x602.png https://gipplab.uni-goettingen.de/wp-content/uploads/2022/04... [2]: https://www.theguardian.com/media/2004/may/26/pressandpublishing.usnews https://www.theguardian.com/media/2004/may/26/pressandpublis...

dalke 7mo ago
The effort to fact check with LLMs is also high. Here's one from a few days ago. Someone used AI to generate an image in the style of a Charles Schulz Peanuts cartoon. Someone else observed that there were 5 fingers on the characters, and quoted as Google AI as saying “Charlie Brown, along with other Peanuts characters, is generally depicted with four fingers on each hand (three fingers and one thumb) ...” Yet if you go to the Wikipedia entry at https://en.wikipedia.org/wiki/Peanuts https://en.wikipedia.org/wiki/Peanuts you'll see the kids have 5 fingers. Or take a look at the actual cartoons. Or read the TVTropes entry https://tvtropes.org/pmwiki/pmwiki.php/Main/FourFingeredHands https://tvtropes.org/pmwiki/pmwiki.php/Main/FourFingeredHand... under "Comic Strips". Fact checking this with human sources is easy and not ambiguous. While LLMs are being trained that many cartoon characters only have a thumb and three fingers - it is a trope for a reason - so isn't it logical for LLMs to give the wrong answer for a comic where the human characters are actually drawn with 5 fingers? My experience with LLMs is they keep getting things wrong, when details matter. Do you ask the LLM to fact check everything? (In which case, why isn't that part of the standard prompt?) Or do you only ask to fact check things where you are unsure about the answer? (In which case, is it the algorithm telling you what you want to hear?) When do you stop the fact checking?
- lovelearning 7mo ago
  > When do you stop the fact checking? Exactly the same calculus as fact checking anything else from any other source. What are the social/economic/ethical consequences to me if the answer is wrong or inaccurate or incomplete? How much time do I have to check? How thorough should I be? I imagine this calculus isn't really that different for most people. Or is it? As for your example, I believe it. But I also feel it's a rather outlier example involving image comprehension of an obscure factoid. That isn't typical of how I use LLMs which is mostly as text-based question answering engines and not what I had in mind when writing the comment. I guess LLMs for image comprehension need a much higher level of skepticism.
  - dalke 7mo ago
    Well, in my case going to a Peanuts comic and looking at hands was pretty easy, and didn't involve any questions about negative environment or labor consequences, the massive hammering of web sites to gather data, centralization of power, and the like. Like, "!w Peanuts" in my search bar, look at the image, and count fingers. "a rather outlier example" You wrote that you use AI to find "obscure connections" - aren't those all by definition outliers? "mostly as text-based question" I just now asked Google AI "how many fingers are on charlie brown's hand?" It replied "In the Peanuts comic strip, Charlie Brown and the rest of the gang are traditionally drawn with four fingers (or three fingers and a thumb) on each hand." No image comprehension, exactly as you had in mind. And completely false. And that's from a training corpus which almost certainly includes statements that the kids are drawn with 5 fingers, since I confirmed that info on TVTropes and Reddit comments, like https://www.reddit.com/r/pics/comments/swod8/charlie_brown_has_4_fingers_and_a_thumb/ https://www.reddit.com/r/pics/comments/swod8/charlie_brown_h... .