3 ms·

> In all those cases my summary could be right or could be wrong. Well that's incredibly nihilistic. Whether the summary is correct or not matters a great deal

> In all those cases my summary could be right or could be wrong. Well that's incredibly nihilistic. Whether the summary is correct or not matters a great deal! And if someone I knew said they read a book, even a very obscure one, and then summarized it to me, I'd have great confidence that they would get such simple facts as "who are the characters" and "what are the major plot points" correct. But ChatGPT? Who the hell knows? You can't trust a thing it says, especially about obscure topics. The summary is useless if you have to do a bunch of verification to see if any of it is even true, a problem that summaries even by moderately competent human writers don't have!

thwarted 3y ago
if someone I knew said they read a book, even a very obscure one, and then summarized it to me, I'd have great confidence that they would get such simple facts as "who are the characters" and "what are the major plot points" correct. People, especially people you know, have reputations, based on history and experience that others have dealing with them. People can be known as liars, and anything they say is colored by such a reputation. Humans have language idioms for communicating about and dealing with such people too, phrases like "take anything that person says with a grain of salt". Look at how George Santos' history of lying about his own experience is being dealt with. ChatGPT can be (is?) the same, and it has a bad reputation for truth telling. And LLMs' reputation is not necessarily getting better in this regard. The problem is that many people attribute output that came from a machine to be of higher quality (on whatever axis) than output that came from a human, even a human they personally know and have experience dealing with. This is the same kind of prejudice as any other, or perhaps a more insidious prejudice.
- CydeWeys 3y ago
  Agree 100% with all of this. LLMs have a huge reputation problem; you simply cannot trust what they say because they've been proven time and time again to hallucinate fictional answers. Until that problem is solved I'm struggling to see how they're as useful as people are claiming they are. You know what would be a fun test of integrity -- look up an obscure novel (potentially even the aforementioned one) that you know LLMs consistently hallucinate about because the details aren't in its training set, and then assign an essay about it as an academic assignment. It'll be pretty obvious who's read the book and who merely consulted an LLM because the latter will just be complete gibberish to anyone who actually knows what happens in that novel.

thwarted 3y ago

if someone I knew said they read a book, even a very obscure one, and then summarized it to me, I'd have great confidence that they would get such simple facts as "who are the characters" and "what are the major plot points" correct. People, especially people you know, have reputations, based on history and experience that others have dealing with them. People can be known as liars, and anything they say is colored by such a reputation. Humans have language idioms for communicating about and dealing with such people too, phrases like "take anything that person says with a grain of salt". Look at how George Santos' history of lying about his own experience is being dealt with. ChatGPT can be (is?) the same, and it has a bad reputation for truth telling. And LLMs' reputation is not necessarily getting better in this regard. The problem is that many people attribute output that came from a machine to be of higher quality (on whatever axis) than output that came from a human, even a human they personally know and have experience dealing with. This is the same kind of prejudice as any other, or perhaps a more insidious prejudice.

CydeWeys 3y ago
Agree 100% with all of this. LLMs have a huge reputation problem; you simply cannot trust what they say because they've been proven time and time again to hallucinate fictional answers. Until that problem is solved I'm struggling to see how they're as useful as people are claiming they are. You know what would be a fun test of integrity -- look up an obscure novel (potentially even the aforementioned one) that you know LLMs consistently hallucinate about because the details aren't in its training set, and then assign an essay about it as an academic assignment. It'll be pretty obvious who's read the book and who merely consulted an LLM because the latter will just be complete gibberish to anyone who actually knows what happens in that novel.