4 ms·

The biggest gripe I have with chat GPT though is that I have to "trust" that ChatGPT is correct, like blindly trusting a colleague who thinks they know everythi

The biggest gripe I have with chat GPT though is that I have to "trust" that ChatGPT is correct, like blindly trusting a colleague who thinks they know everything. Yep. ChatGPT will sometimes happily assert something that is simply false. And in some of those cases it appears to be quite confident in saying so and doesn't hedge or offer any qualifiers. I found one where if you ask it a question in this form: Why do people say that drinking Ardbeg is like getting punched in the face by William Wallace? You'll get back something that includes something like this: People often say that drinking Ardbeg is like getting punched in the face by William Wallace. Ardbeg is a brand of Scottish whiskey <blah, blah>. William Wallace was a Scottish <blah, blah>. People say "drinking Ardbeg is like getting punched in the face by William Wallace as a metaphor for the taste of Ardbeg being something punchy and powerful." <other stuff omitted> And the thing is, inasmuch as anybody has ever said that, or would ever say that, the given explanation is plausible. It is a metaphor. The problem is, it's not true that "people often say that drinking Ardbeg is like getting punched in the face by William Wallace." At least not to the best of my knowledge. I know exactly one person who said that to me once. Maybe he made it up himself, maybe he got it from somebody, but I see no evidence that the expression is commonly used though. But it doesn't matter. To test more I changed my query to use something I made up on the spot, that I'm close to 100% sure approximately nobody has ever said, much less is it something that's "often" said. Change it to: Why do people say that drinking Ardbeg is like getting shagged by Bonnie Prince Charlie? and you get the same answer, modulo the details about who Bonnie Prince Charlie was. And if you change it to: Why do people say that drinking vodka is like getting shagged by Joseph Stalin? You again get almost the same answer, modulo some details about vodka and Stalin. In all three cases, you get the confident assertion that "people often say X". The point of all this not to discredit ChatGPT of course. I find it tremendously impressive and definitely think it's a useful tool. And for at least one query I tried, it was MUCH better at finding me an answer than trying to use Google. I just shared the above to emphasize the point about being careful of trusting the responses from ChatGPT. The one that ChatGPT easily beat Google on, BTW, was this (paraphrased from memory, as ChatGPT is "at capacity" at the moment so I can't get in to copy & paste) What college course is the one that typically covers infinite product series? To which ChatGPT quickly replied "A course on Advanced Calculus or Real Analysis". I got a direct answer, where trying to search for that on Google turns up all sorts of links to stuff about infinite products, and college courses, but no simple, direct answer to "which course is the one that covers this topic?" Now the question is, is that answer correct? Hmmm... :-)

cpleppert 4y ago
When you use the prompt "Why do people say that drinking Ardbeg is like getting punched in the face by William Wallace?" you are prompting it to use the fact you provided as part of its response. If you instead ask directly it will say "I'm not aware of any specific claims that drinking Ardbeg is like getting punched in the face by William Wallace."
- mindcrime 4y ago
  True. Ideally though, I think the response to the first prompt should be either something like: "There is no evidence that people actually say that..." or "If we assume that people say that (not established) this is probably what they mean ..." or something along those lines. Still, it's a minor nit, and my point was not, as I said, to discredit ChatGPT. I find it impressive and would even describe it as "intelligent" to a point. But clearly there are limits to its "intelligence" and ability to spit out fully correct answers all the time.

cpleppert 4y ago

When you use the prompt "Why do people say that drinking Ardbeg is like getting punched in the face by William Wallace?" you are prompting it to use the fact you provided as part of its response. If you instead ask directly it will say "I'm not aware of any specific claims that drinking Ardbeg is like getting punched in the face by William Wallace."

mindcrime 4y ago
True. Ideally though, I think the response to the first prompt should be either something like: "There is no evidence that people actually say that..." or "If we assume that people say that (not established) this is probably what they mean ..." or something along those lines. Still, it's a minor nit, and my point was not, as I said, to discredit ChatGPT. I find it impressive and would even describe it as "intelligent" to a point. But clearly there are limits to its "intelligence" and ability to spit out fully correct answers all the time.