3 ms·

Problem with ChatGPT isn't that they get facts wrong, but that they're what the categorical name suggests, large language models. At one point I came across th

Problem with ChatGPT isn't that they get facts wrong, but that they're what the categorical name suggests, large language models. At one point I came across this series of "are CJK languages related" questions in Quora with cached ChatGPT responses[1], all grammatically correct and very natural, largely turboencabulators, sometimes contradicting even within a single response. Podcasters? They're not _this_ inconsistent. 1: https://imgur.com/a/7tlNDll https://imgur.com/a/7tlNDll

Last5Digits 3y ago
Be aware that you're talking about Quora's implementation of ChatGPT here. As far as I know, the cached answers were generated with an incredibly outdated version, which is definitely not indicative of its current quality. Even worse, I think they actually prime it with answers already posted on the thread, or even just related threads. For example, one of the answers to the first question mentions the same Altaic root as ChatGPTs answer, and I've found multiple people that are seeing their own rephrased answers in the response. If you preprompt ChatGPT with questionable data, then the answer quality will be massively degraded. I've noticed many times now that Bing will rephrase incorrect information or construct a very shallow summary out of unrelated articles when internet searches are allowed, but is able to generate a cohesive and detailed summary when they're disabled. Throwing random answers - some contradicting each other and some talking about subtly different aspects of the topic - into a session without further guidance just isn't a great idea.
kaszanka 3y ago
What's the issue with the answer for "Are Korean and Chinese mutually intelligible"? The highlighted part is definitely true, at least.
- numpad0 3y ago
  That part is correct, but not consistent with others. My problem is not that GPTs are too often wrong, it's that they always prioritize syntax over facts since they are _language_ models. The sentence "Colorless green ideas" always make more sense to LLM than "Water is wet", simply because the latter is syntactically invalid, and that would be problematic for many use cases including Podcast replacement. Sometimes us humans want AI to say "water is definitively wet", and that has been attempted by forcing LLM to accept that factoids are more syntactically correct, but that isn't a solution and it's still an architectural problem for these pseudo-AGI apps.

Last5Digits 3y ago

Be aware that you're talking about Quora's implementation of ChatGPT here. As far as I know, the cached answers were generated with an incredibly outdated version, which is definitely not indicative of its current quality. Even worse, I think they actually prime it with answers already posted on the thread, or even just related threads. For example, one of the answers to the first question mentions the same Altaic root as ChatGPTs answer, and I've found multiple people that are seeing their own rephrased answers in the response. If you preprompt ChatGPT with questionable data, then the answer quality will be massively degraded. I've noticed many times now that Bing will rephrase incorrect information or construct a very shallow summary out of unrelated articles when internet searches are allowed, but is able to generate a cohesive and detailed summary when they're disabled. Throwing random answers - some contradicting each other and some talking about subtly different aspects of the topic - into a session without further guidance just isn't a great idea.

kaszanka 3y ago

What's the issue with the answer for "Are Korean and Chinese mutually intelligible"? The highlighted part is definitely true, at least.

numpad0 3y ago
That part is correct, but not consistent with others. My problem is not that GPTs are too often wrong, it's that they always prioritize syntax over facts since they are _language_ models. The sentence "Colorless green ideas" always make more sense to LLM than "Water is wet", simply because the latter is syntactically invalid, and that would be problematic for many use cases including Podcast replacement. Sometimes us humans want AI to say "water is definitively wet", and that has been attempted by forcing LLM to accept that factoids are more syntactically correct, but that isn't a solution and it's still an architectural problem for these pseudo-AGI apps.