3 ms·
>> "Hallucination" is a terrible term to apply to LLMs. LLMs ONLY produce hallucinations. Seems pedantic. If you define "hallucinations" to include correct res
by zug_zug 3y ago
>> "Hallucination" is a terrible term to apply to LLMs. LLMs ONLY produce hallucinations.
Seems pedantic. If you define "hallucinations" to include correct responses, then nobody cares whether something is a "hallucination," the only important question is -- is it right?
And the answer is yes, across a huge number of metrics [at least for gpt-4]. It'd beat you at jeopardy, it'd beat you at chess, it'd beat you at an AP-biology exam, it'd beat you at a leet code competition.
Enough with semantics.
- willsmith72 3y agoYes, gpt4 will beat 99.9% of humans on 99.9% of subjects. But it's terrible at leetcode. For "easy" questions sure it's great, and even some medium, but often at a medium level and almost always at a hard level, it'll give you a classic confident-but-terribly-wrong response. Prompting it to look at its mistakes will usually lead to the "oops yes that was wrong, here's the corrected version:", which is also wrong.
- zug_zug 3y agoMaybe relative to us on this forum it's not so impressive but that's a rare skill that we've dedicated years to honing. What % of the general population can leetcode better than chatgtp? 5%? 1%?
- willsmith72 3y agodefinitely less than 1%, and we're in agreement in general i only take issue with leetcode as an example. i would say gpt4 is great at coding, and also api and db schema design. but leetcode is specific coding tasks with clear right and wrong answers, where getting 80% of a right answer is still wrong. gpt4 and my mum have roughly the same chance of getting the right answer to a hard leetcode problem, that's my only point