6 ms·

What I love is the explanation and the CONFIDENCE LLMs seem to have in giving the wrong answer.

by cowmix 3y ago

stared 3y ago
Ironically, it is the most human aspect of LLMs.
- xanderlewis 3y ago
  I see this sentiment repeated a lot. I can see where it comes from (humans make mistakes too, and are often confidently wrong), but I don't think it's quite the same. LLMs are confidently and repeatedly wrong about very simple things that a human would never be. They're also not even consistent; ask them one thing and then tell them they're wrong: they'll often apologise profusely and flip their answer right around. Tell them once again that they're wrong and they'll flip back again. Ask them about their thought process, and they'll just produce a load of waffle that looks like it describes a thought process but usually doesn't correspond to anything of the sort (and of course it doesn't; if you understand vaguely how these systems work it'd be a miracle if it did so). Since one can invariably take any criticism of LLMs and immediately reply with "but some humans do that too", I don't think such lines of argument say very much. It's the weird combination of superhuman memory, speed and wit and at times utterly subhuman logical consistency that makes it hard to interpret how 'intelligent' LLMs can be said to be. That said, they can still be very useful.
  - stared 3y ago
    I agree, the lack of consistency sets us (i.e. humans vs LLMs) apart. > LLMs are confidently and repeatedly wrong about very simple things that a human would never be. It is a strong assertion. A substantial fraction of the population (maybe even the majority) would repeat phrases they learned. Some are about Earth being 5000 years old, some are political slogans, and some others are based on the knowledge they derived from TV series. > Since one can invariably take any criticism of LLMs and immediately reply with "but some humans do that too", I don't think such lines of argument say very much. On the contrary - it is interesting to see which problems are uniquely LLM (or human) and which others - are similar instances of the same, especially when we benchmark LLM not only versus some idealized human cognition but one of an ordinary person under ordinary conditions.
    - xanderlewis 3y ago
      > It is a strong assertion... I think you may be right if you consider humanity as a whole, but when evaluating LLMs I think we should perhaps hold them to slightly higher standards. It seems silly to compare a system that clearly has an incredible level of knowledge and writing ability to toddlers or completely uneducated remote tribespeople (or even just bigoted people living in otherwise advanced societies). Since LLMs can operate on vastly different timescales to humans, we should probably evaluate them against reasonably knowledgeable humans given several hours and a chance to look back over their work as many times as they like before final submission. Since we’re trying to abstractly compare some sort of computational ability, we should probably give them equal amounts of computational resources/number of steps/clock cycles/etc. Given this setup, I think some of the ‘human mistakes’ we talk about would no longer apply. Comparing systems like this is not an easy (or objective) matter anyway… I’m not claiming this is the only way of thinking about it and that all others are wrong. > it is interesting to see which problems are uniquely LLM (or human) I certainly agree with that. It’s fascinating to be able to observe and study something in real time that appears on the surface to approximate human thought so shockingly well whilst also being so different.
      - stared 3y ago
        It is hard to define a fair comparison between humans and LLMs. Artificial "neurons" (i.e., vector entries + ReLU) have little in common with biological neurons (each one is a computing unit on its own). Data efficiency is crucial, but humans have a lot of pre-trained multi-modal data. For energy efficiency, ML is orders of magnitude more efficient. Regarding the general powers (and weaknesses) of LLMs, I am shocked they work in an Artificial General(ish) Intelligence way. Text generation with GPT2 and GPT3 - sure, it still felt like a very advanced text autocomplete. GPT4 - here, I could have a bet (and lose money) that this level requires some form of reinforcement learning and a two-way interaction with the environment. Sure, there is some RL there, but I assumed something closer to a bot learning to talk with people, running and getting results from code, and looking at data online (for the training!), or maybe even - literally walking with a camera attached. Yes, there is still a lot to do. There is a difference between a Go model beating novices, advanced players, and everyone, including world champions.
  - chasd00 3y ago
    Maybe in this specific circumstance an LLM being wrong is similar to a human being wrong but it's very jarring to a user. For example, imagine if Excel got the math wrong sometimes like a human does. Throughout all of digital history computers were never wrong, they're either programmed correctly or not. I think a lot of business leaders and others hyping the generative AI technology may not fully grasp the nondeterministic nature of an LLM. I work in consulting and have been on calls with senior leadership going on and on about the miracles ($$) about to unfold. However, when I raise the point that you can never really count on the output of an llm being the same or even correct and using one to guess the parameters of an API for example is going to fail eventually with no explanation or fix and how that narrows their usefulness i get the "stop being a wet blanket" talk.
    - stared 3y ago
      A fair point - that one thing is the absolute skill level, and the other is a comparison with prior expectations. The latter may give a positive surprise or deep disappointment.
nottorp 3y ago
In this they're like google search. They have to give you some answer. Google gives you completely unrelated results, while the LLMs just make something up. Twice in the past weeks I tried both ChatGPT and Gemini for a technical question. The true answer was "you can't do it"/"those devices don't have the feature you're asking about". In both cases they made up some answers that looked consistent but were completely bogus.
chasd00 3y ago
being confidently wrong is a big problem a team i'm working with is facing. They have resorted to the usual "this is an AI don't trust it" disclaimer on the interface. But then, if you have to go lookup and double check every answer, what's the point of asking in the first place? I think we'll see a lot of LLM use cases switching from generative responses back to good old fashioned search this year.

What I love is the explanation and the CONFIDENCE LLMs seem to have in giving the wrong answer.

by cowmix 3y ago

What I love is the explanation and the CONFIDENCE LLMs seem to have in giving the wrong answer.

stared 3y ago

Ironically, it is the most human aspect of LLMs.

xanderlewis 3y ago
I see this sentiment repeated a lot. I can see where it comes from (humans make mistakes too, and are often confidently wrong), but I don't think it's quite the same. LLMs are confidently and repeatedly wrong about very simple things that a human would never be. They're also not even consistent; ask them one thing and then tell them they're wrong: they'll often apologise profusely and flip their answer right around. Tell them once again that they're wrong and they'll flip back again. Ask them about their thought process, and they'll just produce a load of waffle that looks like it describes a thought process but usually doesn't correspond to anything of the sort (and of course it doesn't; if you understand vaguely how these systems work it'd be a miracle if it did so). Since one can invariably take any criticism of LLMs and immediately reply with "but some humans do that too", I don't think such lines of argument say very much. It's the weird combination of superhuman memory, speed and wit and at times utterly subhuman logical consistency that makes it hard to interpret how 'intelligent' LLMs can be said to be. That said, they can still be very useful.
- stared 3y ago
  I agree, the lack of consistency sets us (i.e. humans vs LLMs) apart. > LLMs are confidently and repeatedly wrong about very simple things that a human would never be. It is a strong assertion. A substantial fraction of the population (maybe even the majority) would repeat phrases they learned. Some are about Earth being 5000 years old, some are political slogans, and some others are based on the knowledge they derived from TV series. > Since one can invariably take any criticism of LLMs and immediately reply with "but some humans do that too", I don't think such lines of argument say very much. On the contrary - it is interesting to see which problems are uniquely LLM (or human) and which others - are similar instances of the same, especially when we benchmark LLM not only versus some idealized human cognition but one of an ordinary person under ordinary conditions.
  - xanderlewis 3y ago
    > It is a strong assertion... I think you may be right if you consider humanity as a whole, but when evaluating LLMs I think we should perhaps hold them to slightly higher standards. It seems silly to compare a system that clearly has an incredible level of knowledge and writing ability to toddlers or completely uneducated remote tribespeople (or even just bigoted people living in otherwise advanced societies). Since LLMs can operate on vastly different timescales to humans, we should probably evaluate them against reasonably knowledgeable humans given several hours and a chance to look back over their work as many times as they like before final submission. Since we’re trying to abstractly compare some sort of computational ability, we should probably give them equal amounts of computational resources/number of steps/clock cycles/etc. Given this setup, I think some of the ‘human mistakes’ we talk about would no longer apply. Comparing systems like this is not an easy (or objective) matter anyway… I’m not claiming this is the only way of thinking about it and that all others are wrong. > it is interesting to see which problems are uniquely LLM (or human) I certainly agree with that. It’s fascinating to be able to observe and study something in real time that appears on the surface to approximate human thought so shockingly well whilst also being so different.
    - stared 3y ago
      It is hard to define a fair comparison between humans and LLMs. Artificial "neurons" (i.e., vector entries + ReLU) have little in common with biological neurons (each one is a computing unit on its own). Data efficiency is crucial, but humans have a lot of pre-trained multi-modal data. For energy efficiency, ML is orders of magnitude more efficient. Regarding the general powers (and weaknesses) of LLMs, I am shocked they work in an Artificial General(ish) Intelligence way. Text generation with GPT2 and GPT3 - sure, it still felt like a very advanced text autocomplete. GPT4 - here, I could have a bet (and lose money) that this level requires some form of reinforcement learning and a two-way interaction with the environment. Sure, there is some RL there, but I assumed something closer to a bot learning to talk with people, running and getting results from code, and looking at data online (for the training!), or maybe even - literally walking with a camera attached. Yes, there is still a lot to do. There is a difference between a Go model beating novices, advanced players, and everyone, including world champions.
- chasd00 3y ago
  Maybe in this specific circumstance an LLM being wrong is similar to a human being wrong but it's very jarring to a user. For example, imagine if Excel got the math wrong sometimes like a human does. Throughout all of digital history computers were never wrong, they're either programmed correctly or not. I think a lot of business leaders and others hyping the generative AI technology may not fully grasp the nondeterministic nature of an LLM. I work in consulting and have been on calls with senior leadership going on and on about the miracles ($$) about to unfold. However, when I raise the point that you can never really count on the output of an llm being the same or even correct and using one to guess the parameters of an API for example is going to fail eventually with no explanation or fix and how that narrows their usefulness i get the "stop being a wet blanket" talk.
  - stared 3y ago
    A fair point - that one thing is the absolute skill level, and the other is a comparison with prior expectations. The latter may give a positive surprise or deep disappointment.

nottorp 3y ago

In this they're like google search. They have to give you some answer. Google gives you completely unrelated results, while the LLMs just make something up. Twice in the past weeks I tried both ChatGPT and Gemini for a technical question. The true answer was "you can't do it"/"those devices don't have the feature you're asking about". In both cases they made up some answers that looked consistent but were completely bogus.

chasd00 3y ago

being confidently wrong is a big problem a team i'm working with is facing. They have resorted to the usual "this is an AI don't trust it" disclaimer on the interface. But then, if you have to go lookup and double check every answer, what's the point of asking in the first place? I think we'll see a lot of LLM use cases switching from generative responses back to good old fashioned search this year.