3 ms·

So what? You have no way to know for sure if the human you ask the same question, does either. The question that started this thread was related to verifiabil

by ta8645 2y ago

So what? You have no way to know for sure if the human you ask the same question, does either. The question that started this thread was related to verifiability. And i still think it is a spurious complaint, given that we have exactly the same limitations when dealing with any human agent.

wizzwizz4 2y ago
The problem of other minds is no reason to throw everything out the window. Humans are capable of being conscious of their reasoning processes; token-at-a-time predictive text models wired up as chatbots aren't capable of it. Your choice is between a possibly-mistaken, possibly-lying human, and a 100%-definitely incapable computer program. You don't know either "for sure", but you don't know that the external world exists "for sure" either. It's an insight-free observation, and shouldn't be the focus of anyone's decision-making.
- ta8645 2y ago
  You've made some interesting points, which are debatable, for sure. But you've failed to address the question being asked about "verifiability".
- vanviegen 2y ago
  When you ask an LLM to carefully reason step by step before arriving at its answer, that seems pretty much the same as conscious reasoning to me. Of course, when asked to justify a gut reaction after the fact, it will just come up with something that sounds plausible (and may or may not be true). Just like humans do.
bluefirebrand 2y ago
> And i still think it is a spurious complaint, given that we have exactly the same limitations when dealing with any human agent We're not talking about an LLM that is trying to do the job of a human, here We're talking about an LLM that is trying to give authoritative answers to any question typed into the Google search bar It's already well past the scale that humans could handle Talking about human shortcomings when discussing LLMs is a red herring at best, or some kind of deliberate goalpost shifting at worst
- ta8645 2y ago
  Nothing of the sort. I'm trying to understand why anyone cares about formal verifiability in this context, since it's not something we rely on when asking humans to answer questions for us. We evaluate any answer we get without such mathematical proofs, and instead simply judge the answer we're given on its fit and usefulness. Anyone who doubts the usefulness of even these nascent LLMs is fooling themselves. The proof is in the pudding, they already do a great job, even with all their obvious limitations.
  - bluefirebrand 2y ago
    > since it's not something we rely on when asking humans to answer questions for us Because we interact with computers (which includes LLMs) differently than we do with humans and we hold them to higher standards Ironically, Google played a large part in this, delivering high quality results to us with ease for many years. At one point Google was the standard for finding high quality information
    - ta8645 2y ago
      Shrug. Seems like clutching pearls to me. People seem to have an emotional reaction and obsess on the aspects that differentiate human cognition from LLMs. But that is a lot of wasted energy. To the extent that anyone avoids employing these technologies, they will be at a disadvantage to those who do; because these tools just work. Already. Today. There isn't even room for debate on that issue. Again, the proof is in the pudding. These systems are already successfully, usefully, and correctly answering millions of questions a day. They have failure modes where they produce substandard or even flat out incorrect answers too. They're far from perfect, but they're still incredible tools, even without waiting for the improvements that are sure to come.
  - figassis 2y ago
    The reason verifiability is important is because humans can be incentivized to be truthful and factual. We know we lie, but we also know we can produce verifiable information, and we prefer this to lies, so when it matters, we make the cost to lying high enough that we can reasonably expect that they will not try to deceive (for example by committing perjury, or fabricating research data). We know it still happens, but it’s not widespread and we can adjust the rules, definitions and cost to adapt. An LLM does not have such real world limitations. It will hallucinate nonstop and then create layers of gaslighting explanations to its hallucinations. The problem is that you absolutely must be a domain expert at the LLM’s topic or always go find the facts elsewhere to verify (then why use an LLM?). So a company like Google using an LLM, is not providing information, it’s doing the opposite. It is making it more difficult and time consuming to find information. But it is then hiding their responsibility behind the model. “We didn’t present bad info, our model did, we’re sorry it told you to turn your recipe into poison…models amirite?” A human doing that could likely face some consequences.
Dylan16807 2y ago
> You have no way to know for sure if the human you ask the same question, does either. The human might lie, but they generally don't. An LLM is always confabulating when it explains how it reached a conclusion, because that information was discarded as soon as it picked a word. The limitations are not in the same ballpark.
- TeMPOraL 2y ago
  The human may also be... wrong. Saying things that "feel right", except this one time they're factually wrong. A human can explain their reasoning step by step, if the original reasoning was a System 2, formal, step-by-step process in the first place; otherwise, they're just making shit up after the fact, which feels right, but may or may not be correct (see also the previous paragraph). Note that it's very rare anyone has an interaction with a human that uses this mode of reasoning - it's unnecessary except in special circumstances, usually math-heavy.
- vidarh 2y ago
  We have no evidence that humans can even know, much less that they generally don't. And we do have evidence that there are situations where the brain will readily construct an explanation after the fact which can't possibly be true (experiments with split brain patients where researchers tricked one brain half into thinking the other half, and so the brain as a whole, had made a decision while the action was taken by the researchers, and made it explain how it has made the decision) There is no basis for claiming to know that people don't usually make up explanations like this other than when e.g. breaking the process apart and writing it down step by step during. But even then individual decisions are "suspect".

by ta8645 2y ago

wizzwizz4 2y ago

The problem of other minds is no reason to throw everything out the window. Humans are capable of being conscious of their reasoning processes; token-at-a-time predictive text models wired up as chatbots aren't capable of it. Your choice is between a possibly-mistaken, possibly-lying human, and a 100%-definitely incapable computer program. You don't know either "for sure", but you don't know that the external world exists "for sure" either. It's an insight-free observation, and shouldn't be the focus of anyone's decision-making.

ta8645 2y ago
You've made some interesting points, which are debatable, for sure. But you've failed to address the question being asked about "verifiability".
vanviegen 2y ago
When you ask an LLM to carefully reason step by step before arriving at its answer, that seems pretty much the same as conscious reasoning to me. Of course, when asked to justify a gut reaction after the fact, it will just come up with something that sounds plausible (and may or may not be true). Just like humans do.

bluefirebrand 2y ago

> And i still think it is a spurious complaint, given that we have exactly the same limitations when dealing with any human agent We're not talking about an LLM that is trying to do the job of a human, here We're talking about an LLM that is trying to give authoritative answers to any question typed into the Google search bar It's already well past the scale that humans could handle Talking about human shortcomings when discussing LLMs is a red herring at best, or some kind of deliberate goalpost shifting at worst

ta8645 2y ago
Nothing of the sort. I'm trying to understand why anyone cares about formal verifiability in this context, since it's not something we rely on when asking humans to answer questions for us. We evaluate any answer we get without such mathematical proofs, and instead simply judge the answer we're given on its fit and usefulness. Anyone who doubts the usefulness of even these nascent LLMs is fooling themselves. The proof is in the pudding, they already do a great job, even with all their obvious limitations.
- bluefirebrand 2y ago
  > since it's not something we rely on when asking humans to answer questions for us Because we interact with computers (which includes LLMs) differently than we do with humans and we hold them to higher standards Ironically, Google played a large part in this, delivering high quality results to us with ease for many years. At one point Google was the standard for finding high quality information
  - ta8645 2y ago
    Shrug. Seems like clutching pearls to me. People seem to have an emotional reaction and obsess on the aspects that differentiate human cognition from LLMs. But that is a lot of wasted energy. To the extent that anyone avoids employing these technologies, they will be at a disadvantage to those who do; because these tools just work. Already. Today. There isn't even room for debate on that issue. Again, the proof is in the pudding. These systems are already successfully, usefully, and correctly answering millions of questions a day. They have failure modes where they produce substandard or even flat out incorrect answers too. They're far from perfect, but they're still incredible tools, even without waiting for the improvements that are sure to come.
- figassis 2y ago
  The reason verifiability is important is because humans can be incentivized to be truthful and factual. We know we lie, but we also know we can produce verifiable information, and we prefer this to lies, so when it matters, we make the cost to lying high enough that we can reasonably expect that they will not try to deceive (for example by committing perjury, or fabricating research data). We know it still happens, but it’s not widespread and we can adjust the rules, definitions and cost to adapt. An LLM does not have such real world limitations. It will hallucinate nonstop and then create layers of gaslighting explanations to its hallucinations. The problem is that you absolutely must be a domain expert at the LLM’s topic or always go find the facts elsewhere to verify (then why use an LLM?). So a company like Google using an LLM, is not providing information, it’s doing the opposite. It is making it more difficult and time consuming to find information. But it is then hiding their responsibility behind the model. “We didn’t present bad info, our model did, we’re sorry it told you to turn your recipe into poison…models amirite?” A human doing that could likely face some consequences.

Dylan16807 2y ago

> You have no way to know for sure if the human you ask the same question, does either. The human might lie, but they generally don't. An LLM is always confabulating when it explains how it reached a conclusion, because that information was discarded as soon as it picked a word. The limitations are not in the same ballpark.

TeMPOraL 2y ago
The human may also be... wrong. Saying things that "feel right", except this one time they're factually wrong. A human can explain their reasoning step by step, if the original reasoning was a System 2, formal, step-by-step process in the first place; otherwise, they're just making shit up after the fact, which feels right, but may or may not be correct (see also the previous paragraph). Note that it's very rare anyone has an interaction with a human that uses this mode of reasoning - it's unnecessary except in special circumstances, usually math-heavy.
vidarh 2y ago
We have no evidence that humans can even know, much less that they generally don't. And we do have evidence that there are situations where the brain will readily construct an explanation after the fact which can't possibly be true (experiments with split brain patients where researchers tricked one brain half into thinking the other half, and so the brain as a whole, had made a decision while the action was taken by the researchers, and made it explain how it has made the decision) There is no basis for claiming to know that people don't usually make up explanations like this other than when e.g. breaking the process apart and writing it down step by step during. But even then individual decisions are "suspect".