5 ms·

How exactly can there be "truthfulness" in humans, say? After all, if a human was taught in school all his life that the capital of Connecticut is Moscow...

ben_w 2y ago
I agree that humans and AI are in the same boat here. It's valid to take either position, that both can be aware of truth or that neither can be, and there has been a lot of philosophical debate about this specific topic with humans since well before even mechanical computers were invented. Plato's cave comes to mind.
juliushuijnk 2y ago
You are not disproving the point.
- recursive 2y ago
  If truthfulness doesn't exist at all, then it's meaningless to say that LLMs don't have any data regarding it.
stoniejohnson 2y ago
Humans are not isolated nodes, we are more like a swarm, understanding reality via consensus. The situation you described is possible, but would require something like a subverting effort of propaganda by the state. Inferring truth about a social event in a social situation, for example, requires a nuanced set of thought processes and attention mechanisms. If we had a swarm of LLMs collecting a variety of data from a variety of disparate sources, where the swarm communicates for consensus, it would be very hard to convince them that Moscow is in Connecticut. Unfortunately we are still stuck in monolithic training run land.
- ben_w 2y ago
  Even monolithic training runs take sources more disparate than any human has the capacity to consume. Also, given the lack of imagination everyone has with naming places, I had to check: https://en.wikipedia.org/wiki/Moscow_(disambiguation) https://en.wikipedia.org/wiki/Moscow_(disambiguation)
  - stoniejohnson 2y ago
    I was responding to the idea that an LLM would believe (regurgitate) untrue things if you pretrained them on untrue things. I wasn't making a claim about SOTA models with gigantic training corpora.
- genrilz 2y ago
  We kinda do have LLMs in a swarm configuration though. Currently LLMs training data, which includes all of the non RAG facts they know, come from the swarm that is humans. As LLM outputs seep into the internet, older generations effectively start communicating with newer generations. This last bit is not a great thing though, as LLMs don't have the direct experience needed to correct factual errors about the external world. Unfortunately we care about the external world, and want them to make accurate statements about it. It would be possible for LLMs to see inconsistencies across or within sources, and try to resolve those. If perfect, then this would result in a self-consistent description of some world, it just wouldn't necessarily be ours.
  - stoniejohnson 2y ago
    I get where you are coming from, and it is definitely an interesting thought! I do think it is an extremely inefficient way to have a swarm (e.g. across time through training data) and it would make more sense to solve the pretraining problem (to connect them to the external world as you pointed out) and actually have multiple LLMs in a swarm at the same time.
- FeepingCreature 2y ago
  > Humans are not isolated nodes, we are more like a swarm, understanding reality via consensus. > The situation you described is possible, but would require something like a subverting effort of propaganda by the state. Great! LLMs are fed from the same swarm.
  - stoniejohnson 2y ago
    I was responding to the back and forth of: > If you pretrained an LLM with data saying Moscow is the capital of Connecticut it would think that is true. > Well so would a human! But humans aren't static weights, we update continuously, and we arrive at consensus via communication as we all experience different perspectives. You can fool an entire group through propaganda, but there are boundless historical examples of information making its way in through human communication to overcome said propaganda.
    - ben_w 2y ago
      The main reason for keeping AI static is to allow them to be certified or rolled back (and possibly that the companies can make more money selling fine tuning) — it's not an innate truth of the design or the maths.
      - stoniejohnson 2y ago
        While those are good reasons to keep the weights static from a business perspective, they are not the only reasons, especially when serving SOTA models at the scale of some of the major shops today. Continual/online learning is still an area of active research.
- anon291 2y ago
  Ask the LLM what it thinks of tianenmen and we will understand what truth really means.
deleted 2y ago
[deleted]
RandomLensman 2y ago
There isn't necessarily in humans either, but why build machines that just perpetuate human flaws: Would we want calculators that miscalculate a lot or cars that cannot be faster than humans?
- famouswaffles 2y ago
  What exactly do you imagine is the alternative ? To build generally intelligent machines without flaws ? Where does that exist ? In...ah that's right. It doesn't except in our fiction and in our imaginations. And it's not for a lack of trying. Logic cannot even handle Narrow Intelligence that deals with parsing the real world (Speech/Image Recognition, Classification, Detection etc). But those are flawed and mis-predict so why build them ? Because they are immensely useful, flaws or no.
  - RandomLensman 2y ago
    Why should there not be, for example, reasoning machines - do we know there is no universal method for reasoning? Having deeply flawed machines in the sense that they perform their tasks regularly poorly seems like an odd choice to pursue.
    - famouswaffles 2y ago
      What is a reasoning machine though ? And why is there an assumption that one can exist without flaws? It's not like any of the natural examples exist this way. How would you even navigate the real world without the flexibility to make mistakes ? I'm not saying people shouldn't try but you need to be practical. I'll take the General Intelligence with flaws over the fictional one without any day. >Having deeply flawed machines in the sense that they perform their tasks regularly poorly seems like an odd choice to pursue. State of the art ANNs are generally mostly right though. Even LLMs are mostly right, that's why hallucinations are particularly annoying.
      - RandomLensman 2y ago

ben_w 2y ago

I agree that humans and AI are in the same boat here. It's valid to take either position, that both can be aware of truth or that neither can be, and there has been a lot of philosophical debate about this specific topic with humans since well before even mechanical computers were invented. Plato's cave comes to mind.

juliushuijnk 2y ago

You are not disproving the point.

recursive 2y ago
If truthfulness doesn't exist at all, then it's meaningless to say that LLMs don't have any data regarding it.

stoniejohnson 2y ago

Humans are not isolated nodes, we are more like a swarm, understanding reality via consensus. The situation you described is possible, but would require something like a subverting effort of propaganda by the state. Inferring truth about a social event in a social situation, for example, requires a nuanced set of thought processes and attention mechanisms. If we had a swarm of LLMs collecting a variety of data from a variety of disparate sources, where the swarm communicates for consensus, it would be very hard to convince them that Moscow is in Connecticut. Unfortunately we are still stuck in monolithic training run land.

ben_w 2y ago
Even monolithic training runs take sources more disparate than any human has the capacity to consume. Also, given the lack of imagination everyone has with naming places, I had to check: https://en.wikipedia.org/wiki/Moscow_(disambiguation) https://en.wikipedia.org/wiki/Moscow_(disambiguation)
- stoniejohnson 2y ago
  I was responding to the idea that an LLM would believe (regurgitate) untrue things if you pretrained them on untrue things. I wasn't making a claim about SOTA models with gigantic training corpora.
genrilz 2y ago
We kinda do have LLMs in a swarm configuration though. Currently LLMs training data, which includes all of the non RAG facts they know, come from the swarm that is humans. As LLM outputs seep into the internet, older generations effectively start communicating with newer generations. This last bit is not a great thing though, as LLMs don't have the direct experience needed to correct factual errors about the external world. Unfortunately we care about the external world, and want them to make accurate statements about it. It would be possible for LLMs to see inconsistencies across or within sources, and try to resolve those. If perfect, then this would result in a self-consistent description of some world, it just wouldn't necessarily be ours.
- stoniejohnson 2y ago
  I get where you are coming from, and it is definitely an interesting thought! I do think it is an extremely inefficient way to have a swarm (e.g. across time through training data) and it would make more sense to solve the pretraining problem (to connect them to the external world as you pointed out) and actually have multiple LLMs in a swarm at the same time.
FeepingCreature 2y ago
> Humans are not isolated nodes, we are more like a swarm, understanding reality via consensus. > The situation you described is possible, but would require something like a subverting effort of propaganda by the state. Great! LLMs are fed from the same swarm.
- stoniejohnson 2y ago
  I was responding to the back and forth of: > If you pretrained an LLM with data saying Moscow is the capital of Connecticut it would think that is true. > Well so would a human! But humans aren't static weights, we update continuously, and we arrive at consensus via communication as we all experience different perspectives. You can fool an entire group through propaganda, but there are boundless historical examples of information making its way in through human communication to overcome said propaganda.
  - ben_w 2y ago
    The main reason for keeping AI static is to allow them to be certified or rolled back (and possibly that the companies can make more money selling fine tuning) — it's not an innate truth of the design or the maths.
    - stoniejohnson 2y ago
      While those are good reasons to keep the weights static from a business perspective, they are not the only reasons, especially when serving SOTA models at the scale of some of the major shops today. Continual/online learning is still an area of active research.
anon291 2y ago
Ask the LLM what it thinks of tianenmen and we will understand what truth really means.

deleted 2y ago

[deleted]

RandomLensman 2y ago

There isn't necessarily in humans either, but why build machines that just perpetuate human flaws: Would we want calculators that miscalculate a lot or cars that cannot be faster than humans?

famouswaffles 2y ago
What exactly do you imagine is the alternative ? To build generally intelligent machines without flaws ? Where does that exist ? In...ah that's right. It doesn't except in our fiction and in our imaginations. And it's not for a lack of trying. Logic cannot even handle Narrow Intelligence that deals with parsing the real world (Speech/Image Recognition, Classification, Detection etc). But those are flawed and mis-predict so why build them ? Because they are immensely useful, flaws or no.
- RandomLensman 2y ago
  Why should there not be, for example, reasoning machines - do we know there is no universal method for reasoning? Having deeply flawed machines in the sense that they perform their tasks regularly poorly seems like an odd choice to pursue.
  - famouswaffles 2y ago
    What is a reasoning machine though ? And why is there an assumption that one can exist without flaws? It's not like any of the natural examples exist this way. How would you even navigate the real world without the flexibility to make mistakes ? I'm not saying people shouldn't try but you need to be practical. I'll take the General Intelligence with flaws over the fictional one without any day. >Having deeply flawed machines in the sense that they perform their tasks regularly poorly seems like an odd choice to pursue. State of the art ANNs are generally mostly right though. Even LLMs are mostly right, that's why hallucinations are particularly annoying.
    - RandomLensman 2y ago