4 ms·

Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omi

Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already. If you're worried that there is more text on the internet saying that the Earth is flat than there is text saying that the Earth is round, then I understand where you are coming from, but then how would ANYONE come to believe that the Earth is round on their own? By learning how to reason. Edit: My point is that if the LLM can't weed through bullshit properly then how can we expect it to get anything right at all? We can't just feed it a continuously updated list of facts that are true and facts that are false until the end of time.

eropple 3y ago
> Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already. Because it's not thinking, it's looking at probabilities. If you don't prune branches, you've created a disinformation radiator if that information is sufficiently populous within your corpus to be a pretty-good-bet. The best analogy I've come up with is "it's like if you had an objective reality based on Wikipedia edit wars."
- kristopolous 3y ago
  Exactly. Especially when there's some hustle behind lying to people. Phony medical advice is the classic one here. People do it to sell their sugar pills or do some Peter Popoff style healing touch hustle. Some scams, like MLM, encourage massive volumes of this bullshit to be created and disseminated. The democracy of information does not lead to an increased accuracy based on whatever information spreads the most. The internet is a multi-decade long global proof of this. There's a reason that classrooms have 1 teacher and say 24 students - only 4% of that room is to be trusted with the material. To know if something is true, you need to eventually have some kind of external confirmation. Karl Popper wrote a lot about this. These chat systems can make predictions about the existing corpus but it can't like roll around town and do actual science to confirm things. If all truth must be derived from self-contained systems than you have to make sure your self-contained system isn't bullshit. A game for instance, can have its own rules and consistencies and things can be rationalized through the game. But how much that maps to a reality cannot be concluded strictly from inside the game. For instance, if you want expertise in masonry and construction you need to make sure you aren't just feeding your AI Tetris and SimCity.
- bick_nyers 3y ago
  Just because it is a probability doesn't mean that it's strictly the probability derived from frequency in your training corpus. If you just used that as your model, it would not be nearly as good. Things like reasoning, a world model, and other heuristics can be used to generate said probabilities internally. All that being said, who is to say that humans aren't just looking at probabilities when they think?

eropple 3y ago

> Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already. Because it's not thinking, it's looking at probabilities. If you don't prune branches, you've created a disinformation radiator if that information is sufficiently populous within your corpus to be a pretty-good-bet. The best analogy I've come up with is "it's like if you had an objective reality based on Wikipedia edit wars."

kristopolous 3y ago
Exactly. Especially when there's some hustle behind lying to people. Phony medical advice is the classic one here. People do it to sell their sugar pills or do some Peter Popoff style healing touch hustle. Some scams, like MLM, encourage massive volumes of this bullshit to be created and disseminated. The democracy of information does not lead to an increased accuracy based on whatever information spreads the most. The internet is a multi-decade long global proof of this. There's a reason that classrooms have 1 teacher and say 24 students - only 4% of that room is to be trusted with the material. To know if something is true, you need to eventually have some kind of external confirmation. Karl Popper wrote a lot about this. These chat systems can make predictions about the existing corpus but it can't like roll around town and do actual science to confirm things. If all truth must be derived from self-contained systems than you have to make sure your self-contained system isn't bullshit. A game for instance, can have its own rules and consistencies and things can be rationalized through the game. But how much that maps to a reality cannot be concluded strictly from inside the game. For instance, if you want expertise in masonry and construction you need to make sure you aren't just feeding your AI Tetris and SimCity.
bick_nyers 3y ago
Just because it is a probability doesn't mean that it's strictly the probability derived from frequency in your training corpus. If you just used that as your model, it would not be nearly as good. Things like reasoning, a world model, and other heuristics can be used to generate said probabilities internally. All that being said, who is to say that humans aren't just looking at probabilities when they think?