4 ms·

I don't think there's any fundamental reason why a LLM shouldn't be able to learn the distinction between being creative and being factual, at least much better

I don't think there's any fundamental reason why a LLM shouldn't be able to learn the distinction between being creative and being factual, at least much better than currently. It's just another classification problem (if a challenging one), and classification is what NNs are known for.

szundi 2y ago
Source of your disbelief?
- Sharlin 2y ago
  Call it Bayesian intuition. I’d argue that at this point the specific claim "LLMs cannot solve the confabulation problem" is the one that requires hard evidence. These things forced almost all of us to update our priors on what glorified Markov chains are – and are not – capable of if you throw enough data and compute at them.
b112 2y ago
From what I see, there may be two issues here. 1) LLM has data from a large set of sources, but in some cases it has very little data on certain sub-topics. With so few sources of data, it is easier for some weird corner of the internet to have conflicting data, and some LLM models may randomly favour that instead of the valid answer. So not hallucination in this case, instead, a wrong answer from wrong sources. 2) LLM doesn't know, but has seen from human behaviour that just "making shit up, when you don't know" is entirely acceptable for many humans. Here I'm not referring to people saying (as I am) "may be", but instead "No, it's this!" with immense surety, in order to win discussions. Combine this with 'temperature' settings many model interfaces have, and there you go. 2b) Alternatively, #2 is inline with "I believe in flying spaghetti monsters", and other beliefs such as astrology, ghosts, and endless other "there is zero proof, but humans have immense weird data on it". So #2 is solvable by changing temperature, or keeping temperature but saying "Don't make things up". #1 may be temp + random choice of a lower probability pathway. #2b, not sure what to think here. How will LLMs ingest data, and use it to determine what is acceptable logic processing, when a lot of stuff in category 2b has zero proof behind it. And with zero proof, people are absolutely positive of the status of said thing? #2b trains LLMs that "just having a random idea and then even killing people that don't agree, seems OK for humans". (think religious wars, violent acts, burning witches, etc)
eviks 2y ago
Just calling it "just another classification problem" doesn't mean there is any fundamental reason to assume it's a solvable one (NNs are known for some classifications, not any)
- Sharlin 2y ago
  What results are there that show that deep NNs fundamentally cannot learn to tightly approximate some function or class of functions? (Honest question.) Outside pathological nowhere-differentiable cases, of course.
  - eviks 2y ago
    I'm not sure it makes sense to talk in such absolutes (requiring a proof of impossibility) in such a fluid category - after all, you could include the imaginary perfect replica of the human brain in the NN category, and obviously human brains can do this task. But then again, this categorization provides no information to make any assessment of whether that is in any way achievable.
    - Sharlin 2y ago
      Sure. Not in the sense of "here’s a proof that a perceptron cannot learn XOR", but at least something that would hint that there are qualitative gaps in the expressive power of DLNNs, or even LLMs specifically.
outofpaper 2y ago
LLMs seem to be conversational or interactive in non-training phases because they can generate flows of text that mimic conversations, complete with personas, characters, and personalities. However, this apparent conversation is merely a generative artifact created by the user's input. LLMs do not truly learn or understand; instead, they use their internal data repository to produce "hallucinations" of text that seem conversational. While LLMs do not truly comprehend or understand, their "hallucinations" can be remarkably useful and even enable them to accomplish complex tasks, such as solving mazes or translating languages.
- HeatrayEnjoyer 2y ago
  They very clearly do have at least some level of understanding. Emergence, and all that.
- mewpmewp2 2y ago
  What does "truly" mean and how does human understanding differ to make it "truly"?
  - outofpaper 2y ago
    Truly means real. Assuming you are a human, you can truly relate and connect ideas with your experiences. You can know what a rose is both in relation to having seen and smelled it and if now having experienced flowers and rose like things. Without qualia you can not know and understand a rose. An LLM no mater how big can never have experienced any of this. The closest it can ever come is to have been trained on the tokens in texts relating to roses. If its more than an LLM it might have also been shown images or even video or roses but no LLM alone has ever seen a rose. LLMs can only contain understanding the way a book can. There is no sentience in a book. Even if we jump to the old analogy of the Chinese room the instruction book in not conscious, nor is the reader following the instructions able to actually speak Chinese. It's beyond and between where lies the entity thar can converse in Chinese.
    - mewpmewp2 2y ago

szundi 2y ago

Source of your disbelief?

Sharlin 2y ago
Call it Bayesian intuition. I’d argue that at this point the specific claim "LLMs cannot solve the confabulation problem" is the one that requires hard evidence. These things forced almost all of us to update our priors on what glorified Markov chains are – and are not – capable of if you throw enough data and compute at them.

b112 2y ago

From what I see, there may be two issues here. 1) LLM has data from a large set of sources, but in some cases it has very little data on certain sub-topics. With so few sources of data, it is easier for some weird corner of the internet to have conflicting data, and some LLM models may randomly favour that instead of the valid answer. So not hallucination in this case, instead, a wrong answer from wrong sources. 2) LLM doesn't know, but has seen from human behaviour that just "making shit up, when you don't know" is entirely acceptable for many humans. Here I'm not referring to people saying (as I am) "may be", but instead "No, it's this!" with immense surety, in order to win discussions. Combine this with 'temperature' settings many model interfaces have, and there you go. 2b) Alternatively, #2 is inline with "I believe in flying spaghetti monsters", and other beliefs such as astrology, ghosts, and endless other "there is zero proof, but humans have immense weird data on it". So #2 is solvable by changing temperature, or keeping temperature but saying "Don't make things up". #1 may be temp + random choice of a lower probability pathway. #2b, not sure what to think here. How will LLMs ingest data, and use it to determine what is acceptable logic processing, when a lot of stuff in category 2b has zero proof behind it. And with zero proof, people are absolutely positive of the status of said thing? #2b trains LLMs that "just having a random idea and then even killing people that don't agree, seems OK for humans". (think religious wars, violent acts, burning witches, etc)

eviks 2y ago

Just calling it "just another classification problem" doesn't mean there is any fundamental reason to assume it's a solvable one (NNs are known for some classifications, not any)

Sharlin 2y ago
What results are there that show that deep NNs fundamentally cannot learn to tightly approximate some function or class of functions? (Honest question.) Outside pathological nowhere-differentiable cases, of course.
- eviks 2y ago
  I'm not sure it makes sense to talk in such absolutes (requiring a proof of impossibility) in such a fluid category - after all, you could include the imaginary perfect replica of the human brain in the NN category, and obviously human brains can do this task. But then again, this categorization provides no information to make any assessment of whether that is in any way achievable.
  - Sharlin 2y ago
    Sure. Not in the sense of "here’s a proof that a perceptron cannot learn XOR", but at least something that would hint that there are qualitative gaps in the expressive power of DLNNs, or even LLMs specifically.

outofpaper 2y ago

LLMs seem to be conversational or interactive in non-training phases because they can generate flows of text that mimic conversations, complete with personas, characters, and personalities. However, this apparent conversation is merely a generative artifact created by the user's input. LLMs do not truly learn or understand; instead, they use their internal data repository to produce "hallucinations" of text that seem conversational. While LLMs do not truly comprehend or understand, their "hallucinations" can be remarkably useful and even enable them to accomplish complex tasks, such as solving mazes or translating languages.

HeatrayEnjoyer 2y ago
They very clearly do have at least some level of understanding. Emergence, and all that.
mewpmewp2 2y ago
What does "truly" mean and how does human understanding differ to make it "truly"?
- outofpaper 2y ago
  Truly means real. Assuming you are a human, you can truly relate and connect ideas with your experiences. You can know what a rose is both in relation to having seen and smelled it and if now having experienced flowers and rose like things. Without qualia you can not know and understand a rose. An LLM no mater how big can never have experienced any of this. The closest it can ever come is to have been trained on the tokens in texts relating to roses. If its more than an LLM it might have also been shown images or even video or roses but no LLM alone has ever seen a rose. LLMs can only contain understanding the way a book can. There is no sentience in a book. Even if we jump to the old analogy of the Chinese room the instruction book in not conscious, nor is the reader following the instructions able to actually speak Chinese. It's beyond and between where lies the entity thar can converse in Chinese.
  - mewpmewp2 2y ago