3 ms·
> it doesn't understand its own limitations, ie it has no self awareness. And how much self awareness do humans have, exactly? We walk around with heads filled
by BoppreH 3y ago
> it doesn't understand its own limitations, ie it has no self awareness.
And how much self awareness do humans have, exactly? We walk around with heads filled with false memories[1], accidentally ignore surprisingly prominent stimuli[2], and struggle to understand things as simple as material color[3]. And people are mostly unaware of these limitations, or those examples would not be considered surprising!
I think we plan around the limitations we observe, not our real limitations. If that's the case, then LLMs simply lack training data to notice those limitations.
[1] https://en.wikipedia.org/wiki/False_memory https://en.wikipedia.org/wiki/False_memory
[2] https://www.youtube.com/watch?v=vJG698U2Mvo https://www.youtube.com/watch?v=vJG698U2Mvo (old selective attention test)
[3] https://en.wikipedia.org/wiki/The_dress https://en.wikipedia.org/wiki/The_dress
- Jensson 3y ago> then LLMs simply lack training data to observe those limitations. How would training data add an introspective step to the LLM? At best you can add a lot of common LLM fitted strategies developed by humans to the training set and thus make the LLM solve a few more problems well, but that will always be hampered by the need for humans to develop those. It is basically just programming the LLM. My view is that LLM is results in natural language programming. Which is great, but it isn't any more intelligence than normal programming. Its just that we already have a lot of cool programs already written in natural languages, and now we get to run those for the first time.