3 ms·

> So far I've only seen examples based on tokenization ("what's the third letter in word?"), and the fact that answers are generated sequentially ("write a sent

> So far I've only seen examples based on tokenization ("what's the third letter in word?"), and the fact that answers are generated sequentially ("write a sentence that includes the count of words in itself"). But those feel unsatisfactory because those are not cognitive limitations. It could solve those, the reason it can't is that it doesn't understand its own limitations, ie it has no self awareness. Humans plan things based around their own limitations, if an AI model can't even do that how can it be considered an AGI?

famouswaffles 3y ago
>It could solve those, the reason it can't is that it doesn't understand its own limitations, ie it has no self awareness. I don't think Humans understand their limitations very well at all. We plan around limitations we observe, not necessarily limitations that are present. There are many surprising limitations only recently uncovered by advances in neuroscience. I'm not sure humans would qualify as self aware by your criteria. >if an AI model can't even do that how can it be considered an AGI? By being artificial and generally intelligent. I don't see anything about "self-awareness" (ill-defined as it may be) anywhere in that acronym.
- Jensson 3y ago
  > By being artificial and generally intelligent. I don't see anything about "self-awareness" (ill-defined as it may be) anywhere in that acronym. I said they need to be aware of their own limitations, that is very well defined. Humans might not be aware of all our limitations, but we are aware of a lot of them and that lets us work around them. Working around your own limitations is necessary for AGI, otherwise there are many trivial problems you can't solve. It is possible that you can make many human crafted strategies and that goes far enough, but I doubt it since trying to make human crafted examples never has resulted in AGI advancements just allowing the bot to solve a specific problem well.
  - famouswaffles 3y ago
    >I said they need to be aware of their own limitations, that is very well defined. You seemed to imply that this "awareness" was some inherent ability of humans but it's very clearly not. If it were, we wouldn't be a lot more aware of these limitations today than even a few decades past. and if all the records we kept vanished in an instant today then the next generations of humans would be a lot less aware. This "awareness" you are talking about is little more than accumulated knowledge. By the realistic definition of 'aware of what is observed', i fail to see how LLMs don't demonstrate this.
BoppreH 3y ago
> it doesn't understand its own limitations, ie it has no self awareness. And how much self awareness do humans have, exactly? We walk around with heads filled with false memories[1], accidentally ignore surprisingly prominent stimuli[2], and struggle to understand things as simple as material color[3]. And people are mostly unaware of these limitations, or those examples would not be considered surprising! I think we plan around the limitations we observe, not our real limitations. If that's the case, then LLMs simply lack training data to notice those limitations. [1] https://en.wikipedia.org/wiki/False_memory https://en.wikipedia.org/wiki/False_memory [2] https://www.youtube.com/watch?v=vJG698U2Mvo https://www.youtube.com/watch?v=vJG698U2Mvo (old selective attention test) [3] https://en.wikipedia.org/wiki/The_dress https://en.wikipedia.org/wiki/The_dress
- Jensson 3y ago
  > then LLMs simply lack training data to observe those limitations. How would training data add an introspective step to the LLM? At best you can add a lot of common LLM fitted strategies developed by humans to the training set and thus make the LLM solve a few more problems well, but that will always be hampered by the need for humans to develop those. It is basically just programming the LLM. My view is that LLM is results in natural language programming. Which is great, but it isn't any more intelligence than normal programming. Its just that we already have a lot of cool programs already written in natural languages, and now we get to run those for the first time.

famouswaffles 3y ago

>It could solve those, the reason it can't is that it doesn't understand its own limitations, ie it has no self awareness. I don't think Humans understand their limitations very well at all. We plan around limitations we observe, not necessarily limitations that are present. There are many surprising limitations only recently uncovered by advances in neuroscience. I'm not sure humans would qualify as self aware by your criteria. >if an AI model can't even do that how can it be considered an AGI? By being artificial and generally intelligent. I don't see anything about "self-awareness" (ill-defined as it may be) anywhere in that acronym.

Jensson 3y ago
> By being artificial and generally intelligent. I don't see anything about "self-awareness" (ill-defined as it may be) anywhere in that acronym. I said they need to be aware of their own limitations, that is very well defined. Humans might not be aware of all our limitations, but we are aware of a lot of them and that lets us work around them. Working around your own limitations is necessary for AGI, otherwise there are many trivial problems you can't solve. It is possible that you can make many human crafted strategies and that goes far enough, but I doubt it since trying to make human crafted examples never has resulted in AGI advancements just allowing the bot to solve a specific problem well.
- famouswaffles 3y ago
  >I said they need to be aware of their own limitations, that is very well defined. You seemed to imply that this "awareness" was some inherent ability of humans but it's very clearly not. If it were, we wouldn't be a lot more aware of these limitations today than even a few decades past. and if all the records we kept vanished in an instant today then the next generations of humans would be a lot less aware. This "awareness" you are talking about is little more than accumulated knowledge. By the realistic definition of 'aware of what is observed', i fail to see how LLMs don't demonstrate this.

BoppreH 3y ago

> it doesn't understand its own limitations, ie it has no self awareness. And how much self awareness do humans have, exactly? We walk around with heads filled with false memories[1], accidentally ignore surprisingly prominent stimuli[2], and struggle to understand things as simple as material color[3]. And people are mostly unaware of these limitations, or those examples would not be considered surprising! I think we plan around the limitations we observe, not our real limitations. If that's the case, then LLMs simply lack training data to notice those limitations. [1] https://en.wikipedia.org/wiki/False_memory https://en.wikipedia.org/wiki/False_memory [2] https://www.youtube.com/watch?v=vJG698U2Mvo https://www.youtube.com/watch?v=vJG698U2Mvo (old selective attention test) [3] https://en.wikipedia.org/wiki/The_dress https://en.wikipedia.org/wiki/The_dress

Jensson 3y ago
> then LLMs simply lack training data to observe those limitations. How would training data add an introspective step to the LLM? At best you can add a lot of common LLM fitted strategies developed by humans to the training set and thus make the LLM solve a few more problems well, but that will always be hampered by the need for humans to develop those. It is basically just programming the LLM. My view is that LLM is results in natural language programming. Which is great, but it isn't any more intelligence than normal programming. Its just that we already have a lot of cool programs already written in natural languages, and now we get to run those for the first time.