4 ms·
> because they have not been engineered The fundamental concept behind LLMs is to allow the model to autonomously deduce concepts, rather than explicitly engin
by maister 3y ago
> because they have not been engineered
The fundamental concept behind LLMs is to allow the model to autonomously deduce concepts, rather than explicitly engineering solutions into the system.
- seydor 3y agoThe fundamental concept is to learn the statistics of text, and in the process it models the syntax via long-range connections successfully. There is no indication that it actively generates "concepts" or that it knows what concepts are. In fact the model is not self-reflective at all, it cannot observe its own activation or tell me anything about it.
- circuit10 3y agoThere is an indication, you can find it by clicking on this post. The self-reflection part is probably true, but that’s not strictly necessary to understand concepts
- seydor 3y agoit is important in order for us to accept it as an agent that understands things, because self-reflection is so important and obvious to us.
- timschmidt 3y agoI'm still waiting for someone to prove beyond a shadow of doubt that humans have a single one of these features we're debating about the presence or absence of in LLMs.
- seydor 3y agothere is no way to prove because those are subjective to humans. LLMs would have to at least show they have a subjective view (currently the 'internal world' they report is inconsistent)
- timschmidt 3y agoThe internal worlds of humans are inconsistent: https://en.wikipedia.org/wiki/Shadow_(psychology) https://en.wikipedia.org/wiki/Shadow_(psychology)
- mr_toad 3y ago> learn the statistics > what concepts are How do you know concepts aren’t just statistics?
- seydor 3y ago"concept" is ill-defined , it s a subjective thing that humans invented. It is probably not possible to define it without a sense (a definition) of self.