4 ms·
LLM (and deep learning in general) are to AGI what bogosort is to sorting. For some reason beyond me people think it's very important to not try to understand a
by lapinot 4y ago
LLM (and deep learning in general) are to AGI what bogosort is to sorting. For some reason beyond me people think it's very important to not try to understand anything about the structure of the problem you're trying to solve and just make a very general algorithm which will be suboptimal in just about every way except for the generality of it's code.
Seeing the world as a trillion dimensional token soup is definitely quite general and at the same time very very weak in terms of expressivity.
- iameli 4y agoSure, I think buy this interpretation. As long as we can agree that bogosort and quicksort are both still sorting algorithms. My brain definitely has some structures that are useful for me understanding the world that are more immediately useful than trillion dimensional token soup. But I'm also not convinced that it's impossible those structures could be successfully emulated by quadrillion dimensional token soup. And a lot of folks seem to be convinced that it's some kind of fundamental impossibility.
- canjobear 4y agoNo other approach has worked as well for natural language, and not for lack of trying.