3 ms·
> o3 fixes the fundamental limitation of the LLM paradigm – the inability to recombine knowledge at test time I don't understand this mindset. We have all expe
by oezi 2y ago
> o3 fixes the fundamental limitation of the LLM paradigm – the inability to recombine knowledge at test time
I don't understand this mindset. We have all experienced that LLMs can produce words never spoken before. Thus there is recombination of knowledge at play. We might not be satisfied with the depth/complexity of the combination, but there isn't any reason to believe something fundamental is missing. Given more compute and enough recursiveness we should be able to reach any kind of result from the LLM.
The linked article says that LLMs are like a collection of vector programs. It has always been my thinking that computations in vector space are easy to make turing complete if we just have an eigenvector representation figured out.
- lagrange77 2y ago> Given more compute and enough recursiveness we should be able to reach any kind of result from the LLM. That was always true for NNs in general, yet it took a very specific structure to get to where we are now. (..with a certain amount of time and resources.) > thinking that computations in vector space are easy to make turing complete if we just have an eigenvector representation figured out Sounds interesting, would you elaborate?