4 ms·

> LLMs are plausibility engines. This is one way to look at LLMs. But it doesn't automatically impose an upper bound on their capabilities. Humans are reprodu

by fl7305 3y ago

> LLMs are plausibility engines. This is one way to look at LLMs. But it doesn't automatically impose an upper bound on their capabilities. Humans are reproductive organisms. This is also true. At a glance, it would seem that humans could not evolve to be intelligent. They are only selected on their reproductive abilities. But obviously the categorization, while true, doesn't impose an upper bound on human abilities. LLMs are evolved to be as efficient as possible at retaining knowledge. A simple strategy for retaining knowledge is memorization. Neural networks definitely can do that. A different strategy for retaining knowledge is using algorithms. Neural networks can evolve to retain knowledge using algorithms too. For instance, it has been shown that a small neural network evolved FFT-like structures to perform addition. It started out with memorization, which was far from perfect, but later in the training it switched to an algorithm using FFT for addition which produced perfect results. I think the better LLMs retain knowledge through sophisticated compression. It involves building a world model and ways to relate the input text to that model. I see it as the building blocks of a reasoning machine. It is incomplete and buggy, and the current architectures might hit a ceiling soon, but nonetheless it is a completely different thing than pure memorization.

nyrikki 3y ago
FFT is easy fairly easy for transformers to find as a pattern. Note that OpenAI admits that pre-training (pattern finding and matching) is where the abilities on professional tests comes from. Even proofs in presburger arithmetic have an upper bound on the second level of the polynomial hierarchy, which is way beyond the computational power of any LLM https://arxiv.org/abs/2207.00729 https://arxiv.org/abs/2207.00729 >The model’s capabilities on exams appear to stem primarily from the pre-training process and are not significantly affected by RLHF. On multiple choice questions, both the base GPT-4 model and the RLHF model perform equally well on average across the exams we tested https://arxiv.org/abs/2303.08774 https://arxiv.org/abs/2303.08774 To practically scale, parallelism is required, and the ability to find algorithms within the L or TC0 complexity classes is limited. Remember that individual ANNs as used in transformers are just binary linear classifiers. Which you can do a lot with, but probably limited to P assuming we don't find out L or TC0 are larger than we think now. Presburger arithmetic, or first order logic with (+,=), or (*,=) is the strongest FoL that is both complete and consistent. move up to Peano arithmetic and you start to hit the limits from Gödel, Church, and Turing. There will be instances that an LLM can learn with CoT, zero shot etc... But most of those will be due to luck or parallel patterns in the training set. LLMs simplicity bias is great for plausible answers to our of distribution questions, but puts limits on what can be learned as far as 'algorithms' go. They will work for 'there exists' problems far better than for any 'for all' or 'for most' problems unless they have learnable patterns in the training set.

by fl7305 3y ago

nyrikki 3y ago

FFT is easy fairly easy for transformers to find as a pattern. Note that OpenAI admits that pre-training (pattern finding and matching) is where the abilities on professional tests comes from. Even proofs in presburger arithmetic have an upper bound on the second level of the polynomial hierarchy, which is way beyond the computational power of any LLM https://arxiv.org/abs/2207.00729 https://arxiv.org/abs/2207.00729 >The model’s capabilities on exams appear to stem primarily from the pre-training process and are not significantly affected by RLHF. On multiple choice questions, both the base GPT-4 model and the RLHF model perform equally well on average across the exams we tested https://arxiv.org/abs/2303.08774 https://arxiv.org/abs/2303.08774 To practically scale, parallelism is required, and the ability to find algorithms within the L or TC0 complexity classes is limited. Remember that individual ANNs as used in transformers are just binary linear classifiers. Which you can do a lot with, but probably limited to P assuming we don't find out L or TC0 are larger than we think now. Presburger arithmetic, or first order logic with (+,=), or (*,=) is the strongest FoL that is both complete and consistent. move up to Peano arithmetic and you start to hit the limits from Gödel, Church, and Turing. There will be instances that an LLM can learn with CoT, zero shot etc... But most of those will be due to luck or parallel patterns in the training set. LLMs simplicity bias is great for plausible answers to our of distribution questions, but puts limits on what can be learned as far as 'algorithms' go. They will work for 'there exists' problems far better than for any 'for all' or 'for most' problems unless they have learnable patterns in the training set.