This is the "Anti Stochastic Parrot" paper. I am happy to finally read a philosophical paper on LLMs. I just have a small quip. > While LLMs show promise in v

This is the "Anti Stochastic Parrot" paper. I am happy to finally read a philosophical paper on LLMs. I just have a small quip. > While LLMs show promise in various forms of task generalization, their participation in the ratcheting process of cultural learning thus appears contingent on further advancements in these areas, which might lie beyond the reach of current architectures. I don't think cultural transmission with LLMs is that incipient. Evolution through Large Models - https://arxiv.org/abs/2206.08896 https://arxiv.org/abs/2206.08896 We have seen papers where LLMs are used as "search heads" or "mutation operators" in evolutionary methods. They can rely on their language proficiency to explore the most promising leads, spanning a vast combinatorial space. And another example of "cultural transmission": GPT-4 has its paws all over Mistral and LLaMA finetunes. Plenty of cultural transmission between big and small LLMs. Not to mention Phi, a model trained purely on synthetic GPT-4 data. Synthetic data has been estimated to be 5x more efficient than human data, you can make a model 1/5 size with similar performance.