If you invented the transformer but didn't have trillions of tokens to train it with, no chatGPT. But if you had Mamba/RWKV/SSSM and trillions of tokens you wou

If you invented the transformer but didn't have trillions of tokens to train it with, no chatGPT. But if you had Mamba/RWKV/SSSM and trillions of tokens you would have almost the same thing with chatGPT. The training set is magical. It took humanity a long time to discover all the nifty ideas we have in it. It's the result of many generations of humans working together, using language to share their experience. Intelligence is a social process, even though we like to think about keys and queries, or synapses and neurotransmitters, in fact it is the work of many people that made it possible. And language is that central medium between all of us, an evolutionary system of ideas, evolving at a much faster rate than biology. Now AI have become language replicators like humans, a new era in the history of language has begun. The same language trains humans and LLMs to achieve similar sets of abilities.