4 ms·
Weather and stock market are both chaotic systems. Increasing evidence suggests that AGI will not be attainable solely using LLMs/transformers/current architec
by lossolo 3y ago
Weather and stock market are both chaotic systems.
Increasing evidence suggests that AGI will not be attainable solely using LLMs/transformers/current architecture, as LLMs can't extrapolate beyond the patterns in their training data (according to a paper from DeepMind last month):
"Together our results highlight that the impressive ICL abilities of high-capacity sequence models may be more closely tied to the coverage of their pretraining data mixtures than inductive biases that create fundamental generalization capabilities."[1]
1. https://arxiv.org/abs/2311.00871 https://arxiv.org/abs/2311.00871