Ask a toddler how the world works and you'll get a very similar response. It is entirely likely the 80%-of-human-intelligence barrier is not a "dead end" but m

Ask a toddler how the world works and you'll get a very similar response. It is entirely likely the 80%-of-human-intelligence barrier is not a "dead end" but merely a temporary limitation until these models are made to hone their understanding and update over time (i.e. get feedback) instead of going for zero-shot perfection. The GPT models incorporating video should start developing this "memory" naturally as they incorporate temporal coherence (time) into the model. The fact we got this far through brute force is just insanely telling. This is a natural phenomena we're stumbling upon, not something crafted by humans. Also - fun fact, the Facebook Llama model that fits on a Raspberry Pi and is almost as good as GPT3? Also basically brute force. They just trained it a lot longer and it shrunk the model. Food for thought.