3 ms·
This reminds me of the self-driving car hype where we -- the public, not waymo/cruise -- didn't actually get fully self-driving cars ushering in an "end of dri
by sottol 3y ago
This reminds me of the self-driving car hype where we -- the public, not waymo/cruise -- didn't actually get fully self-driving cars ushering in an "end of driving" (or had to reimagine our cities and lives) but instead got very good lane-keeping assistants that work 99% of the time.
- famouswaffles 3y agoNon-LLM self-driving (some companies are starting to look into them for driving now - https://wayve.ai/thinking/lingo-natural-language-autonomous-driving/ https://wayve.ai/thinking/lingo-natural-language-autonomous-...) has multiple vectors (it's not just one system, it's loads of crafted and trained sensors and algorithms working together to plot a path). Improvement is subsequently harder to pin down or ascertain. Say you massively improve how well your sensor disambiguates objects or signs or whatever on the road, that's very good but it doesn't necessarily result in equivalently better decision making. This is one system/model and improvement is about as straightforward and guaranteed as you could ever hope for - Train longer, Reduce loss. Since loss trends down to a perfect model of the dataset (i.e a GPT that is fed Chess games doesn't simply care about playing games but modelling the distributions of human chess games and will not suddenly stop improving its prediction capabilities once it can play a valid game), this is all it takes to get a GPT that is better than what it started out as. GPT-5 will be much better than 4 and the only thing they'll need to do to achieve that is 'Train longer, Reduce Loss'.