3 ms·

>My prediction here is that exponential improvements continue at least for the new few years and likely beyond. GPT-3 to GPT-4 was an exponential improvement?

>My prediction here is that exponential improvements continue at least for the new few years and likely beyond. GPT-3 to GPT-4 was an exponential improvement? Progress is not usually exponential. Your phone now isn’t x10 better than it was a few years ago. Progress in AI is a huge jump, then refinement of that jump, then stagnation until the next big jump is discovered. Look at CNNs dominating image classification competitions out of nowhere, then they got refined, then they kind of stagnated and didn’t get much % accuracy better in whatever benchmark was used, in fact I think humans are still way better at many vision tasks and it’s been over a decade of research now since CNNs jumped on the scene as the hot thing. I don’t know why people refuse to understand or see this, but it’s tiring constantly hearing people pretend everything is exponential and AGI is two years away when AI hasn’t even beaten humans at some pretty trivial vision benchmarks.

richardfeynman 3y ago
I agree that there are still differences between humans and AI, and vision is one of them. Humans also remember across conversations, ChatGPT doesn't. We have longer context windows. We think in our downtime, unprompted. We take input from 5 senses, today's chatGPT only takes input from text. But--and this is a big but--the set of things that people can do better than computers has shrunk significantly over the past five months. Today, thanks to GPT4, AI can get a B in a Quantum Computing class, generate engaging stories, know that the color yellow is closer to orange than blue (despite never having seen color!), answer emotionally laden questions with the sort of facility that IMO is better than most humans, write code, rhyme, and much more. All of this stuff was unthinkable before. I personally thought it would be centuries until this stuff was possible. I was very wrong. Several developments in Deep Learning, like the Transformers paper (https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762), set off this growth, as did big data and increased computing power. The insight of particular humans, like Ilya Sutskever, played a role as well. But taken together, I actually don't understand how one can argue that we aren't at the beginning of a massive exponential. Of course there are things humans can still do better than AI, but the number of things is shrinking rapidly, while the number of things computers can do better than humans is growing rapidly. I argue that we are indeed at the beginning of an exponential, and we'll see both new classes of products and faster development time.

richardfeynman 3y ago

I agree that there are still differences between humans and AI, and vision is one of them. Humans also remember across conversations, ChatGPT doesn't. We have longer context windows. We think in our downtime, unprompted. We take input from 5 senses, today's chatGPT only takes input from text. But--and this is a big but--the set of things that people can do better than computers has shrunk significantly over the past five months. Today, thanks to GPT4, AI can get a B in a Quantum Computing class, generate engaging stories, know that the color yellow is closer to orange than blue (despite never having seen color!), answer emotionally laden questions with the sort of facility that IMO is better than most humans, write code, rhyme, and much more. All of this stuff was unthinkable before. I personally thought it would be centuries until this stuff was possible. I was very wrong. Several developments in Deep Learning, like the Transformers paper (https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762), set off this growth, as did big data and increased computing power. The insight of particular humans, like Ilya Sutskever, played a role as well. But taken together, I actually don't understand how one can argue that we aren't at the beginning of a massive exponential. Of course there are things humans can still do better than AI, but the number of things is shrinking rapidly, while the number of things computers can do better than humans is growing rapidly. I argue that we are indeed at the beginning of an exponential, and we'll see both new classes of products and faster development time.