4 ms·

I'm not convinced that this pace will continue. We're seeing a lot of really cool, rapid evolution of this tech in a short amount of time, but I do think we'll

I'm not convinced that this pace will continue. We're seeing a lot of really cool, rapid evolution of this tech in a short amount of time, but I do think we'll hit a soft ceiling in the not too distant future as well. If you look at something like smartphones, for example. Smartphones, from my perspective, got drastically better and better from about ~2006-2015 or so. They were rapidly improving cameras and battery life and it felt like a new super cool app that would change our lives was being released every day, but it feels like by ~2016 or so, phones more or less hit a ceiling on how cool they were going to get. Obviously things still improve, but I feel like the pace slowed down eventually. I think AI is going to have the same path. GANNs and transformers and LLMs and the like have opened the floodgates and for the next few years clever people are going to figure out a ton of really clever uses for them, but eventually it's going to plateau and progress will become substantially more gradual. I don't think progress is linear, I think it's more like a staircase.

satellite2 3y ago
There is clearly a plateau in how good a ux can be. It might be a local optimum but still you solve the task the user wants. I don't see a clear ceiling in intelligence. And if the ceiling is how much of the human tasks can be replaced then I think when we reach it the world is going to look very different from now. (Let's also not discount how much the world changed since the introduction of the smartphone.)
- chimprich 3y ago
  > I don't see a clear ceiling in intelligence The plateau in this case is presumably how far you can advance intelligence from the current model architectures. There seems to be diminishing returns from throwing more layers, parameters or training data at these things. We will see improvements but for dramatic increases I think we'll need new breakthroughs. New inventions are hard to predict, pretty much by definition.
  - tombert 3y ago
    That's more or less what I was getting at; the cool new GANN and LLM models have a certain set of problems that they will solve exceptionally well, and then another set of problems that they will solve "pretty well", but I don't think they'll solve every problem.
whimsicalism 3y ago
Maybe that's true but I honestly don't think we can reason at all about how this will progress from a consumer hardware product like the iPhone.
johnyzee 3y ago
> I do think we'll hit a soft ceiling in the not too distant future ... it's going to plateau and progress will become substantially more gradual. I don't think this will age well. It's a matter of simple compute power to advance from realistic text/token prediction, to realistic synthesis of stuff like human (or animal) body movement, for all kinds of situations, including realistic facial/body language, moods, and so on. Of course perfect voice synthesis. Coupled with good enough robotics, you can see where I'm going with this, and that's only because my imagination is limited to sci-fi movie tropes. I think this is going to be wilder than we can imagine, while still just copying training sets.
- troupo 3y ago
  > It's a matter of simple compute power to advance Yup. It's "just" a compute advance away. Never mind it's already consuming as much computing as we can throw at it. It's "just" there.
- FiberBundle 3y ago
  Isn't video prediction a substantially harder problem than text prediction? At least that was the case a couple of years ago with RNNs/LSTMs. Haven't kept up with the research, maybe there's been progress.
richardw 3y ago
The worry is that in your analogy, we’re the SLR. ChatGPT is a 1MP digital camera.
tim333 3y ago
Or an exponential perhaps. Like the Wait But Why thing (https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html https://waitbutwhy.com/2015/01/artificial-intelligence-revol... bottom of the article)

satellite2 3y ago

There is clearly a plateau in how good a ux can be. It might be a local optimum but still you solve the task the user wants. I don't see a clear ceiling in intelligence. And if the ceiling is how much of the human tasks can be replaced then I think when we reach it the world is going to look very different from now. (Let's also not discount how much the world changed since the introduction of the smartphone.)

chimprich 3y ago
> I don't see a clear ceiling in intelligence The plateau in this case is presumably how far you can advance intelligence from the current model architectures. There seems to be diminishing returns from throwing more layers, parameters or training data at these things. We will see improvements but for dramatic increases I think we'll need new breakthroughs. New inventions are hard to predict, pretty much by definition.
- tombert 3y ago
  That's more or less what I was getting at; the cool new GANN and LLM models have a certain set of problems that they will solve exceptionally well, and then another set of problems that they will solve "pretty well", but I don't think they'll solve every problem.

whimsicalism 3y ago

Maybe that's true but I honestly don't think we can reason at all about how this will progress from a consumer hardware product like the iPhone.

johnyzee 3y ago

> I do think we'll hit a soft ceiling in the not too distant future ... it's going to plateau and progress will become substantially more gradual. I don't think this will age well. It's a matter of simple compute power to advance from realistic text/token prediction, to realistic synthesis of stuff like human (or animal) body movement, for all kinds of situations, including realistic facial/body language, moods, and so on. Of course perfect voice synthesis. Coupled with good enough robotics, you can see where I'm going with this, and that's only because my imagination is limited to sci-fi movie tropes. I think this is going to be wilder than we can imagine, while still just copying training sets.

troupo 3y ago
> It's a matter of simple compute power to advance Yup. It's "just" a compute advance away. Never mind it's already consuming as much computing as we can throw at it. It's "just" there.
FiberBundle 3y ago
Isn't video prediction a substantially harder problem than text prediction? At least that was the case a couple of years ago with RNNs/LSTMs. Haven't kept up with the research, maybe there's been progress.

richardw 3y ago

The worry is that in your analogy, we’re the SLR. ChatGPT is a 1MP digital camera.

tim333 3y ago

Or an exponential perhaps. Like the Wait But Why thing (https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html https://waitbutwhy.com/2015/01/artificial-intelligence-revol... bottom of the article)