3 ms·

Progress is sigmoidal approaching a limit. If the minimum acceptable performance is into the logarithmic portion of the curve, you'll see progress seem to stal

Progress is sigmoidal approaching a limit. If the minimum acceptable performance is into the logarithmic portion of the curve, you'll see progress seem to stall out in that manner. Keep in mind that SoTA self driving AI has a better track record than people outside of rare corner cases, but the liability issue is crippling, so it won't see widespread adoption divorced from autopilot until it's "perfect." Unlike full self driving, I don't think applications of AI have a functional floor that is well into diminishing returns.

mcphage 4y ago
> Progress is sigmoidal approaching a limit. Yep, but a lot of the current hype for GPT is that we're in the near-vertical segment of the sigmoid, and so people are like "woah, if that's where we are now, where will we be in 5-10 years?!". And the answer to that might be, pretty much where we are now. And maybe not, maybe it'll be better, but you can't assume linear improvements on a sigmoid curve.
- CuriouslyC 4y ago
  Considering that AI is just "taking off" now, I'd say we're still pretty early in the growth curve (the alternate hypothesis, that people have been oblivious for a while, seems unlikely). Regardless, I think even if we were well into the linear portion of the curve, we could still do a lot with layers of tooling on top of marginal improvements on current models and technology.
  - telotortium 4y ago
    We're in the hype cycle now at the moment, but the capabilities have been taking off for a few years. GPT-3 came out in 2020, and in retrospect, could have been trained several years before that, given that the 175 billion parameters of GPT-3 turned out to be way too many for its level of performance (as discovered by Chinchilla). I agree that there are a lot of refinements that can be made even with the current LLMs, but it's unclear how far LLMs will continue to improve.

mcphage 4y ago

> Progress is sigmoidal approaching a limit. Yep, but a lot of the current hype for GPT is that we're in the near-vertical segment of the sigmoid, and so people are like "woah, if that's where we are now, where will we be in 5-10 years?!". And the answer to that might be, pretty much where we are now. And maybe not, maybe it'll be better, but you can't assume linear improvements on a sigmoid curve.

CuriouslyC 4y ago
Considering that AI is just "taking off" now, I'd say we're still pretty early in the growth curve (the alternate hypothesis, that people have been oblivious for a while, seems unlikely). Regardless, I think even if we were well into the linear portion of the curve, we could still do a lot with layers of tooling on top of marginal improvements on current models and technology.
- telotortium 4y ago
  We're in the hype cycle now at the moment, but the capabilities have been taking off for a few years. GPT-3 came out in 2020, and in retrospect, could have been trained several years before that, given that the 175 billion parameters of GPT-3 turned out to be way too many for its level of performance (as discovered by Chinchilla). I agree that there are a lot of refinements that can be made even with the current LLMs, but it's unclear how far LLMs will continue to improve.