4 ms·

The training data thing is a problem mainly for LLMs, so it might be a limitation if we purely scale up LLMs but there are other types of AI around too Chip sc

The training data thing is a problem mainly for LLMs, so it might be a limitation if we purely scale up LLMs but there are other types of AI around too Chip scaling still seems to be going pretty fast, and we may discover new ways to make better use of the chips we currently have, like better methods of quantisation, or just using more of them, which could get us just far enough to reach the self improvement threshold So we could end up hitting a wall with chip scaling or something but I don’t think it’s that likely

blibble 4y ago
> Chip scaling still seems to be going pretty fast it's not been exponential for years > So we could end up hitting a wall with chip scaling we did, years ago
- circuit10 4y ago
  “it's not been exponential for years” Really? Even a 5% generation-to-generation improvement would be exponential, it’s just 1.05 to the power of the generation. If it was linear you’d have benchmark results scaling by a fixed number of points each generation, which doesn’t seem to be a thing as far as I know
  - blibble 4y ago
    > Even a 5% generation-to-generation improvement would be exponential, it’s just 1.05 to the power of the generation. if you change the exponent from 2 to 1.05 at some point then it is no longer an "exponential" function (guess what happened to chip scaling?) if the exponent changes (EVER) then it's no longer "exponential", it's likely sigmoidal