4 ms·
But if the limiting factor is the data on which the models are trained and not the actual “computation” than this would be exactly expected right?
by hnaccount_rng 1y ago
But if the limiting factor is the data on which the models are trained and not the actual “computation” than this would be exactly expected right?
- Ldorigo 1y agoThe data might be the limiting factor of current transformer architectures, but there's no reason to believe it's a general limiting factor of any language model (e.g. humans brains are "trained" on orders of magnitude less data and still generally perform better than any model available today)
- hnaccount_rng 1y agoThat depends on whether these current learning models can really generalise or whether they can only interpolate within their training set