4 ms·
Or you can invent better models or discover more efficient ways to train existing ones. You know - do something other than dumb scaling up - like what Hinton (b
by p1esk 3y ago
Or you can invent better models or discover more efficient ways to train existing ones. You know - do something other than dumb scaling up - like what Hinton (backprop, 1987), Lecun (convnets, 1989), or Vaswani, et al. (transformers, 2017) did.
- FanaHOVA 3y ago> do something other than dumb scaling up This is exactly what people told OpenAI 8 years ago and look where we are now.
- p1esk 3y ago8 years ago the dumb scaling up was exactly what we needed. 8 years we’ve been riding that train. Don’t you think it’s time to try something new?
- voz_ 3y agoI love this comment. Very HN. You’re absolutely right, everyone should just try to make paradigm shifts in the field.
- p1esk 3y agoThe key word here is “try”. And we are not talking about “everyone“, just those who complain they don’t have access to $5k/hr GPU clusters.
- PoignardAzur 3y ago"Or better, you can do the same thing that three people managed to do in the entire industry in the last 36 years." I mean, don't get me wrong, I'm all for improvements in AI efficiency, but maybe there isn't that much low-hanging fruit to pick? Tons of papers get published on transformers optimization techniques and barely any of them seem to stick.