Hardware is very much the limit. The more compute you have to train your models the better the models become. I've been working on a family of (new) deep lear

Hardware is very much the limit. The more compute you have to train your models the better the models become. I've been working on a family of (new) deep learning models which outperform everything else that's been published, at between x1e3 to 1e9 the compute on the same data. If I had the GPT4 training cluster I'd be able to run it on cifar10 without having to make sacrifices, yes 32x32 color images solved with a trillion parameters.