I'm a little bit tired of the Bitter Lesson copypasta. It doesn't seem to me like point 1 to 3 have been proven to be true about ConvNets. Sure, scaling convne

I'm a little bit tired of the Bitter Lesson copypasta. It doesn't seem to me like point 1 to 3 have been proven to be true about ConvNets. Sure, scaling convnets further help, I don't think anyone would argue that training a big model on more data would not at least see some improvement. I also think there are irrefutable lower bounds of computation that it would be important to consider when building these meta-methods. If your problem requires N steps of computation for a problem of N size, but your method can only always perform at most K steps of computation, no amount of data or compute you throw at it will result in the right solution.