AlexNet #s IMO provide an excellent ballpark estimate of how well balanced compute and communication are in terms of both the framework and the underlying platf

AlexNet #s IMO provide an excellent ballpark estimate of how well balanced compute and communication are in terms of both the framework and the underlying platform. A platform that runs AlexNet well has excellent computation performance for the convolution layers but it also has excellent algorithms/communication for parallelizing the model/data by whatever means. Networks that attempt to minimize computation and/or communication are cool, but they should be considered in that light IMO. It's also a great estimate of the low-end for strong scaling. There's a lot of bread and butter machine learning at this level in my experience.