In a conv layer, weights are shared across many input features. E.g. assume a 1x1 conv layer with a 28x28x3 input. You only need to load 3 weights even though t

In a conv layer, weights are shared across many input features. E.g. assume a 1x1 conv layer with a 28x28x3 input. You only need to load 3 weights even though there are effectively 28x28=748 different dot products. In practice, the input and output activations can be stored on chip as well (except for the first layer) which means the ratio of operations to DRAM accesses can be incredibly high. For some real world examples, take a look at the classic Eyeriss paper[1] which finds a ratio of 345-285 for AlexNet and VGG-16 respectively. You can also check out the TPU paper[2] which places the ratio at >1000 for some unnamed CNNs. Compare that to your analysis which yields a ratio of 2. [1] https://people.csail.mit.edu/emer/papers/2017.01.jssc.eyeriss_design.pdf https://people.csail.mit.edu/emer/papers/2017.01.jssc.eyeris... [2] https://arxiv.org/pdf/1704.04760.pdf https://arxiv.org/pdf/1704.04760.pdf