The intermediate result will all fit into a relatively small amount of memory. During inference you only need to keep layer outputs until the next layer's outp

The intermediate result will all fit into a relatively small amount of memory. During inference you only need to keep layer outputs until the next layer's outputs are computed. If we talk about memory bandwidth, it is space requirements that are important, not so much time complexity.