XLA, the JIT compiler in tensorflow could hide this from you. It's a thing of beauty. It can vectorize along multiples of the memory higherachies of CPUs, GPUs,

XLA, the JIT compiler in tensorflow could hide this from you. It's a thing of beauty. It can vectorize along multiples of the memory higherachies of CPUs, GPUs, and the family of TPUs.