Looking at their block diagrams: They have two large "cache-like" structures: The unified buffer (24 MiB) and the accumulators (4 MiB). Bandwidth between these

Looking at their block diagrams: They have two large "cache-like" structures: The unified buffer (24 MiB) and the accumulators (4 MiB). Bandwidth between these and the matrix multiply unit is high (167 GiB/s), bandwidth out of that complex is low. So it would seem that they just don't need a very large bandwidth out of that function complex.