At least for LLMs and transformers this isn't relevant. Having 8x the chips and 8x the memory bandwidth is always better. Interchip communication for matrix mul

At least for LLMs and transformers this isn't relevant. Having 8x the chips and 8x the memory bandwidth is always better. Interchip communication for matrix multiplication against a constant left matrix with a tiny right matrix isn't bandwidth bound, only latency bound.