You're right. My caveat not exactly accurate but I wanted to point out where DisTrO might comes in and why it's relevant here. I mean it reduces the communicat

You're right. My caveat not exactly accurate but I wanted to point out where DisTrO might comes in and why it's relevant here. I mean it reduces the communication overhead by more orders than DiLoCo.