3 ms·

I believe it works in distributed training because the models never have time to diverge far enough to be "incompatible".