3 ms·

For FlashMLA? No. The code here runs on one GPU only and do not have a builtin communication part.

• pk-protect-ai 2y ago
  But for the training it does. You need to communicate gradient changes between GPUs.