4 ms·
What about NVLink? Does it plays a role here?
by pk-protect-ai 2y ago
What about NVLink? Does it plays a role here?
- rfoo 2y agoFor FlashMLA? No. The code here runs on one GPU only and do not have a builtin communication part.
- pk-protect-ai 2y agoBut for the training it does. You need to communicate gradient changes between GPUs.