4 ms·
> [1] The training code for AMD-135M is based on TinyLlama, utilizing multi-node distributed training with PyTorch FSDP. I thought PyTorch didn't work well wit
by bjt12345 2y ago
> [1] The training code for AMD-135M is based on TinyLlama, utilizing multi-node distributed training with PyTorch FSDP.
I thought PyTorch didn't work well with AMD architecture, and read of many people using JAX instead?