3 ms·
DeepSpeed became popular soon after this post was originally published and is natively supported by many PyTorch training frameworks. https://www.deepspeed.ai
by minimaxir 4y ago
DeepSpeed became popular soon after this post was originally published and is natively supported by many PyTorch training frameworks.
https://www.deepspeed.ai https://www.deepspeed.ai
https://www.deepspeed.ai/training/ https://www.deepspeed.ai/training/
- dauertewigkeit 4y agoI tried that as well, but maybe I did not use it correctly. I did not see the full sharding that I was hoping for. I only saw results similiar to FSDP.
- cma 4y agoHow about flexflow? https://huggingface.co/transformers/v4.9.2/parallelism.html#flexflow https://huggingface.co/transformers/v4.9.2/parallelism.html#...