4 ms·
Support for this was also added to [Fairscale](https://fairscale.readthedocs.io/en/latest/ https://fairscale.readthedocs.io/en/latest/) and [Fairseq](https://gi
by stephenroller 6y ago
Support for this was also added to [Fairscale](https://fairscale.readthedocs.io/en/latest/ https://fairscale.readthedocs.io/en/latest/) and [Fairseq](https://github.com/pytorch/fairseq https://github.com/pytorch/fairseq) last week. In particular, the Fairscale implementation can be used in any pyotrch project without requiring the use of the Deepspeed trainer.
- diptanu 6y agoWhat are the relevant commits in Fairseq for this? I couldn't figure out the changes by looking at the commits from last week.
- stephenroller 6y agohttps://github.com/pytorch/fairseq/pull/3331 https://github.com/pytorch/fairseq/pull/3331 and https://github.com/pytorch/fairseq/pull/3327 https://github.com/pytorch/fairseq/pull/3327