44 ms·
I tried that as well, but maybe I did not use it correctly. I did not see the full sharding that I was hoping for. I only saw results similiar to FSDP.
by dauertewigkeit 4y ago
I tried that as well, but maybe I did not use it correctly.
I did not see the full sharding that I was hoping for.
I only saw results similiar to FSDP.
- cma 4y agoHow about flexflow? https://huggingface.co/transformers/v4.9.2/parallelism.html#flexflow https://huggingface.co/transformers/v4.9.2/parallelism.html#...