3 ms·
Depends on if you are using tensor parallelism or pipeline parallelism, in the second case you don't need any sharing.
by YetAnotherNick 9mo ago
Depends on if you are using tensor parallelism or pipeline parallelism, in the second case you don't need any sharing.