>Sequence parallelism is the hardest parallelism, but it is required for longer sequence In terms of difficulty of implementation it's arguably much easier tha

>Sequence parallelism is the hardest parallelism, but it is required for longer sequence In terms of difficulty of implementation it's arguably much easier than pipeline parallelism, which I'd argue is the hardest kind (at least to implement it efficiently without bubbles), and takes the most lines of code to implement (especially in Jax, where sequence parallelism is almost trivial).