3 ms·
I attached all 3 algorithms 1F1B (1 forward 1 backward), ZB1P (zero bubble pipeline parallelism) and DualPipe as a picture here: https://x.com/danielhanchen/sta
by danielhanchen 2y ago
I attached all 3 algorithms 1F1B (1 forward 1 backward), ZB1P (zero bubble pipeline parallelism) and DualPipe as a picture here: https://x.com/danielhanchen/status/1894937006352031832 https://x.com/danielhanchen/status/1894937006352031832 for those interested :)
- Bimos 2y agoMaybe add Chimera as well? https://arxiv.org/pdf/2107.06925 https://arxiv.org/pdf/2107.06925
- danielhanchen 2y agoOh more nice pictures :)
- isoprophlex 2y agoit looks as if Chimera has marginally less bubbles than DualPipe?
- alphan0n 2y agoOff topic, but this is the Rick and Morty episode where Rick creates a perfectly level space. The symmetry is uuugh.
- danielhanchen 2y agoYou'll have to refresh my memory :) Is there like a Youtube clip for it?
- Cyphase 2y agohttps://www.youtube.com/watch?v=-MwCJpEuC44 https://www.youtube.com/watch?v=-MwCJpEuC44