3 ms·
It's really just a gpu vram limitation: affordable GPUs are rather memory starved. Fortunately people have started writing implementations for pipelining acros
by nullc 4y ago
It's really just a gpu vram limitation: affordable GPUs are rather memory starved.
Fortunately people have started writing implementations for pipelining across multiple gpus.
https://github.com/Ying1123/FlexGen https://github.com/Ying1123/FlexGen