5 ms·
It's a red flag that the 1.2bil model has to fit in gpu memory, happy to be provided wrong when the code drops
by amrb 2y ago
It's a red flag that the 1.2bil model has to fit in gpu memory, happy to be provided wrong when the code drops
- arilotter 2y agoThat's not something that DisTrO solves, but there's plenty of research in that area! See https://arxiv.org/abs/2301.11913 https://arxiv.org/abs/2301.11913 , https://arxiv.org/abs/2206.01288 https://arxiv.org/abs/2206.01288 , https://arxiv.org/abs/2304.11277 https://arxiv.org/abs/2304.11277 etc :)