3 ms·
I got it running using Colab Pro+ (immediately got a V100 40GB VRAM GPU) - the 7B model works with batch size of 8 and a max seq len of 1024
by arthurcolle 4y ago
I got it running using Colab Pro+ (immediately got a V100 40GB VRAM GPU) - the 7B model works with batch size of 8 and a max seq len of 1024
- koheripbal 4y agoSure, but the real value here is the 65B. Can you have multiple GPUs on colab?
- arthurcolle 4y agoI can't even get the 13B on colab to do inference with a very small sequence length.