3 ms·
Sorry about 1080 Ti - sadly Triton and Xformers support Cuda 7.0, so unless if OpenAI and Meta makes it support Cuda 6.0, we sadly don't support it. The main r
by danielhanchen 3y ago
Sorry about 1080 Ti - sadly Triton and Xformers support Cuda 7.0, so unless if OpenAI and Meta makes it support Cuda 6.0, we sadly don't support it.
The main reason is because from Turing, Tensor Cores got provided, and so the matrix matmuls are all different in Tensor Cores
- 3abiton 3y agoAny source recommendations on howto find out the minimum hw requirementsto train LLM with Xbit and YB parameters?
- danielhanchen 3y agoGreat question - I'm actually not sure - I'll probably write up some minimum requirements - I know QLoRA's paper https://arxiv.org/pdf/2305.14314.pdf https://arxiv.org/pdf/2305.14314.pdf has an approximate calculation of the weights size, but not the gradient updates