4 ms·
Any source recommendations on howto find out the minimum hw requirementsto train LLM with Xbit and YB parameters?
by 3abiton 3y ago
Any source recommendations on howto find out the minimum hw requirementsto train LLM with Xbit and YB parameters?
- danielhanchen 3y agoGreat question - I'm actually not sure - I'll probably write up some minimum requirements - I know QLoRA's paper https://arxiv.org/pdf/2305.14314.pdf https://arxiv.org/pdf/2305.14314.pdf has an approximate calculation of the weights size, but not the gradient updates