3 ms·
We are talking about very big models which training requires an enormous amount of hardware. Not sure how scalable HuggingFace transformer for training such mod
by kkzz99 3y ago
We are talking about very big models which training requires an enormous amount of hardware. Not sure how scalable HuggingFace transformer for training such models is.
- minimaxir 3y agotransformers outsources those types of tasks to accelerate, also developed by Hugging Face: https://huggingface.co/docs/accelerate/index https://huggingface.co/docs/accelerate/index I can't speak to the efficacy of it at large scale, though.