3 ms·
later he said HF's tensor has almost zero copy, so 720GB is not needed using this format
by netdur 3y ago
later he said HF's tensor has almost zero copy, so 720GB is not needed using this format
- leiframming 3y agoNo, I think the author gets this mixed up. Zero copy does not play a role here - the problem is how the model is being loaded in the steps that are shown in the article and they have nothing to do with Safe tensors' zero copy ability. Check out the article by Hugging face to see how to do it with only 360GB instead of 720: https://huggingface.co/docs/accelerate/usage_guides/big_modeling https://huggingface.co/docs/accelerate/usage_guides/big_mode...