3 ms·
HF uses chunk-level deduplication that greatly reduces the amount of physical storage needed for a given amount of logical storage. https://huggingface.co/docs
by anon373839 3mo ago
HF uses chunk-level deduplication that greatly reduces the amount of physical storage needed for a given amount of logical storage.
https://huggingface.co/docs/xet/en/deduplication https://huggingface.co/docs/xet/en/deduplication