3 ms·
Because human-readable text-based formats are really inefficient to both download and load, especially when in the hundreds of GB range. And no human cares to r
by oefrha 4y ago
Because human-readable text-based formats are really inefficient to both download and load, especially when in the hundreds of GB range. And no human cares to read billions of weights.
- Yoric 4y agoAgreed. However, there are much better formats than Python pickles for exchanging binary data. As it is, using PyTorch means that you force your users to also use PyTorch, which is a shame, as libtorch (which is what makes PyTorch work) offers a much more portable format (which I suspect might also be more efficient at least in terms of raw size, but I haven't checked).