3 ms·
I believe it's roughly the same size as the model files. If you look in the transformers folder you can see there are around 9 5gb files, so I would expect you
by mortsnort 1y ago
I believe it's roughly the same size as the model files. If you look in the transformers folder you can see there are around 9 5gb files, so I would expect you need ~45gb vram on your GPU. Usually quantized versions of models are eventually released/created that can run on much less vram but with some quality loss.
- foobarqux 1y agoWhy doesn't huggingface list the aggregate model size?
- bavell 1y agoI've been wondering this for literally years now...
- matcha-video 1y agoHuggingface is just a git hosting service, like github. You can add up the sizes of all the files in the directory yourself
- Auracle 1y agoThat’s what we have computers for though - to compute.
- simonw 1y agoI've been bugging them about this for a while. There are repos that contain multiple model weights in a single repo which means adding up the file sizes won't work universally, but I'd still find it useful to have a "repo size" indicator somewhere. I ended up building my own tool for that: https://tools.simonwillison.net/huggingface-storage https://tools.simonwillison.net/huggingface-storage
- deleted 1y ago[deleted]
- Gracana 1y agoHF does this for ggufs, and it’ll show you what quantizations will work on the GPU(s) you’ve selected. Hopefully that feature gets expanded to support more model types.
- halJordan 1y agoModel size = file for fp8, so if this was released at fp16 then 40-ish, if it's quantized to fp4 then 10ish
- slickytail 1y ago[dead]