4 ms·
How long does model loading take? Loading 19GB into a machine can't be instantaneous (especially if the model is a network share).
by sodality2 3y ago
How long does model loading take? Loading 19GB into a machine can't be instantaneous (especially if the model is a network share).
- loloquwowndueo 3y agoThere are no “network shares”. The typical way to store model data would be in a volume, which is basically local nvme storage.
- xena 3y agoWellllllll, technically there is LSVD which would let you store model weights in S3. God that's a horrible idea. Blog time!
- carl_dr 3y agoIt takes about 7s to load a 9GB model on Beam (they claim, and tested as about right), I imagine it is similar with Fly - I've not had any performance issues with Fly.