5 ms·
> This means the parameters of the trained model fit in something like 7GB (decoder only, half-precision floats) to 24GB (full model, full-precision) > you wou
by woojoo666 4y ago
> This means the parameters of the trained model fit in something like 7GB (decoder only, half-precision floats) to 24GB (full model, full-precision)
> you would probably want an enterprise cloud/data-center GPU like an NVIDIA A100, especially if running batches of more than one image.
That doesn't seem so bad.
looks up price of NVIDIA A100 - $20,000
oh...ok I'll probably just pay for the service then
- fennecfoxen 4y agop4d.24xlarge is only $33/hr! And you get 400 Gbe so it should be quick to load.
- nacs 4y agoI know you're half joking here but there are more consumer-affordable versions like the Geforce RTX 3090ti ($1600 for 24GB). It may not do CUDA work as fast as the A100 but it'll be able to run the model. For the half-precision version at 7GB there are a ton more options (the RTX 3060 has 12GB for example at ~$450).