3 ms·
OOMs even in quantized mode on a 3090. What's a better option for personal use? > torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 43.71 GiB. GPU
by jbellis 2y ago
OOMs even in quantized mode on a 3090. What's a better option for personal use?
> torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 43.71 GiB. GPU 0 has a total capacity of 24.00 GiB of which 20.74 GiB is free.
- yellow_lead 2y agoNot sure if you mean in general, or options for this particular project, but Whisper should work for you.