3 ms·
With this card you can also run Open AI's Whisper with the Large model (the multilingual one!), as it requires 10GB.
by BatteryMountain 4y ago
With this card you can also run Open AI's Whisper with the Large model (the multilingual one!), as it requires 10GB.
- crabbycarrot 4y agoHighly recommend quantizing the model (https://pytorch.org/tutorials/recipes/recipes/dynamic_quantization.html https://pytorch.org/tutorials/recipes/recipes/dynamic_quanti...). I converted the large model to use int8, and I'm able to run it 5x real-time on CPU with pretty low RAM requirements with still very good quality.
- Const-me 4y agoMy implementation of Whisper uses slightly over 4GB VRAM running their large multilingual model: https://github.com/Const-me/Whisper https://github.com/Const-me/Whisper