3 ms·
Can we run this locally? (on normal machines?) How fast are generations on consumer machines?
by qup 3y ago
Can we run this locally? (on normal machines?)
How fast are generations on consumer machines?
- JonathanFly 3y agoIt's like 50% realtime on a 3090, not quite real time on a 4090. You can also use smaller models and it's a bit faster. 3080 is same speed, you don't need the extra memory.
- JonathanFly 3y agoActually I was just checking, and Bark isn't that close to maxing out GPU utilization. Running two instances on a 3090 seems like a throughput increase and the models fit. Update: And getting weird CUDA issues. Hmn...
- woodson 3y agoJust to add a datapoint: the main audioLM based models (not the BERT embedding part) fully utilize an RTX 2080 Ti.
- JonathanFly 3y agoMust be some low hanging fruit to optimize in Bark. It would be somewhat close to realtime if it was close to 100% and scaled linearly.
- shrx 3y agoFrom the README: > You can now use Bark with GPUs that have low VRAM (<4GB).
- code51 3y agoThis runs amazingly fast on M1/M2 Macs (MPS device).
- fivestones 3y agoCan you explain how you got this running? I'm trying it on an M1 Pro Mac now and it's taking about 3 minutes to generate 10 seconds of audio.
- dragonwriter 3y agoOn a laptop with a GTX 1650Ti (4GB), I’m seeing very roughly twice the audio time with “SUNO_USE_SMALL_MODELS” turned on, and about 4 times the audio time with that flag turn off. Given what people are reporting for high-end cards, that’s much higher than I expected, which seems to underline the descriptions that its not fully utilitizing higher-end cards.