4 ms·
I am right now playing with it running it locally using ollama. It is a 19GB download and it runs nicely on a nvidia A100 GPU. https://ollama.com/library/qwq h
by fxj 2y ago
I am right now playing with it running it locally using ollama.
It is a 19GB download and it runs nicely on a nvidia A100 GPU.
https://ollama.com/library/qwq https://ollama.com/library/qwq
- delusional 2y agoRuns nicely on my AMD 7900XTX too.
- kkzz99 2y agoHow are AMD cards performing? I heard it was still very hit and miss in regards to support. Have you also tried things like F5-TTS?
- zamalek 2y ago7900XT here, previously 6900XT. Support for 6000 series and beyond is great on Linux. You have to use an override envar (Ollama has info on this in their readme). ComfyUI has instructions for anything based on torch. TensorFlow is unusable.
- elorant 2y agoCare to say how many tokens per second you're getting?
- delusional 2y agoFor the "How many r's in the word strawberry" total duration: 15.278476756s load duration: 14.982999ms prompt eval count: 47 token(s) prompt eval duration: 5ms prompt eval rate: 9400.00 tokens/s eval count: 377 token(s) eval duration: 15.257s eval rate: 24.71 tokens/s
- SubiculumCode 2y agoA100 geez. The privileged few.