3 ms·
Yes, on a consumer 4090 card it's 12x faster than real-time. We'll benchmark some older cards as well for comparison. I think it should work pretty good with t
by jpcl 3y ago
Yes, on a consumer 4090 card it's 12x faster than real-time. We'll benchmark some older cards as well for comparison.
I think it should work pretty good with the Apple's MLX framework as well if anyone would be willing to convert it. :)
- WhackyIdeas 3y agoAnd totally private, as in no internet needed?
- jpcl 3y agoYes, you download the weights once from Huggingface and you can do whatever you want with it. :) We have no cloud APIs or usage tracking of any kind.
- kholk 3y agoSince it's 12x faster than real time on a 4090, I wonder how fast would it be on a small form factor device (a SBC); I get it as this is using CUDA, so I really wonder how would that perform on my nV Xavier NX (and the more common Nano's out there)...!