3 ms·
Well, I'd like it to respond in something close to real-time, and since I have a pretty good non-Nvidia GPU, it makes more sense to wait for the WebGPU port.
by tormeh 3y ago
Well, I'd like it to respond in something close to real-time, and since I have a pretty good non-Nvidia GPU, it makes more sense to wait for the WebGPU port.
- programd 3y ago7 tokens per sec on an i5-11400 CPU using llama.cpp - that's pretty real time for personal use I would think.