3 ms·
Latency is a huge factor in performance, and local models often have a huge edge. Especially on mobile devices that could be offline entirely.
by griomnib 2y ago
Latency is a huge factor in performance, and local models often have a huge edge. Especially on mobile devices that could be offline entirely.
- deleted 2y ago[deleted]
- KoolKat23 2y agoDefinitely not when it comes to LLM's, the larger more useful local models are not that fast and latency is not an issue, just look at this Google models voice function or even openai's advanced voice.