3 ms·
Latency is mostly a matter of hardware, model size and small tweaks. LLaMa running locally is very fast. And current LLMs could definitely be finetuned to inter
by redox99 3y ago
Latency is mostly a matter of hardware, model size and small tweaks. LLaMa running locally is very fast. And current LLMs could definitely be finetuned to interrupt conversations like you mention.
- Waterluvian 3y agoThis excites me. The biggest issue I have with assistants is the awkward cadence of any interaction with them. I can’t wait to see this actually existing in practice.