16 ms·
> There is no latency Every chat bot I was ever forced to use has built-in latency, together with animated … to simulate a real user typing. It’s the worst of
by Semaphor 9mo ago
> There is no latency
Every chat bot I was ever forced to use has built-in latency, together with animated … to simulate a real user typing. It’s the worst of all worlds.
- rurban 9mo agoBecause they are all using some cloud service and external LLM for that. We not. We sell our users a strong server, where he has all his data and all his services. The LLM is local, and trained by us.
- williamcotton 9mo ago> to simulate a real user typing The models return a realtime stream of tokens.
- Semaphor 9mo agoThis was already the case before LLMs became a thing. This is still the case for no-intelligence step by step bots.