3 ms·
Faster inference won't save you
- shreyash3087 4mo agoThe latency table says it all. Cloud-to-cloud is 40ms for 20 turns. Hotel Wi-Fi is 16 seconds. You can halve inference time and still have a broken product on bad connections.
- Var1377 4mo agois this an LLM?
- Var1377 4mo agodoes this mean you can disconnect from the internet entirely with the agent loop still running?
- ramstar3000 4mo agoyes this is central to our thesis :)