2 ms·
Network availability, latency, privacy, etc. many qualities to consider beyond model size and performance for applications.
by transitorykris 2y ago
Network availability, latency, privacy, etc. many qualities to consider beyond model size and performance for applications.
- happypumpkin 2y agoAnd cost-efficiency, if I'm using an LLM as an Siri-like assistant on my phone, most of the tasks I'll want it to do won't be that complicated and it would be a waste to send them to some SOTA LLM in the cloud, which I'll have to pay for by a monthly subscription or on a per-token basis.