3 ms·
> eventually your edge hardware is going to be able to infer a lot faster than the 50ms+ per call to the cloud. This is interesting. Is that based on any upcom
by gtirloni 2y ago
> eventually your edge hardware is going to be able to infer a lot faster than the 50ms+ per call to the cloud.
This is interesting. Is that based on any upcoming technology improvement already in the works?
- a_t48 2y agoGP is likely referring to network latency here. There's a tradeoff between smaller GPUs/etc at home that have no latency to use and beefier hardware in the cloud that have a minimum latency to use.
- yjftsjthsd-h 2y agoSure, but if the model takes multiple seconds to execute, then even 100 milliseconds of network latency seems more or less irrelevant
- datameta 2y agoComms is also the greatest battery drain for a remote edge system. Local inference can allow for longer operation, or operation with no network infra.