2 ms·
GP is likely referring to network latency here. There's a tradeoff between smaller GPUs/etc at home that have no latency to use and beefier hardware in the clou
by a_t48 2y ago
GP is likely referring to network latency here. There's a tradeoff between smaller GPUs/etc at home that have no latency to use and beefier hardware in the cloud that have a minimum latency to use.
- yjftsjthsd-h 2y agoSure, but if the model takes multiple seconds to execute, then even 100 milliseconds of network latency seems more or less irrelevant