4 ms·
Define "good edge compute" in a way that doesn't have expectations set by server-based inference. I don't mean this to sound like a loaded request - we simply c
by datameta 1y ago
Define "good edge compute" in a way that doesn't have expectations set by server-based inference. I don't mean this to sound like a loaded request - we simply can't expect to perform the same operations at the same latency as cloud-based models.
These are two entirely separate paradigms. In many instances it is quite literally impossible to depend on models reachable by RF like in an ultra-low power forest mesh scenario for example.
- echelon 1y agoWe're in agreement that not all problem domains are amenable to data center compute. Those that don't have internet, etc. But for consumer software that can be internet connected, data center GPU is dominating local edge compute. That's simply because the models are being designed to utilize a lot of VRAM.
- datameta 1y agoI think the concept/term Edge has been diluted in meaning... Near edge, far edge, etc. It includes everything from truly remote ultra-low power compute to essentially a branched local data center that is certainly closer to the cloud if you ask me. "Not on the trunk" isn't a sufficient threshold for me nor is "not the cloud" a good filter.