4 ms·
Given how slow AI inference is (and for training it doesn't matter at all), the advantage of it being a few milliseconds closer to the user is greatly diminishe
by ec109685 2y ago
Given how slow AI inference is (and for training it doesn't matter at all), the advantage of it being a few milliseconds closer to the user is greatly diminished. The latency to egress to a regional data center is inconsequential.
Good point about at the very least not exposing placement to customers. That is a definite win.
- DrStartup 2y agoWe’re gonna have so much local inference available.