4 ms·
Or it means that the compute on the inference nodes is more efficient? Or that it’s tenanted in a way that decreases oversaturation? Or you’re getting programma
by nostrebored 3y ago
Or it means that the compute on the inference nodes is more efficient? Or that it’s tenanted in a way that decreases oversaturation? Or you’re getting programmatic improvements in the inference layer that are being funded by the enterprise spend?
- sebzim4500 3y agoIf they had a code improvement that made inference faster without damaging capability they would roll it out everywhere. Compute is money, after all. Worst case just add a `sleep()` to the non-enterprise version.