3 ms·
They’ve been achieving ~two 9s for the past few months. Given the surge in traffic they must be receiving, that’s honestly pretty decent.
by sophiabits 4y ago
They’ve been achieving ~two 9s for the past few months. Given the surge in traffic they must be receiving, that’s honestly pretty decent.
- PaulKeeble 4y agoThey are doing a lot of calculation per question and request as well, its a big model and presumably uses quite a bit of GPU resources just to run it. They are well past the sort of typical web business and marginal cost per customer that traffic in GPU/memory must be expensive.