They are doing a lot of calculation per question and request as well, its a big model and presumably uses quite a bit of GPU resources just to run it. They are

They are doing a lot of calculation per question and request as well, its a big model and presumably uses quite a bit of GPU resources just to run it. They are well past the sort of typical web business and marginal cost per customer that traffic in GPU/memory must be expensive.