It's not really clear to me that the OP is talking about hardware costs. If so, yeah, once you have enough scale and with a read-only service like an LLM, those

It's not really clear to me that the OP is talking about hardware costs. If so, yeah, once you have enough scale and with a read-only service like an LLM, those are perfectly linear. If it's about saving the users time, it's very non-linear. And if it's not a scalable read-only service, the costs will be very non-linear too.