Yeah, the main issue is if they are charging per token it's essentially them adding an extra fee per response, like a gas fee in Etherium. But thinking about it

Yeah, the main issue is if they are charging per token it's essentially them adding an extra fee per response, like a gas fee in Etherium. But thinking about it as an API is really strange, I don't expect an API to do anything but respond to my query yet LLMs act like an API that has to explain why it did (or didn't) do what you asked every time.