I don't think it's that hard. You just need to count the tokens in and tokens out. I guess distinguishing between the cost of input versus output is a bit of a

I don't think it's that hard. You just need to count the tokens in and tokens out. I guess distinguishing between the cost of input versus output is a bit of a wrinkle. I think to make it convenient then you probably need an SDK that is a proxy for the OpenAI package. Maybe include a way to do streaming (for a better user experience) in Python but still get the normal synchronous flow somehow. That would be a nice feature that could help sell your service. The issue is that these things need to be integrated into the billing and there is a focus on costs, so it might incline people to think twice about paying for an external service since they are so cost-conscious while looking at that part of their system. You also have to count tokens in ahead of time anyway and need to avoid duplicating the time of doing that.