4 ms·

I've dealt with this before. The problem with doing a billing cap is that now you are bringing an out-of-band batch system (billing) in-band. The only way to

by jedberg 4mo ago

I've dealt with this before. The problem with doing a billing cap is that now you are bringing an out-of-band batch system (billing) in-band. The only way to do a dollar cap is to constantly calculate your bill. Capping it at 100K requests is a lot easier, because it's a single number that can be incremented easily in a distributed way. It's the same reason it took AWS forever to offer billing caps, and even today, they label it as best effort. They don't guarantee you won't pay more than your set limit.

MikeNotThePope 4mo ago
How about implementing a prepaid system where you set your budget, and if you exceed it, everything just pauses until you pay more?
- _heimdall 4mo ago
  That wouldn't change anything in this case. Either way you would need to (a) be able to know the state of billing in band before executing a request or (b) accept a slight out of band delay where cloudflare may allow a few extra requests through and eat the cost. It is a very solvable problem, I just don't think pre- vs postpaid changes it.
londons_explore 4mo ago
Seems like a weak excuse... If you build a billing system which can calculate bills with low latency 99.9% of the time, but perhaps fails when there is a net split for example, then you can simply let the company write off the loss for that 0.1% of the time the billing system is down.
- manwe150 4mo ago
  This seems the real answer. The question of whether the billing team can set accurate maximums is largely disjoint from the question of whether the engineering team shuts off service precisely at that moment.
nlitened 4mo ago
I am very sure what you’re saying is not true. It’s a viable-looking technical excuse, but you can easily understand that it’s false. Let’s make a thought experiment. CEO of Cloudflare introduces hard billing caps at a _business_ (not technical) level. Your organization will never be billed more than the level you set. Your app may stop, or may continue running, but above the monthly cap it’s free for you, it becomes Cloudflare’s expense if they didn’t pause the services. I guarantee you that in this case, all technical issues you’re talking about would be solved in three weeks, and your service would go down within 3 seconds after hitting the cap. If CEO decision were that any over-usage above the cap is deducted from employee bonuses from the specific product division that didn’t stop spending in time — all technical challenges would be solved in 48 hours. Currently, there are literally zero organizational incentives for CloudFlare to develop any usage caps
- AnthonyMouse 4mo ago
  You're describing a way to get the technical people to do the work, but that was never the problem. If you want them to do it you just tell them to, and then they spend however many hours doing it, instead of doing something else. All technical problems are like that. You throw labor hours and hardware at them and progress is made. The question is, how much does it take, and is it worth that much? Which in turn depends on how willing you are to patronize someone else if they don't.
  - nlitened 4mo ago
    That’s kinda my point. We don’t have usage caps not because they are technically hard or impossible to implement (they’re not). It’s because the leadership thinks (likely correctly) that implementing them would just hurt company’s bottom line.
    - jt2190 4mo ago
      > [L]eadership thinks (likely correctly) that implementing them would just hurt company’s bottom line. Exactly. There's little "win" here for the business, especially when the simpler "just cap at n requests" solution exists. Pursuing this solution: - introduces more technical complexity (increased cost of developing, testing and maintentance) - forces overage costs onto the vendor, who will then... increase prices to cover these costs, which upsets customers and makes the company less competitive. - doesn't improve cost prediction (since n requests can be costed out a $/request quite easily and reliably) (The cynical will say something about "the business has an incentive to _not_ fix this issue because they get paid for overages", but this assumes that Cloudflare has the, "monopolistic" market power to make customers unhappy and still charge whatever they want. In this case implementing billing caps wouldn't do anything because Cloudflare could just... charge whatever they want.)
ustad 4mo ago
The cost of each request or token may have a fixed cost - you do not have to query the main billing system for such things.
dbbk 4mo ago
Vercel achieved it perfectly fine
simonw 4mo ago
I'd be happy for the vendor to eat the difference if their accounting systems let me spend a little over my configured cap, personally. (I get that they're not happy to do that.)

by jedberg 4mo ago

MikeNotThePope 4mo ago

How about implementing a prepaid system where you set your budget, and if you exceed it, everything just pauses until you pay more?

_heimdall 4mo ago
That wouldn't change anything in this case. Either way you would need to (a) be able to know the state of billing in band before executing a request or (b) accept a slight out of band delay where cloudflare may allow a few extra requests through and eat the cost. It is a very solvable problem, I just don't think pre- vs postpaid changes it.

londons_explore 4mo ago

Seems like a weak excuse... If you build a billing system which can calculate bills with low latency 99.9% of the time, but perhaps fails when there is a net split for example, then you can simply let the company write off the loss for that 0.1% of the time the billing system is down.

manwe150 4mo ago
This seems the real answer. The question of whether the billing team can set accurate maximums is largely disjoint from the question of whether the engineering team shuts off service precisely at that moment.

nlitened 4mo ago

I am very sure what you’re saying is not true. It’s a viable-looking technical excuse, but you can easily understand that it’s false. Let’s make a thought experiment. CEO of Cloudflare introduces hard billing caps at a _business_ (not technical) level. Your organization will never be billed more than the level you set. Your app may stop, or may continue running, but above the monthly cap it’s free for you, it becomes Cloudflare’s expense if they didn’t pause the services. I guarantee you that in this case, all technical issues you’re talking about would be solved in three weeks, and your service would go down within 3 seconds after hitting the cap. If CEO decision were that any over-usage above the cap is deducted from employee bonuses from the specific product division that didn’t stop spending in time — all technical challenges would be solved in 48 hours. Currently, there are literally zero organizational incentives for CloudFlare to develop any usage caps

AnthonyMouse 4mo ago
You're describing a way to get the technical people to do the work, but that was never the problem. If you want them to do it you just tell them to, and then they spend however many hours doing it, instead of doing something else. All technical problems are like that. You throw labor hours and hardware at them and progress is made. The question is, how much does it take, and is it worth that much? Which in turn depends on how willing you are to patronize someone else if they don't.
- nlitened 4mo ago
  That’s kinda my point. We don’t have usage caps not because they are technically hard or impossible to implement (they’re not). It’s because the leadership thinks (likely correctly) that implementing them would just hurt company’s bottom line.
  - jt2190 4mo ago
    > [L]eadership thinks (likely correctly) that implementing them would just hurt company’s bottom line. Exactly. There's little "win" here for the business, especially when the simpler "just cap at n requests" solution exists. Pursuing this solution: - introduces more technical complexity (increased cost of developing, testing and maintentance) - forces overage costs onto the vendor, who will then... increase prices to cover these costs, which upsets customers and makes the company less competitive. - doesn't improve cost prediction (since n requests can be costed out a $/request quite easily and reliably) (The cynical will say something about "the business has an incentive to _not_ fix this issue because they get paid for overages", but this assumes that Cloudflare has the, "monopolistic" market power to make customers unhappy and still charge whatever they want. In this case implementing billing caps wouldn't do anything because Cloudflare could just... charge whatever they want.)

ustad 4mo ago

The cost of each request or token may have a fixed cost - you do not have to query the main billing system for such things.

dbbk 4mo ago

Vercel achieved it perfectly fine

simonw 4mo ago

I'd be happy for the vendor to eat the difference if their accounting systems let me spend a little over my configured cap, personally. (I get that they're not happy to do that.)