3 ms·
> cause outages That's fine. The major LLM providers work like this. If you're out of credit, or hit your monthly recharge limit, it stops working, bringing do
by maeil 2y ago
> cause outages
That's fine. The major LLM providers work like this. If you're out of credit, or hit your monthly recharge limit, it stops working, bringing down prod with it if your product relies on it. Not heard anyone complain about the concept.
If it's really a problem for you, you can be all enterprisey and contact sales, then they'll be very excited to offer you extremely high limits and post billing.
This way everyone gets what they wants.
- danpalmer 2y agoTo be clear, the outages I'm referring to are not when you hit your billing cap. Try designing a billing system for a cloud provider that implements caps, while still retaining the performance necessary for the services you're providing to make sense, and without introducing huge, common, failure modes.