2 ms·
How are you handling the unit costs, rate limits, and model risk?
by jsemrau 2y ago
How are you handling the unit costs, rate limits, and model risk?
- philip1209 2y agoThat’s a deep question. Under the hood we use some LLM gateways like Helicone and https://usevelvet.com https://usevelvet.com to track cost per request and optimize it.