5 ms·
I think this is for a different kind of rate limiting. E.g. you have a website with a couple of servers, and you want to limit the number of chatGPT requests th
by zinodaur 3y ago
I think this is for a different kind of rate limiting. E.g. you have a website with a couple of servers, and you want to limit the number of chatGPT requests those servers are making per day. 10ms is meaningless to the chatGPT 12s p50.
You could spend an hour making your own persistence/coordination solution - or spend a minute and 5 bucks to call this service.
- tazu 3y agoSo... redis?
- zinodaur 3y agoyep, you could spend the time to do that - and if you don't have a cache set up spend the time to set one up. Or, you could pay 5 bucks. I won't be using it, but I think there are some people, in some situations where the time savings are actually worth it. Kind of a funny niche
- true_religion 3y agoIn the time it takes you to research and vet this solution, you’d have already written or installed a rate limiter package in your language of choice.