3 ms·
What would you recommend if requests are highly parameterized and some can be many orders of magnitude more taxing on the system than others?
by winternewt 2y ago
What would you recommend if requests are highly parameterized and some can be many orders of magnitude more taxing on the system than others?
- pastage 2y agoWe usually implement queueing on the route to those specific things that are vulnarable. If you can not discern what traffic does what, you just need move the rate limiter close to the application or the problem hot spot. It's perfectly valid to give a HTTPS response 429 from a backend and let your frontend handle that in some graceful way. The same is valid as an exception in code, the nearer the problem spot you get the harder it is to get right. EDIT clarification.
- kevincox 2y agoIn the abstract sense instead of pulling from queues round-robin you can assign "tokens" to each queue round robin. When the number of tokens a queue has is equal to the cost of the request reset the tokens and pull that request. This can also be used to handle priority. Maybe paying customers or customers on the enterprise plan get 2 tokens per round or their requests only have half of the cost.