4 ms·
The thing that trips me up along with request caps a lot is timeouts (briefly mentioned). There are so many network / API / retry timeouts set to arbitrarily la
by makmanalp 9y ago
The thing that trips me up along with request caps a lot is timeouts (briefly mentioned). There are so many network / API / retry timeouts set to arbitrarily large amounts (e.g. 30 seconds) that it's hard to predict how they'll interact, and often you don't know that some of these exist and at which layers they exist (application layer, network layer, DNS, etc).
How you often see this is that your request limiter keeps booting new requests (FIFO/drop tail), and then when you check your currently processing requests, they seem to be all in some busy loop, and hopefully you have some means of checking why. You check, and you find that they're all querying a service that accepts the request but doesn't respond ever (e.g. server down behind a proxy), but for some reason the timeout on the client side is set to 30 seconds so stuff keeps hanging until it's killed.
Anyway, this is to say that in any sufficiently complex networked system scenario, it helps to review all these timeouts holistically and in the context of your application.
- puzzle 9y agoYou also have the issue that you might be dropping new requests, but there's so much backlog that old requests eventually time out in droves, too. You might end up spending a lot of CPU cycles doing partial work on some requests, only to eventually see them hit their deadline. That's one of the reasons why Facebook has a mechanism that switches their FIFO queues to LIFO, once the server has passed a certain load/latency threshold.
- rrdharan 9y agoThereby inadvertently rewarding people who mash their reload button when the app/site is slow :)
- adrianratnapala 9y agoBut mashing reload is what users are supposed to do by design of the web. That is, errors in internetworked systems can happen at a level high enough that machines can't decide if it is sane to retry or not, they should simply bubble the error all the way back to the user and let her press F5 if she wants to. Modern UIs might hide this, but users well know the web is flaky and understand it in their bones. So when they press reload they are sending important information.
- sofaofthedamned 9y agoWhich are the ones that generate most load...
- kijin 9y agoHeh, timeouts are the bane of my life as well as my most amusing source of extra income. Client: I'm getting 504 Gateway Timeout errors on my site. Can you help? Me: After how many seconds? Client: About 30. Me: Reduce the timeout to 20 seconds in your PHP-FPM pool configuration. Client: But then I'll get even more timeouts! Me: No you won't. While you're at it, reduce it even further to 10 seconds. Client: What? That doesn't make any sense! Me: Let's try 5 seconds then. ... Silence ... Client: Hey, no more timeouts! But the widget on the sidebar is broken. Me: That widget was trying to access an API that is currently down. Due to the long timeout and poor caching, it also took down your whole website. I heard you paid $cheapCompetitor to make the widget. If you'd like me to make you a better version by Saturday, that would be $XXX.