3 ms·
Anyone have a good idea of what threshold should be introduced into the Coalescing Queue? For example, on a slow server you could imagine a command coming in e
by rkalla 15y ago
Anyone have a good idea of what threshold should be introduced into the Coalescing Queue?
For example, on a slow server you could imagine a command coming in every second; you likely don't want to hold onto those commands for a second artificially to find out there are no new methods to try and combine it with; so you just want to answer right away.
Now your server is getting busier... 4 commands a second... do you want to artificially hold commands for 250ms+ or longer to see if there are new commands to combine with it?
I imagine not...
In the scenario where you are getting 500+ req/sec it makes a lot of sense to coalesce them, but at some point your queue needs to decide when to make a cut in the queue, take that slice of commands, process/combine them THEN send them to the server and it is that heuristic I'm curious about.
- willvarfar 15y agoI put in the batching point at: 1) over a certain time-limit, e.g. 100ms 2) over a certain number of commands, e.g. 2000 3) encountering a specific 'flush' command, that I injected sometimes to simply the logic Those were the numbers I, by trial and error, determined for my server. But remember, I didn't do SELECTs; I put everything I might SELECT into memory. So my numbers are rather application-specific perhaps. Its important to measure the average and worst case response times, not the best-case. Under light load, best-case times are from blocking calls, which is why conventional ORM looks good enough in prototyping ;)
- rkalla 15y agoThanks Will; I missed all that for some reason. Appreciate you sharing your actual threshold numbers (would have been my second question).