4 ms·
> - scaling (there is no cliff when resources get saturated just a bit of a "back pressure") If you haven't found the cliffs, you're just not scaling hard enou
by toast0 5y ago
> - scaling (there is no cliff when resources get saturated just a bit of a "back pressure")
If you haven't found the cliffs, you're just not scaling hard enough. Of course, they are hard to find them in 'normal' server patterns, but if you do unconvential things, the cliffs may be closer. I retired, but my last project hit a bottleneck with FreeBSD 12.0 on outgoing tcp connection rate that's much lower than any kernel bottleneck on incomming connection. I don't know the kernel bottleneck on incomming connections because I never hit it; maybe I've hit configured FD limits, but other than that, I've only been application limited other than ocassionally running into a bug that reduces capacity.
- rwaksmunski 5y agoAre you familiar with "vmstat -z"? It will show you any kernel limit that was hit under the "FAIL" column. When you know what is happening/failing its much easier to tune and scale. dtrace is also invaluable for this.
- toast0 5y agoNo I wasn't, that's quite nice, thanks. At work, we had tools to look at selected vm stats, but a fuller pictures is nice, and I don't have access to the tools I did anyway. The fail column of course only applies to explicit limits; not things like doing task X requires lock Y, so if you're doing X a lot on a 56-thread machine, you can only do it so much before lock contention grinds you to a halt. (OTOH, top usually shows what lock you're waiting on, so it's pretty easy to track from there, and I know there are better lock debugging tools, I've just never needed them because either the lock is blindingly obvious and I can work on it, or it's kind of subtle and the other fixer knows how to use the tools for subtle problems :)