2 ms·
Oh, hello, queue theory and parallel processing theory! As you increase utilization (remove slack), throughput may grow, but latency grows more. If your nodes i
by nine_k 5y ago
Oh, hello, queue theory and parallel processing theory! As you increase utilization (remove slack), throughput may grow, but latency grows more. If your nodes interact and depend on each other, the tail latency can grow completely out of hand, in exchange to tiny gains in utilization, and the actual throughput may go down as utilization goes up past some point.
Always have some free disk space. Do not run your web servers at 100% CPU utilization. Overprovision your clusters. Add redundant network links. These ideas look completely obvious, don't they?
Now apply them at the org level. Always have someone with spare cycles. Leave some time at the end of the sprint when all planned tickets are done. Allocate time during the day to sit and think, or to experiment, or to take a walk. Let the person oncall to mostly sit around and watch the system operate instead of churning out something. Now this sounds a little bit crazy, doesn't it. So much efficiency is left on the table!
(Also, certain Nassim Taleb used to be popular around here; he wrote a whole book, "Antifragile", based on these ideas.)
- mbrodersen 5y agoNot crazy at all. Just smart thinking combined with a good understanding of how the world actually works. You want to automate things as much as possible so that people have time to observe, think, and improve things. For example, a manager who is always 100% busy, working 80 hour weeks, is a bad manager. One step away from a heart attack or a major team/system failure.
- natmaka 5y agoThis seems pertinent to me, and is a quite old concept. Jacques Ellul, for example, posited that our "technical system" aims at efficiency to the point of engulfing everything, in a detrimental way because (composed with physic laws) it leads to betting on economy of scale (gigantism, centralization, specialization...). Stephen Jay Gould "criticized the belief that evolution comes up with the best, or only, solution to a biological problem", and that organisms are always working towards fitness.
- Andys 5y agore: org level. This is especially true for anyone on call (pager duty). Leaving a bit in the tank every day increases ability to respond to sudden emergencies more intelligently and with less burnout afterwards.
- eternityforest 5y agoNassim Taleb also has a lot of stuff that goes against this kind of thing. The core of antifragile is that you should position yourself to gain from disorder rather than fight it. I kind of hate the whole thing, because while you do occasionally get some great innovation that way, he really undervalues the army of "Fragilistas" that take those innovations and make them reliable and practical. I think he'd probably be against "Max efficiency" thinking, but "Max effectiveness" thinking is still not the same as "Minimum major trouble". He's a bit too "Move fast and break things", which is kind of like max efficiency, if you define efficiency as "How much business growth can I pack in ten years". I've always thought that things should be planned with "Time firebreaks", that are absolutely excluded from all ahead of time planning whatsoever, with the intent of cleaning up any unexpected trouble you run into. Not quite a break time, those are planned ahead and known to be necessary.