3 ms·

This general idea of randomization is definitely in use. Randomized jitter is fairly common in exponential backoff algorithms for networked APIs. The jitter smo

This general idea of randomization is definitely in use. Randomized jitter is fairly common in exponential backoff algorithms for networked APIs. The jitter smooths out the waves of retries when a service goes down. Without it, the thundering herd all arrives at once and can keep the service from coming back up (because caches are cold, for example). This turns out to be really, really effective [0]! Separately, this thesis has a comment that "Lack of extreme sensitivity is fundamentally a desirable property of systems" which I think has been underappreciated. I worked on a team that used a lot of "circuit breakers" [1], which intentionally introduce a very sharp change in system behavior based on error rate: once you get too many errors from a backend, you stop sending it _any_ traffic until it responds consistently with non-errors. But this whiplash can cause as many problems as it solves - and the problems were much harder to understand. Anyway - it's too bad these ideas didn't catch on more broadly. They seem really useful. --- [0] https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/ https://aws.amazon.com/blogs/architecture/exponential-backof... [1] https://en.wikipedia.org/wiki/Circuit_breaker_design_pattern https://en.wikipedia.org/wiki/Circuit_breaker_design_pattern

motohagiography 5y ago
Around this same time (1998) backoff thresholds on dampening and other issues with BGP convergence were benefiting from another recent and conceptually related paper on how random_drop being configured on tcp ports prevented SYN flooding. Lately, we have ChaosMonkey and other random failure generators for uptime as well. The thesis has the underlying principle so well articulated though. Adding randomness to improve robustness is a really beautiful idea. What caught me is that you found the effect was a performance increase in the overall system as a result of the additonal entropy. Nassim Talib took it up later and made it famous, but it's a deep principle. I was wondering how generalized we could say if, that you have a process of any kind, which is subject to another random process that stresses it, and it evolves and adapts in response to the random stressors, and performs better as a result, could we find a general accelerating principle for a given system? As in, at a level of abstraction, is this the same principle at work as in a random neural net? (Edit: this was just posted to HN frontpage, which runs down this rabbit hole: https://www.quantamagazine.org/anil-seth-finds-consciousness-in-lifes-push-against-entropy-20210930/ https://www.quantamagazine.org/anil-seth-finds-consciousness... )

motohagiography 5y ago

Around this same time (1998) backoff thresholds on dampening and other issues with BGP convergence were benefiting from another recent and conceptually related paper on how random_drop being configured on tcp ports prevented SYN flooding. Lately, we have ChaosMonkey and other random failure generators for uptime as well. The thesis has the underlying principle so well articulated though. Adding randomness to improve robustness is a really beautiful idea. What caught me is that you found the effect was a performance increase in the overall system as a result of the additonal entropy. Nassim Talib took it up later and made it famous, but it's a deep principle. I was wondering how generalized we could say if, that you have a process of any kind, which is subject to another random process that stresses it, and it evolves and adapts in response to the random stressors, and performs better as a result, could we find a general accelerating principle for a given system? As in, at a level of abstraction, is this the same principle at work as in a random neural net? (Edit: this was just posted to HN frontpage, which runs down this rabbit hole: https://www.quantamagazine.org/anil-seth-finds-consciousness-in-lifes-push-against-entropy-20210930/ https://www.quantamagazine.org/anil-seth-finds-consciousness... )