5 ms·
One of the things that I learned from Google is how much breaks on a machine that has somehow corrupted memory in a dynamically linked library. Every applicati
by btilly 9y ago
One of the things that I learned from Google is how much breaks on a machine that has somehow corrupted memory in a dynamically linked library. Every application that you deploy using that library is broken. By contrast if you deploy a statically linked application, it probably works.
If you deploy to enough machines in the cloud, you'll eventually hit this use case.
- kpcyrd 9y agoIf you deploy enough machines that corrupted memory becomes a common problem you should look at your healthchecks instead of static linking.
- btilly 9y agoYou need to do both. Large distributed jobs at Google can literally take several times the expected lifetime of an average server to run. When you run such a job, you must expect that it will wind up running on computers of all stages of repair, and some of the computers that it runs on will come out of commission during your job and will have to be replaced by others. This scale requires defense in depth. Health checks are necessary, but not sufficient. You also need to minimize the problems if the machine you're on is not yet known to be bad. And deal with double checking results because at that scale you can't trust the hardware. And so on.