2 ms·
There needs to be some level of conformity between instances, they stop being a heard and more of a zoo if the skew is too large. The workloads running on the i
by sitkack 6y ago
There needs to be some level of conformity between instances, they stop being a heard and more of a zoo if the skew is too large. The workloads running on the instances shouldn't be able to tell which instance type they are running on, or your workloads should be written such that it doesn't matter (but at some point it will). Things that grow and move together, wear together, so you will end up with a system that is designed against the empirical contract, not the stated one.
- jeffbee 6y agoMy approach has always been that somewhere in my fleet there is a heat sink that fell off and a CPU running at 400MHz. My last two jobs have started with me sitting down at my desk on day 1 and demonstrating this fact. After concluding that the zoo is unavoidable, the only thing left to do is write the software accordingly.
- saagarjha 6y ago> After concluding that the zoo is unavoidable, the only thing left to do is write the software accordingly. What do you do to write software accordingly? Make it detect when it's running on a dud? Have it run as best as it can anyways?
- jeffbee 6y agoSuicide is a good solution, if some higher-level thing will notice and move the task to another machine. Batch frameworks can kill slow shards, or re-assign their work to faster shards. Clients of online services can direct more traffic to working shards and less or none to slow or broken ones.