3 ms·
FWIW, it seems to me that a test like this would more likely compare network architecture & topology (and plain dumb luck) rather than hardware failure rates. S
by solardev 3y ago
FWIW, it seems to me that a test like this would more likely compare network architecture & topology (and plain dumb luck) rather than hardware failure rates. Since you're not controlling for other variables, and your sample size is likely to be tiny, it's probably just looking at the results of failover handling (or lack thereof) and regional network issues.
A cloud, after all, is just a set of someone else's data centers, abstracted and managed for large-scale resale. In the cloud you can have load balancers and reverse proxies and CDNs and database clones and robust DNS setups all handling failovers across multiple data centers. You can do the same thing across several smaller data centers that you rent or colo in. You can also do the same thing across several bare-metal Raspberry Pis you install at friends' houses around the world. But none of those will really be a good test of hardware failure rates, because they're subject to so many other things that can go wrong: buggy code, a mistake in config, different kernel versions or distro patches, firewall settings, small ISP outages, cut wires... you won't really be able to tell what is hardware-related vs software vs environmental vs user error.
It's not even really a good test of overall reliability because it's not an apples-to-apples comparison between host setups. It's entirely possible for a few homebrewed PCs set up by an experienced admin to be faster and more reliable than an unpatched EC2-micro setup in a single data center/availability zone. It's also possible for free cloud host to be faster and more reliable than some enterprise's botched $50k rollout that they didn't architect well.
High uptime isn't (just) the result of reliable hardware, but good failover and route management. Hardware will fail no matter what, but good hosts -- whether it's a 100-person team at a big cloud or a single wizened beardadmin -- anticipate that, plan for it, code for it, replicate your data, and regularly test the whole stack (and charge accordingly, or at least use economies of scale to benefit smaller customers with the lessons learned from bigger ones).
In a simple network test like yours, the geographic proximity (in network hops) between the source and destination are likely to have a far bigger impact (by reducing network variability, caching and firewall/anti-bot issues) than any actual hardware failures on either side. It's pretty rare for even cheap off-the-shelf hardware to fail out of the blue, and you're unlikely to catch it happening in real time unless the whole website goes down and the sysadmin just doesn't care enough to fix it, or it regularly breaks by design (like Low-Tech Magazine's solar server). On the other hand, a well-configured, redundant host probably has hardware failures all day long, every day, but you won't even notice because your traffic just seamlessly routes around it.
What you're doing is like visiting a corner store and a big grocery store every day, looking for the same can of soda. Most days both will have it in stock, but some days the big grocery store might be out. But it doesn't really tell you much about either store's behind-the-scenes logistics. Maybe the big grocer has a vertically integrated international freight network that brings in 10,000 cans every week, but they're often out because of their 50,000 customers. Meanwhile the corner store guy just buys a few cases from Costco once a year, but you happen to be their only customer out of 50 or so who likes that soda, so they never have to restock. It doesn't really tell you anything except that the testing methodology isn't sufficient for measuring that particular failure mode.