4 ms·
We already have well-documented, reliable tools for managing where a job is being processed, testing various aspects of a distributed job, and monitoring them.
by peterwwillis 13y ago
We already have well-documented, reliable tools for managing where a job is being processed, testing various aspects of a distributed job, and monitoring them. This is because distributed computing (and distributed systems by extension) have been used for decades by research institutes, governments, and corporations. But most people on HN don't know about them because they weren't developed on github by silicon valley start-ups. (Sorry if that sounds condescending)
What does OS-level mean, by the way? In the kernel? A userland tool installed by default by most Unix-like operating systems? Something else?
As an aside, he seems to be asserting that you need to implement Paxos to have consensus, or that you have to have consensus for a distributed system to work. But your application should determine your systems design, not the other way around. For example, part of the impetus behind Paxos is asynchronous algorithms. But what critical distributed systems do you know of that are asynchronous? For example, if someone's life was on the line, would you rather a synchronous or asynchronous process be used? How about a billion-dollar financial transaction?
Some more fun aspects of Paxos: asynchronous distributed systems cannot have fixed timing, and therefore cannot be a reliable real-time system. And Paxos is dependent on the lack of Byzantine events, which we all know is ridiculous in real-world applications; Byzantine events happen all the time! Between attacks by blackhats to admins mis-configuring services and servers literally catching on fire (causing unexpected corruption of data before the CRC wrapper is applied), shit happens.
But truly synchronous systems are a bitch and a half to build and maintain, so most people settle for partially-synchronous systems, and then call them either synchronous or asynchronous depending on what sounds better to their customers. Anyway, it's not realistic to just focus on Paxos.