3 ms·
I don't look at the Jepsen reports as being a representation of hubris. Most of what Jepsen finds may have been preventable problems, but they are also fixable
by cbsmith 6y ago
I don't look at the Jepsen reports as being a representation of hubris. Most of what Jepsen finds may have been preventable problems, but they are also fixable problems, which is why I don't see hubris.
There's just a reality that we don't really build these systems on top of a distributed operating system. They're starting with an platform abstraction that is already terrible imperfect for the job. There is a ton of pressure to get something out quickly and make it accessible to a broad set of developers rather than to do it correctly (MongoDB is perhaps the most classic example of this). I was building big data systems in the early days, and the whole mentality was that you were building tools that were riddled with flaws, limitations, and outright bugs, but in the right context would make capabilities accessible to a broad set of developers that were otherwise completely off the table.
The truth is, if developers were already experts at distributed systems, they wouldn't need or want most of these tools; a LOT of the value is in accessibility and having something that is "good enough" rather than correct. The products reflect that more than they reflect hubris.
What I do see is a combination of marketing and customer ignorance that does believe in silver bullets and belief that these tools don't have to be understood to be effective. It's an unfortunate byproduct of focusing on that accessibility over correctness.