4 ms·

The knobs don't just sound good on the surface, they provide real value and are critical to many large scale distributed deployments including multi-petabyte da

The knobs don't just sound good on the surface, they provide real value and are critical to many large scale distributed deployments including multi-petabyte data systems I have been chiefly responsible for. The idea that people won't understand the ramifications of temporarily-reduced availability, or intentional requests for data that could be slightly inconsistent is kind of silly to me. If your data engineers do not understand the basics of the data systems they are working on you need different engineers. Spanner does not simplify anything. It is just the old poor relational data model that has been replaced by the aggregate data model in any system of substance. The real value is that you get an RDB some of the time and then you wait until they bring the system back online to get your RDB back.

ztorkelson 9y ago
Thank you for sharing your perspective! My argument is not that such controls can't be reasoned about and applied to great effect. Rather, my argument is that having to do so is much more difficult, time consuming, and error prone than not. Perhaps it's just a difference of domain, but in my experience, there's real value to be had by a DBMS which doesn't require a team of trained and highly-disciplined "data engineers" to use effectively. Organizations are composed of individuals with varying levels of skill and areas of expertise, but even if everyone was both interested in and capable of building correctly functioning distributed applications under weak consistency models, I'd still prefer a database system which relieves those individuals from having to do so themselves. As I recall, Google came to the same realization, which helped motivate Spanner. They found that their highly capable engineers were spending a significant amount of time and effort working around the weaker consistency of Spanner's predecessors, to varying degrees of success. By developing Spanner, they were able to eliminate large swaths of application complexity, freeing up their software engineers to focus more on the inherent complexity of their business domain, rather than the incidental complexity of their chosen database. YMMV.

ztorkelson 9y ago

Thank you for sharing your perspective! My argument is not that such controls can't be reasoned about and applied to great effect. Rather, my argument is that having to do so is much more difficult, time consuming, and error prone than not. Perhaps it's just a difference of domain, but in my experience, there's real value to be had by a DBMS which doesn't require a team of trained and highly-disciplined "data engineers" to use effectively. Organizations are composed of individuals with varying levels of skill and areas of expertise, but even if everyone was both interested in and capable of building correctly functioning distributed applications under weak consistency models, I'd still prefer a database system which relieves those individuals from having to do so themselves. As I recall, Google came to the same realization, which helped motivate Spanner. They found that their highly capable engineers were spending a significant amount of time and effort working around the weaker consistency of Spanner's predecessors, to varying degrees of success. By developing Spanner, they were able to eliminate large swaths of application complexity, freeing up their software engineers to focus more on the inherent complexity of their business domain, rather than the incidental complexity of their chosen database. YMMV.