4 ms·
This is a pretty limited view of distributed systems engineering, particularly at the high-end. Some topics that are essential for designing efficient, massivel
by jandrewrogers 12y ago
This is a pretty limited view of distributed systems engineering, particularly at the high-end. Some topics that are essential for designing efficient, massively distributed systems:
- Greedy routing theory i.e. protocol design that is robust in a Nash equilibria sense. It still amazes me how many distributed systems are built on protocol designs that are provably unstable and inefficient.
- Space decomposition data structures. The only class of data structure that is efficiently distributable at massive scales due to their mathematical relationship to space-filling curves. Hash tables are often inadvertently implemented as space decomposition structures but there is a much richer and more expressive universe of data structures that exist.
- Game theoretic schedule awareness. This is a concept sometimes used in massively parallel systems for HPC but distributed systems designers seem to be unaware of it. It allows extremely high throughput by eliminating much of the need for distributed coordination with respect to accessing shared resources because every process can dynamically schedule its operations based on its modeling of the decisions of other schedulers it interacts with such that it almost never conflicts. I've never seen a paper on it but people have designed systems based on it (hell, I've designed a number of massively parallel systems based on it).
- Practical topology. On the surface it is about understanding how to maximize the throughput of a fabric of switch fabrics and most messaging patterns used in distributed systems today are naive at an elementary level; HPC has a much better understanding of packet flow optimization. There is also the more theoretical algebraic topology that hints how you efficiently do computational operations over space decomposition structures (mentioned above) with minimal data motion. Algebraic topology is enormously relevant to massively distributed and parallel databases but I rarely meet people who understand it outside of Oracle and similar (not that most of them are using it well).
A lot of distributed systems were engineered without any awareness of these things. Most of their fundamental weaknesses follow from that. The kinds of distributed systems that you can design if you really know the above topics greatly exceed the capabilities of popular distributed systems used today.
To reduce it to a simple metric: if you can show me a competitive algorithm design for the Graph500 benchmark, you actually understand massively scalable distributed systems. The efficiency at scale demonstrated in that benchmark is so far beyond common distributed systems because the designers of the top entries actually understand the above points I raised.
- tburns 12y agoAny lists/specific resources you would suggest that cover the things you mention well? This is a topic I'm actively trying to break into the basics of :)
- gima 12y agoInstead of providing resources, continuation, or even references to these topics for interested minds to dive into, I find bashing and gloating. Or am I reading this completely wrong? Don't get me wrong, it's good to point out the writing's shortcomings, but as you appear to be knownledgeable in the field, please don't force others to begin from the bottom. Share your knownledge.
- zodiac 12y agoI see a listing of what the link lacks. It's not the parent's responsibility to provide references without being asked to do so - gathering resources for this kind of post can easily take 2-3x the effort of writing up the original post, especially if the parent's author hasn't studied the subject formally recently. Also, he doesn't know that there's demand for more references without first posting, and if we expect all such posts to contain references, we discourage their authors from writing them.
- Game_Ender 12y agoTook a little bit of reading but the OP is founder of company [0] making a massively scalable geospatial database. This is the closest article [1] I can find which has anything close to details. It lists the following technologies: * A low level stack that bypasses the OS for at least disk IO (supposed to provide a 2-3x throughput increase) * "polymorphic" space-filling curves for distributed parallel indexing * hyper-dimensional spatial sieves [2] * Allen’s Interval Algebra to parallelise SQL statements More discussion on some of the OP's previous hacker news posts [3]. [0] - http://spacecurve.com/ http://spacecurve.com/ [1] - http://www.it-director.com/blogs/Bloor_IM_Blog/2014/7/how-do-you-map-a-sphere-to-a-plane.html http://www.it-director.com/blogs/Bloor_IM_Blog/2014/7/how-do... [2] - http://www.google.com/patents/US20090182837 http://www.google.com/patents/US20090182837 [3] - https://news.ycombinator.com/item?id=7482151 https://news.ycombinator.com/item?id=7482151