18 ms·

My advisor’s advisor worked with Lamport, and my advisor’s thesis was in distributed consensus in file systems. My advisor worked heavily on a lot of the Paxos

My advisor’s advisor worked with Lamport, and my advisor’s thesis was in distributed consensus in file systems. My advisor worked heavily on a lot of the Paxos follow up work. He told me that Paxos is actually so complicated that only Lamport really understands it. He said that after the original paper came out, everyone was trying to simplify it, but no one was successful. He said almost any implementation he saw (in those early years) contained serious errors. It’s a truly remarkable thing.

fulafel 7y ago
It almost seems like TLA paradoxically made systems harder to understand, since presumably it was the enabler for pushing the limit of complexity.
- contingencies 7y ago
  Isn't this a truism with all technology? One could say the same about, for example, human language itself or any domain-specific professional vocabulary. As programmers we are relatively unique in that we are encouraged to analyze, create and maintain the behavior of complex systems across disparate domains, which demands developing a certain comfort with new vocabularies and an independent capacity for identifying behavioral and structural similarities across systems and domains. Some ~relevant quotes: Tesler's Law of Conservation of Complexity: Every application has an inherent amount of irreducible complexity. The only question is who will have to deal with it - the user, the application developer, or the platform developer? - Larry Tesler (ca. 1984) With a good programming language and interface, one - even children - can create from scratch important simulations of complex non-linear systems that can help one's thinking about them. - Alan Kay (2016) An evolving system increases its complexity unless work is done to reduce it. - Meir Lehman Any sufficiently complex system acts as a black box when it becomes easier to experiment with than to understand. Hence, black-box optimization has become increasingly important as systems become more complex. - Google Vizier: a service for black-box optimization Golovin et al., KDD'17 We're more aware of simple processes that don't work well than of complex ones that work flawlessly. - Marvin Minsky, MIT AI lab co-founder Tainter's Law of Problem Solving: As easier solutions are exhausted, problem solving moves inexorably to greater complexity, higher costs, and diminishing returns. The endogenous nature of [the] buildup of complexity eventually makes the system fundamentally illegible to the human operator - a phenomenon that is ironic given that the fundamental aim of the control revolution is to increase legibility. - Ashwin Parameswaran (2012) It is time to unmask the computing community as a Secret Society for the Creation and Preservation of Artificial Complexity. - Edsger Dijkstra You're not going to find the best algorithm in terms of computational complexity by coding. - Leslie Lamport I have struggled with complexity my entire career. One of our key approaches for managing complexity is to 'walk away' and start fresh. Often new tools or languages force us to start from scratch which means that developers end up conflating the benefits of the tool with the benefits of the clean start. The clean start is what is fundamental. The simplest way of controlling complexity growth is to build a smaller system with fewer developers. - Terry Crowley Successful programming is all about managing complexity. Managing complexity has many aspects. You can eliminate, avoid, simplify, hide or automate complexity, and so on. - Keith Bentley The most robust programs isolate complexity in a way that lets significant parts of the system appear simple and straightforward and interact in simple ways with other components in the system. - Terry Crowley ... from https://github.com/globalcitizen/taoup https://github.com/globalcitizen/taoup
  - yanowitz 7y ago
    Great quotes and wow, is that a link worth following! If you’re reading this and haven’t gone to read more of the wisdom in taoup but you liked what’s above, I urge you to go do so now.
ahelwer 7y ago
I earnestly believe that "Paxos is impossible to understand" is a baseless meme, a sort of learned helplessness in the CS community. I understand Paxos perfectly well. You can see it all laid out there in the TLA+ spec, and explained in many videos. It is absolutely untrue that only Lamport understands Paxos.
- yodsanklai 7y ago
  I'm sure many people understand it. I personally find it very hard to fully understand, and "Paxos Made Simple" didn't help. About one year ago, I went through all the steps of the proof, I implemented it. I really tried to understand it. Today, without external reference, I probably wouldn't be able to implement it again. For such a short piece of code, it's surprisingly difficult.
richdougherty 7y ago
Check out Heidi Howard's work for some nice generalisations and variations of Paxos. Her approach using immutable state makes things more tractable. https://blog.acolyer.org/2019/03/08/a-generalised-solution-to-distributed-consensus/ https://blog.acolyer.org/2019/03/08/a-generalised-solution-t... https://youtu.be/KTHOwgpMIiU https://youtu.be/KTHOwgpMIiU

fulafel 7y ago

It almost seems like TLA paradoxically made systems harder to understand, since presumably it was the enabler for pushing the limit of complexity.

contingencies 7y ago
Isn't this a truism with all technology? One could say the same about, for example, human language itself or any domain-specific professional vocabulary. As programmers we are relatively unique in that we are encouraged to analyze, create and maintain the behavior of complex systems across disparate domains, which demands developing a certain comfort with new vocabularies and an independent capacity for identifying behavioral and structural similarities across systems and domains. Some ~relevant quotes: Tesler's Law of Conservation of Complexity: Every application has an inherent amount of irreducible complexity. The only question is who will have to deal with it - the user, the application developer, or the platform developer? - Larry Tesler (ca. 1984) With a good programming language and interface, one - even children - can create from scratch important simulations of complex non-linear systems that can help one's thinking about them. - Alan Kay (2016) An evolving system increases its complexity unless work is done to reduce it. - Meir Lehman Any sufficiently complex system acts as a black box when it becomes easier to experiment with than to understand. Hence, black-box optimization has become increasingly important as systems become more complex. - Google Vizier: a service for black-box optimization Golovin et al., KDD'17 We're more aware of simple processes that don't work well than of complex ones that work flawlessly. - Marvin Minsky, MIT AI lab co-founder Tainter's Law of Problem Solving: As easier solutions are exhausted, problem solving moves inexorably to greater complexity, higher costs, and diminishing returns. The endogenous nature of [the] buildup of complexity eventually makes the system fundamentally illegible to the human operator - a phenomenon that is ironic given that the fundamental aim of the control revolution is to increase legibility. - Ashwin Parameswaran (2012) It is time to unmask the computing community as a Secret Society for the Creation and Preservation of Artificial Complexity. - Edsger Dijkstra You're not going to find the best algorithm in terms of computational complexity by coding. - Leslie Lamport I have struggled with complexity my entire career. One of our key approaches for managing complexity is to 'walk away' and start fresh. Often new tools or languages force us to start from scratch which means that developers end up conflating the benefits of the tool with the benefits of the clean start. The clean start is what is fundamental. The simplest way of controlling complexity growth is to build a smaller system with fewer developers. - Terry Crowley Successful programming is all about managing complexity. Managing complexity has many aspects. You can eliminate, avoid, simplify, hide or automate complexity, and so on. - Keith Bentley The most robust programs isolate complexity in a way that lets significant parts of the system appear simple and straightforward and interact in simple ways with other components in the system. - Terry Crowley ... from https://github.com/globalcitizen/taoup https://github.com/globalcitizen/taoup
- yanowitz 7y ago
  Great quotes and wow, is that a link worth following! If you’re reading this and haven’t gone to read more of the wisdom in taoup but you liked what’s above, I urge you to go do so now.

ahelwer 7y ago

I earnestly believe that "Paxos is impossible to understand" is a baseless meme, a sort of learned helplessness in the CS community. I understand Paxos perfectly well. You can see it all laid out there in the TLA+ spec, and explained in many videos. It is absolutely untrue that only Lamport understands Paxos.

yodsanklai 7y ago
I'm sure many people understand it. I personally find it very hard to fully understand, and "Paxos Made Simple" didn't help. About one year ago, I went through all the steps of the proof, I implemented it. I really tried to understand it. Today, without external reference, I probably wouldn't be able to implement it again. For such a short piece of code, it's surprisingly difficult.

richdougherty 7y ago

Check out Heidi Howard's work for some nice generalisations and variations of Paxos. Her approach using immutable state makes things more tractable. https://blog.acolyer.org/2019/03/08/a-generalised-solution-to-distributed-consensus/ https://blog.acolyer.org/2019/03/08/a-generalised-solution-t... https://youtu.be/KTHOwgpMIiU https://youtu.be/KTHOwgpMIiU