3 ms·

I am a Lamport admirer. I gradually realized that Lamport is more of the godfather of distributed system than Hinton is to deep learning. Lamport is less promi

I am a Lamport admirer. I gradually realized that Lamport is more of the godfather of distributed system than Hinton is to deep learning. Lamport is less prominent than Shannon is to information theory. Shannon is the closest to any title of "gold-like" figure to a scientific discipline of universal relevance in modern society. Lamport specifically revealed a philosophical connections between computer systems and physics, in the parallel of distributed consensus to relativity theory. To me, the enlightenment is that, the relations between events happening in a distributed system, is more fundamental than their absolute ordering, thus the central role of an "observer". I haven't really analyzed if this realization is from Lamport's papers, or my general reading and thinking, but I am moderately confident that general readings are heavily influenced by Lamport's papers, or can be traced back to be compatible with Lamport's thinking. I have not seriously study if this connection is valid in depth, which might be another pure amateur speculation of mine. One thing I think Lamport falls short is that his writing is not easy to read and understand. I unconciously feel that Lamport (and Dario from Anthropic) probably share a hidden sense of intellectual supriority grew from their own experience throughout their career. So their writing (and Dario's gospel) all share a unchangable sense of narration from their own delicate and graceful ideas, much less of faciliating the understanding to their audience. In this cateogry, Shannon is abosolutely superior in any measure, in his writing, ideas are so naturally presented, although the implications of the ideas remain elusive due to the inherent depth. Also, among the 3 prominent figures of modern AI: Hinton/Bengio are more like Shannon, Lecun is closer to Lamport. Enough random rambling. Lamport, as indicated by the outweight presence in this list, is no doubt the single most important scientist in distributed systems.

toast0 12d ago
> the relations between events happening in a distributed system, is more fundamental than their absolute ordering The important thing in most distributed systems is having an order. Having a single observer serialize events as it receives them is so much more tractable than trying to use absolute order. Using absolute order requires very precise time synchronization which is hard; using absolute order requires knowing when you have received all the reports of events that already happened which is hard. Determining a designated observer isn't typically easy, but having it determine the order it observes events is easy. If two events happen at a similar time it's typically not a big deal which one is considered first as long as all nodes will agree on the result --- let the designated observer just pick the first one it sees works pretty well. If your report takes an unexpectedly long time to make it to the designated observer, then it won't be first and you'll deal. Much better than trying to figure out unknowable questions of relativity. :P
- throwaway894345 12d ago
  > absolute order requires very precise time synchronization which is hard Presumably relativity is the reason precise time synchronization (and thus absolute ordering) is hard.
  - toast0 12d ago
    That forms a basis for the difficulty. But then there's additional layers of difficulty in real networks where the path between nodes is often asymmetric, and you may also observe that elapsed time (A -> B -> A) is sometimes greater than elapsed time (A -> C -> B -> C -> A) or (A -> C -> B -> A) or (A -> B -> C -> A)
    - jeremyjh 12d ago
      Doesn't each node just need to know its exact path (or latency of same) to the time source each is synchronizing from? The path from node to node doesn't matter because we're ordering log entries from the timestamp of the receiving node. Imagine you take three atomic clocks, synchronize them, and move each within exactly 1 meter of one of those nodes, directly connected in an identical manner. Relativistic effects are constant. If there are elevation differences, you factor it once and done. The problem with this setup is not relativity. It is quantum uncertainty (and various interference sources, thermal radiation, etc).
      - toast0 11d ago
        How do you know the latency of the path if you haven't synchronized the clocks... You might be able to control the latency on a LAN, but once you have servers in different locations, good luck. (GPS helps a lot, of course...)
        jeremyjh 11d ago
        There is this command called 'ping' that tells you that. But in my example, I included the network topology as a known, static factor.
olooney 12d ago
Hot take of the day: Computer scientists are in denial about it, but CS is a branch of theoretical physics, not mathematics. You can point to this or that model of computation, such as lambda calculus or mu-recursive functions and try to claim its abstracted well beyond the particular laws of physics for some specific universe, but they all have some kind of rate limit built into them... and where does the motivation for this idea, that it takes something (time, space, work) to compute something ultimately come from? That's right - from underlying physics itself[1] - from the Bekenstein bound or Bremermann's limit or the like. Even apparently non-physically-realizable models of computation like non-deterministic Turing machines are ultimately informed by and motivated by concepts in physics... otherwise they would just be examples of chmess[2] and of no interest to anyone. Computer science is of course somewhat abstracted from the details, but no more so than, say, thermodynamics, where concepts like entropy or Gibbs free energy can be studied in the abstract without reference to whether we are talking about a gas of non-interacting molecules or the spins of a bunch of electrons trapped in a lattice. So, it's of no surprise whatsoever that the fundamental problems of distributed computing are ultimately the same as those found in the relativity of simultaneity[3]. You've all been studying the same things all along, just with different tools and at different levels of abstraction. [1]: https://en.wikipedia.org/wiki/Limits_of_computation https://en.wikipedia.org/wiki/Limits_of_computation [2]: https://link.springer.com/article/10.1007/s11245-006-0005-2 https://link.springer.com/article/10.1007/s11245-006-0005-2 [3]: https://en.wikipedia.org/wiki/Relativity_of_simultaneity https://en.wikipedia.org/wiki/Relativity_of_simultaneity
- projectileboy 12d ago
  I suspect this is old news for you, but just in case you haven't heard of it, check out Feynman's Lectures on Computation. A surprising amount of the book is still relevant, and it's fun how much he always brings everything back to the physics.
- moritz 12d ago
  Related hot take Maybe thats why one of the not-so-bad ideas how to go about distributed systems came from a guy who was trained as a physicist and used to complain to his fellow programmers that “a lot of systems actually break the laws of physics”[1]? > In distributed systems there is no real shared state (imagine one machine in the USA another in Sweden) where is the shared state? In the middle of the Atlantic? - shared state breaks laws of physics. State changes are propagated at the speed of light - we always know how things were at a remote site not how they are now. What we know is what they last told us. If you make a software abstraction that ignores this fact you’ll be in trouble.[2] [1]: “The Mess We’re In”, 2014 https://www.youtube.com/watch?v=lKXe3HUG2l4 https://www.youtube.com/watch?v=lKXe3HUG2l4 [2]: https://news.ycombinator.com/item?id=19708900 https://news.ycombinator.com/item?id=19708900
tombert 11d ago
> One thing I think Lamport falls short is that his writing is not easy to read and understand Interesting; I actually grew to be a fellow admirer of Lamport primarily because I actually found his papers to be a lot more approachable and relatively straightforward.

toast0 12d ago

> the relations between events happening in a distributed system, is more fundamental than their absolute ordering The important thing in most distributed systems is having an order. Having a single observer serialize events as it receives them is so much more tractable than trying to use absolute order. Using absolute order requires very precise time synchronization which is hard; using absolute order requires knowing when you have received all the reports of events that already happened which is hard. Determining a designated observer isn't typically easy, but having it determine the order it observes events is easy. If two events happen at a similar time it's typically not a big deal which one is considered first as long as all nodes will agree on the result --- let the designated observer just pick the first one it sees works pretty well. If your report takes an unexpectedly long time to make it to the designated observer, then it won't be first and you'll deal. Much better than trying to figure out unknowable questions of relativity. :P

throwaway894345 12d ago
> absolute order requires very precise time synchronization which is hard Presumably relativity is the reason precise time synchronization (and thus absolute ordering) is hard.
- toast0 12d ago
  That forms a basis for the difficulty. But then there's additional layers of difficulty in real networks where the path between nodes is often asymmetric, and you may also observe that elapsed time (A -> B -> A) is sometimes greater than elapsed time (A -> C -> B -> C -> A) or (A -> C -> B -> A) or (A -> B -> C -> A)
  - jeremyjh 12d ago
    Doesn't each node just need to know its exact path (or latency of same) to the time source each is synchronizing from? The path from node to node doesn't matter because we're ordering log entries from the timestamp of the receiving node. Imagine you take three atomic clocks, synchronize them, and move each within exactly 1 meter of one of those nodes, directly connected in an identical manner. Relativistic effects are constant. If there are elevation differences, you factor it once and done. The problem with this setup is not relativity. It is quantum uncertainty (and various interference sources, thermal radiation, etc).
    - toast0 11d ago
      How do you know the latency of the path if you haven't synchronized the clocks... You might be able to control the latency on a LAN, but once you have servers in different locations, good luck. (GPS helps a lot, of course...)
      - jeremyjh 11d ago
        There is this command called 'ping' that tells you that. But in my example, I included the network topology as a known, static factor.

olooney 12d ago

Hot take of the day: Computer scientists are in denial about it, but CS is a branch of theoretical physics, not mathematics. You can point to this or that model of computation, such as lambda calculus or mu-recursive functions and try to claim its abstracted well beyond the particular laws of physics for some specific universe, but they all have some kind of rate limit built into them... and where does the motivation for this idea, that it takes something (time, space, work) to compute something ultimately come from? That's right - from underlying physics itself[1] - from the Bekenstein bound or Bremermann's limit or the like. Even apparently non-physically-realizable models of computation like non-deterministic Turing machines are ultimately informed by and motivated by concepts in physics... otherwise they would just be examples of chmess[2] and of no interest to anyone. Computer science is of course somewhat abstracted from the details, but no more so than, say, thermodynamics, where concepts like entropy or Gibbs free energy can be studied in the abstract without reference to whether we are talking about a gas of non-interacting molecules or the spins of a bunch of electrons trapped in a lattice. So, it's of no surprise whatsoever that the fundamental problems of distributed computing are ultimately the same as those found in the relativity of simultaneity[3]. You've all been studying the same things all along, just with different tools and at different levels of abstraction. [1]: https://en.wikipedia.org/wiki/Limits_of_computation https://en.wikipedia.org/wiki/Limits_of_computation [2]: https://link.springer.com/article/10.1007/s11245-006-0005-2 https://link.springer.com/article/10.1007/s11245-006-0005-2 [3]: https://en.wikipedia.org/wiki/Relativity_of_simultaneity https://en.wikipedia.org/wiki/Relativity_of_simultaneity

projectileboy 12d ago
I suspect this is old news for you, but just in case you haven't heard of it, check out Feynman's Lectures on Computation. A surprising amount of the book is still relevant, and it's fun how much he always brings everything back to the physics.
moritz 12d ago
Related hot take Maybe thats why one of the not-so-bad ideas how to go about distributed systems came from a guy who was trained as a physicist and used to complain to his fellow programmers that “a lot of systems actually break the laws of physics”[1]? > In distributed systems there is no real shared state (imagine one machine in the USA another in Sweden) where is the shared state? In the middle of the Atlantic? - shared state breaks laws of physics. State changes are propagated at the speed of light - we always know how things were at a remote site not how they are now. What we know is what they last told us. If you make a software abstraction that ignores this fact you’ll be in trouble.[2] [1]: “The Mess We’re In”, 2014 https://www.youtube.com/watch?v=lKXe3HUG2l4 https://www.youtube.com/watch?v=lKXe3HUG2l4 [2]: https://news.ycombinator.com/item?id=19708900 https://news.ycombinator.com/item?id=19708900

tombert 11d ago

> One thing I think Lamport falls short is that his writing is not easy to read and understand Interesting; I actually grew to be a fellow admirer of Lamport primarily because I actually found his papers to be a lot more approachable and relatively straightforward.