4 ms·

> The words "it's a miracle it works at all" routinely popped up in those conversations, which is... something you don't want to hear about any sort of power ge

> The words "it's a miracle it works at all" routinely popped up in those conversations, which is... something you don't want to hear about any sort of power generation - especially not nuclear - but it's true. Funny thing is, those are the exact words I use when talking to people about networking. And realistically anytime I dig deep into the underlying details of any big enough system I walk away with that impression. At scale, I think any system is less “controlled and planned precision” and more “harnessed chaos with a lot of resiliency to the unpredictability of that chaos”

lisper 1y ago
> those are the exact words I use when talking to people about networking Or the U.S. financial system. Or civilization in general.
- psunavy03 1y ago
  It ultimately comes down to shared norms, shared expectations, and trust.
  - _kb 1y ago
    Which is why the long tail impact of current times is frankly terrifying.
    - chairmansteve 1y ago
      Yes. The preppers are starting to look sane.
      - organsnyder 1y ago
        The preppers can only buy themselves a small amount of time, though—no more than a year or two. Eventually, their stockpiled supplies will run out, or some piece of equipment will need a replacement part. I'd much rather focus on "prepping" by building social resiliency, instead. The local community I'm plugged into is much stronger together than anything I could possibly build individually.
  - somenameforme 1y ago
    A bit of a tangent, but I don't think this is it. There are plenty of species with plenty of shared norms, expectations, and trust - but no civilization. And, vice versa, many of the greatest societies have been riddled with completely incompatible worldviews yet created amazing civilizations. Consider that Sparta and Athens were separated by only 130 miles, yet couldn't possibly have been further apart! The reason people work together is fundamentally the same reason you go to work - self interest. You're rarely there because you genuinely believe in the mission or product - mostly you just want to get paid and then go do your own thing. And that's basically the gears of society in a nutshell. But you need the intelligence to understand the bigger picture of things. For instance Chimps have intricate little societies that at their peak have reached upwards of 200 chimps. They even wage war over them and in efforts to expand them or control their territory. This [1] war was something that revolutionized our understanding of primates behaviors, which had been excessively idealized beforehand. But they lack the intelligence to understand how to bring their little societies up in scale. They understand full well how to kill the other tribe and "integrate" their females, but they never think to e.g. enslave the males, let alone higher order forms of expansion with vassalage, negotiated treaties, and so on. All of which over time trend towards where we are today, where it turns out giving somebody a little slice of your pie and letting him otherwise roam free is way more effective than just trying to dominate him. [1] - https://en.wikipedia.org/wiki/Gombe_Chimpanzee_War https://en.wikipedia.org/wiki/Gombe_Chimpanzee_War
    - namaria 1y ago
      > There are plenty of species with plenty of shared norms, expectations, and trust Citation needed on that one. > Consider that Sparta and Athens were separated by only 130 miles, yet couldn't possibly have been further apart! They spoke the same language, shared the same literature, practiced the same religion, had a long history of diplomatic ties. When the Persians razed Athens, they took refuge with the Spartans. > For instance Chimps have intricate little societies that at their peak have reached upwards of 200 chimps. Again, I don't think this claim stands to evidence. The so called chimp war you mention is about a group of about a dozen and a huge fight that broke out among them. That doesn't support the idea that they are capable of 200-strong 'intricate' groupings.
- BrandoElFollito 1y ago
  For me it is the financial system as a whole. I am an ex-scientist and an engineer and had a look at the books of my son who studies finance in the best finance school in the world (I am saying this to highlight that he will be one of the perpetrators, possibly with influence, of this mess) The things in there are crazy. There are whole blocks that are obvious but made to sound complicated. I spent some time on a graph just to realize that they ultimately talk about solving a set of two linear equations (midfle school level). Some pieces were not comprehensible because they did not make sense. And then bam! A random differential equation and explanation as it was the answer to the universe. With an incorrect interpretation. And then there are statistics that would make "sociology science" blush. Yes, they are so bad that even the, ahem, experts who do stats in sociology would be ashamed (no hate for sociology, everyone needs to eat, it is just that I was several times reviewer of thesises there and I have trauma afterwards). The fact that finance works is because we have some kind of magical "local minimum of finance energy" from which the Trumps of this world somehow did not maybe to break from (fingers crossed) by disrupting the world too much.
r3trohack3r 1y ago
This is one of the key insights in my early SRE career that changed how I viewed software engineering at scale. Components aren’t reliable. The whole thing might be duct tape and popsicle sticks. But the trick for SRE work is to create stability from unreliable components by isolating and routing around failures. It’s part of what made chaos engineering so effective. From randomly slowing down disk/network speed to unplugging server racks to making entire datacenters go dark - you intentionally introduce all sorts of crazy failure modes to intentionally break things and make sure the system remains metastable.
- ForOldHack 1y ago
  Message on a mug: "if carpenters built houses the way programmers write software, a woodpecker could destroy civilization." The syncronasation of a power grid ... Wow.
  - vkou 1y ago
    If houses could be torn down and recreated with the press of a key, we probably wouldn't have a housing shortage.
    - alpaca128 1y ago
      We would instead have HaaS, with monthly subscriptions for a license to use the house. Which can be randomly revoked at any moment if the company doesn't feel like supporting it is profitable enough, or if an AI thinks your electricity usage is suspicious and permabans you from using a home in the entire town.
      - organsnyder 1y ago
        Isn't that called "renting"?
      - vkou 1y ago
        So, in short, being a tenant in a low-regulatory environment. Tell me more about this paradise.
gerdesj 1y ago
"Funny thing is, those are the exact words I use when talking to people about networking" Computer networking is not the same. Our networks will not explode. I will grant you that they can be shite if not designed properly but they end up running slowly or not at all, but it will not combust nor explode. If you get the basics right for ethernet then it works rather well as a massive network. You could describe it as an internetwork. Basically, keep your layer 1 to around 200 odd maximum devices per VLAN - that works fine for IPv4. You might have to tune MAC tables for IPv6 for obvious reasons. Your fancier switches will have some funky memory for tables of one address to other address translation eg MAC to IP n VLAN and that. That memory will be shared with other databases too, perhaps iSCSI, so you have to decide how to manage that lot.
- lillecarl 1y ago
  You tried to nerdsnipe someone without mentioning L2 is effectively dead within datacenters since VXLAN became hardware accelerated in both Broadcom and "NVIDIA"(Mellanox) gear. And for those that don't need/care about L2 they don't even bother and run L3 all the way. EVPN uses BGP to advertise MAC addresses in VXLAN networks which solves looping without magic packets, scales better and is easier to introspect. And we didn't even get into the provider side which has been using MPLS for decades. A problem with high bandwidth networking over fiber is that since light refracts within the fiber some light will take a longer path than other, if the widow is too short and you have too much scattering you will drop packets. So hopefully someone doesn't bend your 100G fiber too much, if that isn't finicky idk what is, DAC cables with twinax solve it short-range for cheaper however.
chasd00 1y ago
I did a lot of work for a major airline earlier in my career and came away with same impression. I just couldn’t see how they kept planes in the air based on my experiences through out the organization. I think in a big enough org the sheer momentum keeps things moving despite all the fires happening constantly.

lisper 1y ago

> those are the exact words I use when talking to people about networking Or the U.S. financial system. Or civilization in general.

psunavy03 1y ago
It ultimately comes down to shared norms, shared expectations, and trust.
- _kb 1y ago
  Which is why the long tail impact of current times is frankly terrifying.
  - chairmansteve 1y ago
    Yes. The preppers are starting to look sane.
    - organsnyder 1y ago
      The preppers can only buy themselves a small amount of time, though—no more than a year or two. Eventually, their stockpiled supplies will run out, or some piece of equipment will need a replacement part. I'd much rather focus on "prepping" by building social resiliency, instead. The local community I'm plugged into is much stronger together than anything I could possibly build individually.
- somenameforme 1y ago
  A bit of a tangent, but I don't think this is it. There are plenty of species with plenty of shared norms, expectations, and trust - but no civilization. And, vice versa, many of the greatest societies have been riddled with completely incompatible worldviews yet created amazing civilizations. Consider that Sparta and Athens were separated by only 130 miles, yet couldn't possibly have been further apart! The reason people work together is fundamentally the same reason you go to work - self interest. You're rarely there because you genuinely believe in the mission or product - mostly you just want to get paid and then go do your own thing. And that's basically the gears of society in a nutshell. But you need the intelligence to understand the bigger picture of things. For instance Chimps have intricate little societies that at their peak have reached upwards of 200 chimps. They even wage war over them and in efforts to expand them or control their territory. This [1] war was something that revolutionized our understanding of primates behaviors, which had been excessively idealized beforehand. But they lack the intelligence to understand how to bring their little societies up in scale. They understand full well how to kill the other tribe and "integrate" their females, but they never think to e.g. enslave the males, let alone higher order forms of expansion with vassalage, negotiated treaties, and so on. All of which over time trend towards where we are today, where it turns out giving somebody a little slice of your pie and letting him otherwise roam free is way more effective than just trying to dominate him. [1] - https://en.wikipedia.org/wiki/Gombe_Chimpanzee_War https://en.wikipedia.org/wiki/Gombe_Chimpanzee_War
  - namaria 1y ago
    > There are plenty of species with plenty of shared norms, expectations, and trust Citation needed on that one. > Consider that Sparta and Athens were separated by only 130 miles, yet couldn't possibly have been further apart! They spoke the same language, shared the same literature, practiced the same religion, had a long history of diplomatic ties. When the Persians razed Athens, they took refuge with the Spartans. > For instance Chimps have intricate little societies that at their peak have reached upwards of 200 chimps. Again, I don't think this claim stands to evidence. The so called chimp war you mention is about a group of about a dozen and a huge fight that broke out among them. That doesn't support the idea that they are capable of 200-strong 'intricate' groupings.
BrandoElFollito 1y ago
For me it is the financial system as a whole. I am an ex-scientist and an engineer and had a look at the books of my son who studies finance in the best finance school in the world (I am saying this to highlight that he will be one of the perpetrators, possibly with influence, of this mess) The things in there are crazy. There are whole blocks that are obvious but made to sound complicated. I spent some time on a graph just to realize that they ultimately talk about solving a set of two linear equations (midfle school level). Some pieces were not comprehensible because they did not make sense. And then bam! A random differential equation and explanation as it was the answer to the universe. With an incorrect interpretation. And then there are statistics that would make "sociology science" blush. Yes, they are so bad that even the, ahem, experts who do stats in sociology would be ashamed (no hate for sociology, everyone needs to eat, it is just that I was several times reviewer of thesises there and I have trauma afterwards). The fact that finance works is because we have some kind of magical "local minimum of finance energy" from which the Trumps of this world somehow did not maybe to break from (fingers crossed) by disrupting the world too much.

r3trohack3r 1y ago

This is one of the key insights in my early SRE career that changed how I viewed software engineering at scale. Components aren’t reliable. The whole thing might be duct tape and popsicle sticks. But the trick for SRE work is to create stability from unreliable components by isolating and routing around failures. It’s part of what made chaos engineering so effective. From randomly slowing down disk/network speed to unplugging server racks to making entire datacenters go dark - you intentionally introduce all sorts of crazy failure modes to intentionally break things and make sure the system remains metastable.

ForOldHack 1y ago
Message on a mug: "if carpenters built houses the way programmers write software, a woodpecker could destroy civilization." The syncronasation of a power grid ... Wow.
- vkou 1y ago
  If houses could be torn down and recreated with the press of a key, we probably wouldn't have a housing shortage.
  - alpaca128 1y ago
    We would instead have HaaS, with monthly subscriptions for a license to use the house. Which can be randomly revoked at any moment if the company doesn't feel like supporting it is profitable enough, or if an AI thinks your electricity usage is suspicious and permabans you from using a home in the entire town.
    - organsnyder 1y ago
      Isn't that called "renting"?
    - vkou 1y ago
      So, in short, being a tenant in a low-regulatory environment. Tell me more about this paradise.

gerdesj 1y ago

"Funny thing is, those are the exact words I use when talking to people about networking" Computer networking is not the same. Our networks will not explode. I will grant you that they can be shite if not designed properly but they end up running slowly or not at all, but it will not combust nor explode. If you get the basics right for ethernet then it works rather well as a massive network. You could describe it as an internetwork. Basically, keep your layer 1 to around 200 odd maximum devices per VLAN - that works fine for IPv4. You might have to tune MAC tables for IPv6 for obvious reasons. Your fancier switches will have some funky memory for tables of one address to other address translation eg MAC to IP n VLAN and that. That memory will be shared with other databases too, perhaps iSCSI, so you have to decide how to manage that lot.

lillecarl 1y ago
You tried to nerdsnipe someone without mentioning L2 is effectively dead within datacenters since VXLAN became hardware accelerated in both Broadcom and "NVIDIA"(Mellanox) gear. And for those that don't need/care about L2 they don't even bother and run L3 all the way. EVPN uses BGP to advertise MAC addresses in VXLAN networks which solves looping without magic packets, scales better and is easier to introspect. And we didn't even get into the provider side which has been using MPLS for decades. A problem with high bandwidth networking over fiber is that since light refracts within the fiber some light will take a longer path than other, if the widow is too short and you have too much scattering you will drop packets. So hopefully someone doesn't bend your 100G fiber too much, if that isn't finicky idk what is, DAC cables with twinax solve it short-range for cheaper however.

chasd00 1y ago

I did a lot of work for a major airline earlier in my career and came away with same impression. I just couldn’t see how they kept planes in the air based on my experiences through out the organization. I think in a big enough org the sheer momentum keeps things moving despite all the fires happening constantly.