5 ms·

It is not impossible to build a system that can handle this on x86_64, it's more a question of software architecture rather than hardware architecture - modern

by Qasaur 6y ago

It is not impossible to build a system that can handle this on x86_64, it's more a question of software architecture rather than hardware architecture - modern computers are insanely powerful.

pulse7 6y ago
They are powerful, but show me a x86_64 that has a decades long up-time and no failures...
- smabie 6y ago
  The point is that it's unnecessary. The cost of such a reliable machine is far too high. It's better to have a bunch of cheap machines and program some failover system. It's like the actor model. It's better for an actor to just fail instead of trying and recovering. Just make sure another actor can take its place and that the failures are isolated from each other. By using this method, you avoid the enormous cost it takes to make something very reliable. It's the 80/20 problem. Let's say we have a service than whenever it handles a requests has a 80% chance of succeeding and a 20% chance of dying. The easy way to lower the chance of failing is to just have two services that handle the same request. If one fails, the other one might succeed. Using this strategy, we've now just lowered our failure rate to 4%, instead of 20%, with very little additional work. The cost of making only one service have a 4% failure rate (instead of 20%) is so much higher than a service that has a 20% failure rate. This is the perfect analogy, imo, for mainframes. So much effort and cost has gone into making these things super reliable. But it turns out, we don't actually need a super reliable system. For example, NYSE ditched it's mainframes a long time ago, because just using x86_64 and Linux to achieve the reliability needed is much cheaper and easier. And you would be hard pressed to find a company that needs as much reliability as a security exchange. If mainframes aren't worth it for NYSE (the mainframe apologists would say that NYSE is exactly the kind of company that needs mainframes the most), then they probably aren't worth it for anyone at all.
  - pulse7 6y ago
    Good insight! I have to agree!
burntoutfire 6y ago
That software architecture will be extremely hard to pull off if you really want that kind of stability (distributed systems are HARD). Only the most hardcore engineering organizations can pull this off (i.e. not banks or airlines). Meanwhile, IBM is selling out-of-the-box solution that you just need to buy and plug in.
- zozbot234 6y ago
  It's less about it being hard (banks and airlines can pay for lots of software engineers if they need to) and more about it not being necessarily appropriate to every workload. Distributed systems must be designed assuming that any single node might fail at any time; even communication among nodes cannot be assumed to be reliable, and every communication step introduces latency. When a random processing error means that some airplane might fall out of the sky or some banking transaction might go unaccounted for, these problems become very relevant.

by Qasaur 6y ago

It is not impossible to build a system that can handle this on x86_64, it's more a question of software architecture rather than hardware architecture - modern computers are insanely powerful.

pulse7 6y ago

They are powerful, but show me a x86_64 that has a decades long up-time and no failures...

smabie 6y ago
The point is that it's unnecessary. The cost of such a reliable machine is far too high. It's better to have a bunch of cheap machines and program some failover system. It's like the actor model. It's better for an actor to just fail instead of trying and recovering. Just make sure another actor can take its place and that the failures are isolated from each other. By using this method, you avoid the enormous cost it takes to make something very reliable. It's the 80/20 problem. Let's say we have a service than whenever it handles a requests has a 80% chance of succeeding and a 20% chance of dying. The easy way to lower the chance of failing is to just have two services that handle the same request. If one fails, the other one might succeed. Using this strategy, we've now just lowered our failure rate to 4%, instead of 20%, with very little additional work. The cost of making only one service have a 4% failure rate (instead of 20%) is so much higher than a service that has a 20% failure rate. This is the perfect analogy, imo, for mainframes. So much effort and cost has gone into making these things super reliable. But it turns out, we don't actually need a super reliable system. For example, NYSE ditched it's mainframes a long time ago, because just using x86_64 and Linux to achieve the reliability needed is much cheaper and easier. And you would be hard pressed to find a company that needs as much reliability as a security exchange. If mainframes aren't worth it for NYSE (the mainframe apologists would say that NYSE is exactly the kind of company that needs mainframes the most), then they probably aren't worth it for anyone at all.
- pulse7 6y ago
  Good insight! I have to agree!

burntoutfire 6y ago

That software architecture will be extremely hard to pull off if you really want that kind of stability (distributed systems are HARD). Only the most hardcore engineering organizations can pull this off (i.e. not banks or airlines). Meanwhile, IBM is selling out-of-the-box solution that you just need to buy and plug in.

zozbot234 6y ago
It's less about it being hard (banks and airlines can pay for lots of software engineers if they need to) and more about it not being necessarily appropriate to every workload. Distributed systems must be designed assuming that any single node might fail at any time; even communication among nodes cannot be assumed to be reliable, and every communication step introduces latency. When a random processing error means that some airplane might fall out of the sky or some banking transaction might go unaccounted for, these problems become very relevant.