6 ms·
There have been machines tolerant to CPU and Mem failures, and to a certain extent this sorta works on some of the higher end machines that support ram/cpu hotp
by StillBored 5y ago
There have been machines tolerant to CPU and Mem failures, and to a certain extent this sorta works on some of the higher end machines that support ram/cpu hotplug. (historically see hp/tandem/nonstop, sunos/imp, etc).
The problem is linux's monolithic model, doesn't work well for kernel checkpoint/restore despite it actually supporting hotplug cpu/ram it they have to be gracefully removed.
So, this is less about the machine being distributed, and more about the fact that linux is the opposite of a microkernel/etc that can isolate and restart its subsystems in the face of failure. Its also sorta funny that while these types of operations tend to need to be designed into the system, the last major OS's designed this way were done in the 1980's.
- dwohnitmok 5y agoI know of no OSes that are resilient to CPU cores producing wrong results (or incorrect mem results: I consider ECC a lower level concern that is not part of the OS), whereas a lot of distributed consensus algorithms have this built into their requirements. EDIT: I have heard through the grapevine that something like this might be done for aerospace, but I have no personal experience with that. I agree with parent. The major reason why programming on a single computer is easier than a distributed system is that we assume total resilience of various components that we cannot for a distributed system. From the article: > This offers hope that it is possible to some day abstract away the distributed nature of larger-scale systems. To do this is not a question of software abstractions, but hardware resilience. If we have a network which we can reasonably assume to have 100% uptime and absolutely no corruption between all its components then we can program distributed systems as single computers.
- catern 5y agoMost distributed consensus algorithms, or distributed systems in general, are not resilient to nodes producing arbitrary wrong results. That's the realm of systems like Bitcoin, which achieve such resilience by paying big performance costs. So it shouldn't be surprising that computers have the same lack of resilience.
- anonymousDan 5y agoSorry what? That is exactly the purpose of Byzantine fault tolerant consensus algorithms, which have been around for many years.
- catern 5y agoSure, that doesn't contradict what I said. They have been around for years, and they are very expensive and almost all systems do not use them.
- anonymousDan 5y agoThey are a lot more efficient than bitcoin. But agreed, not cheap.
- StillBored 5y agoThe tandems I listed above, originally used lock stepped processors, along with stratus/etc. edit: Googling yields few results that aren't actual books, Try this https://books.google.com/books?id=wBuy0oLXEuQC&pg=PA218&lpg=PA218&dq=tandem+nonstop+lockstep&source=bl&ots=8zSlQDK79w&sig=ACfU3U15AHxS58ij3JVSAt7hNZ839ZNJEg&hl=en&sa=X&ved=2ahUKEwjP7brlte72AhVWmmoFHRLtD3QQ6AF6BAgkEAM#v=onepage&q=tandem%20nonstop%20lockstep&f=false https://books.google.com/books?id=wBuy0oLXEuQC&pg=PA218&lpg=...
- dwohnitmok 5y agoAh well there you go. Had no idea they used lock stepping!
- gnufx 5y agoIt doesn't count as resilient in the mainframe sense, but in an effort to encourage system management, I ran the Node Health Check system on our "commodity" HPC cluster and found multiple failed DIMMs and a failed socket no-one had noticed. (I'd had enough alerts from that on a cluster I managed.)