6 ms·

This reminds me of a meetup I attended last fall, they were talking about the Spectre/Meltdown issues. I asked the presenters if anything in chip manufacturing/

This reminds me of a meetup I attended last fall, they were talking about the Spectre/Meltdown issues. I asked the presenters if anything in chip manufacturing/verification processes had changed as a result of that and they seemed surprised. To me, when a software bug shows up in a critical system, that means you actually have a logistics bug. Airplane control software should not be allowed to have bugs. CPUs should not be allowed to have bugs. And OS's should not be allowed to crash (looking at you Microsoft). When one of these things happens, in my opinion the correct response is _not_ to just release fixes and workarounds and then say "we'll try really hard to not let it happen again." You do that, sure. But the first time you see airplane software malfunction, that means you need to change the way the software is written and released so that the whole class of issues will not ever happen again. You don't stop at a public apology, you don't fire the person that unintentionally wrote the bug. If you have to hire mathematicians to formally prove the critical paths of the software, you do that. If it costs 10x more to release bug-free software, oh well, you do that. All of these corporate people thinking they can save money by spending less on quality are extremely naive. You can do a financial analysis of this, but they're doing it wrong. Did you ever consider what the cost of a whole generation just not trusting air travel at all would be?

nickff 6y ago
Are you willing to pay 10x more for the product with that supposed extra reliability (100% vs 99.99966%)? Before you answer, you must remember that perfection cannot be proven ex-ante, it can only be assured. You should also keep in mind that real systems have fault modes aside from software bugs and hardware glitches, such as unanticipated edge cases and user error, which may dominate your actual failure statistics.
na85 6y ago
>But the first time you see airplane software malfunction, that means you need to change the way the software is written and released so that the whole class of issues will not ever happen again. This is pretty good intuition but often a systemic change is not economically feasible. For avionics software at least, a rewrite of the software would likely have to be recertified from scratch before it would be allowed to fly. We do, however, have several different quality assurance programs in Aerospace that are supposed to address this sort of thing. Once you identify the root cause, the process found to be deficient is supposed to have a Process Owner who is required to create a preventive and corrective action plan to prevent a recurrence, with more severe problems requiring more robust action plans. Done right, the process owner is supposed to be empowered to make the changes that need to be made. These systems tend to be evolutions of ISO 9000 as pioneered by Toyota (IIRC). They are highly bureaucratic and soul-sucking, but they are also the least-shitty solution that's been tried.
ashtonkem 6y ago
On one hand, I understand your sentiment, on the other hand even with these bugs air travel is as safe as it’s ever been. We’ve reached a point where fewer people die in air travel per year than at any other point in the history of air travel, and that’s before you account for the number of miles travelled. It’s almost ridiculous how safe air travel is on average.
- londons_explore 6y ago
  commercial air travel. Private planes and industrial planes still have an awful safety record. Most stats also exclude 'unrelated' deaths which happen during a flight (even though there is a good chance the changes in air pressure, stress, lack of medical care, and cramped conditions at least contributed to the death). Stats also often exclude terrorist or war shootdowns of commercial planes, which are starting to become significant.
  - bsder 6y ago
    > Private planes and industrial planes still have an awful safety record. I don't know about industrial, but I assume "private" is a combination of 1) private pilots suck and 2) too much catering to client. Kobe Bryant would be my unfortunate shining example of 2). The pilot either wanted to cater to Kobe or would get fired if he didn't, and so went up in weather that it was stupid to go up in. As for 1), I've seen far too many sleep-deprived, hungover, drunk, or stoned private airplane pilots. And this is on top of the fact that they probably aren't the most experienced pilots to begin with. What is it about piloting that seems to attract frat boys who never grew up?
- martinald 6y ago
  That was true until 737 MAX, which statistically must have been one of the most dangerous planes (or jets at least) in history. Very few miles and 2 complete hull loss incidents very close together. These bugs really do matter. You can have quite a lot of minor issues and get away with it, but when you hit a serious failure like the MAX had, even if only triggered 1 in 10,000 flights ends up with an awful lot of casualties.
LifeLiverTransp 6y ago
Naive does not even begin to describe it. You do not save money by writting software cheap. You are borrowing it from the future as tech debt & hidden bugs. All debts are owned and paid for - one way or another. Managers who claim to have overcome this - are paying one credit card with two new.
jacquesm 6y ago
You are about 100% right on the mark here. There is only one slight problem: people don't want to pay for very high quality software except in a very limited number of fields. In a way every real software improvement (not fancy language flavor 'x' of the year but entirely new ways of developing software) have always been with the main goals of writing software with fewer bugs faster. That's the whole reason we have abstractions, compilers, syntax checkers, statical analyzers and so on. In spite of all those, software still has bugs and budgets are still not sufficient to write bug free software. On another note: this problem is getting worse over time. As tools improved codebases got larger and the number of users multiplied at an astounding rate resulting in many more live instances of bugs popping up. After all, software that contains bugs but that is never run is harmless, only when you run buggy software many times does the price of those bugs really add up. Somewhere we took a wrong turn and we decided that more of the same is a better way to compete than to have one of each that is perfected and honed until the bugs have been (mostly...) ironed out.
cryptonector 6y ago
This is not about saving money! You can't simply shutdown manufacturing of Intel or other chips that have Spectre/Meltdown issues because that would leave us with essentially no usable CPUs for new computers! The Spectre/Meltdown issues are deep and architectural, not simple to fix. It's not just a batch of CPUs that's the problem, but all of them. Besides, if a CPU ships with a bug that can be fixed via a microcode patch, then it would be a tremendous economic waste for all humanity to throw those CPUs out. Even when new CPUs come out that can be shown not to have Spectre/Meltdown issues, it will take a long time to replace the installed base of those that do because it's not a matter of a little bit of money, but a matter of a great deal of money and opportunity costs. So microcode patches and software mitigations is all there is. Absolutist attitudes don't help.
Ididntdothis 6y ago
“But the first time you see airplane software malfunction, that means you need to change the way the software is written and released so that the whole class of issues will not ever happen again. “ This sounds pretty good in theory but in practice you will just trade the current set of issues against new issues. In reality, Systems and their interactions are so complex that there is no amount of software design that can avoid bugs and fixing them. We sure can improve but it would be naive to think you can design 100% reliability into something like an airplane.
Veserv 6y ago
You are correct, but airplane companies already do that for the most part and much much more. The difference in reliability between normal software and airplane software is so vast that "best practices" from normal software can not be applied to airplane software since that would be gross criminal negligence. To explain, in the 10 years prior to the 737-MAX problems there were 50,000,000 flights and software was not implicated in a single passenger air fatality. The average flight is ~5,000 KM which is ~4-5 hours. So, in ~250,000,000 flight-hours, there were two crashes due to software. A plane takes ~3 minutes to fall from cruising altitude, so we can model this as a downtime of 6 minutes per 250,000,000 hours which gives us an downtime of 1 in 2,500,000,000 or a 99.99999996% uptime (yes, that is 9 9s). In contrast, I think most software people would agree that AWS is high quality. The AWS SLA specifies a 99.99% uptime (1 in 10,000 downtime). So, by this metric, airplane software is 250,000x more reliable than normal high quality software. The point of this is that the standard for airplanes is almost inconceivably high compared to normal software. To think that they are incompetent or suggest that all they need to do is adopt X or Y common-sense/best-practice is a gross misunderstanding of what is being done and what needs to be done to improve. It would be like someone trying to tell a civil engineer making a 50-story skyscraper that they really need to adopt high quality wood construction techniques from makers of doghouses. To actually improve it, you need to consider practices 250,000x better than "best practices" and go from there. To put it another way, the solutions are actually really really good, unfortunately the problems are really really really really hard.
- jfim 6y ago
  Not to detract from your point that aeronautical industry software is reliable (it is), but the 737 MAXes that crashed were all new planes. There wasn't even 24 months between the first delivery of a MAX to the model being grounded. The issues with the MAX were also clearly preventable and there were multiple failures of the systems (regulators, internal reviews, etc.) that were in place to catch these kinds of issues. But as you point out, the aeronautical industry has an excellent track record for software reliability, if you evaluate reliability by hull losses. By other metrics, it's a bit more debatable (eg. the integer overflow for Dreamliners such that they need to be restarted at least every 248 days), but still keeps people moving safely.
sokoloff 6y ago
> If you have to hire mathematicians to formally prove the critical paths of the software, you do that. If it costs 10x more to release bug-free software, oh well, you do that. If you’re trying to keep planes from crashing at all costs, sure. If you’re trying to reduce deaths from travel, that’s a terrible plan. Every family that you price out of commercial air travel and convert over to private auto travel instead has been placed at significantly higher risk as a result of the excessive pursuit of safety. It’s the reason the FAA allows lap infants under 2 years old. Not because that’s “safe” in absolute terms, but because it’s safer than the likely alternative.

nickff 6y ago

Are you willing to pay 10x more for the product with that supposed extra reliability (100% vs 99.99966%)? Before you answer, you must remember that perfection cannot be proven ex-ante, it can only be assured. You should also keep in mind that real systems have fault modes aside from software bugs and hardware glitches, such as unanticipated edge cases and user error, which may dominate your actual failure statistics.

na85 6y ago

>But the first time you see airplane software malfunction, that means you need to change the way the software is written and released so that the whole class of issues will not ever happen again. This is pretty good intuition but often a systemic change is not economically feasible. For avionics software at least, a rewrite of the software would likely have to be recertified from scratch before it would be allowed to fly. We do, however, have several different quality assurance programs in Aerospace that are supposed to address this sort of thing. Once you identify the root cause, the process found to be deficient is supposed to have a Process Owner who is required to create a preventive and corrective action plan to prevent a recurrence, with more severe problems requiring more robust action plans. Done right, the process owner is supposed to be empowered to make the changes that need to be made. These systems tend to be evolutions of ISO 9000 as pioneered by Toyota (IIRC). They are highly bureaucratic and soul-sucking, but they are also the least-shitty solution that's been tried.

ashtonkem 6y ago

On one hand, I understand your sentiment, on the other hand even with these bugs air travel is as safe as it’s ever been. We’ve reached a point where fewer people die in air travel per year than at any other point in the history of air travel, and that’s before you account for the number of miles travelled. It’s almost ridiculous how safe air travel is on average.

londons_explore 6y ago
commercial air travel. Private planes and industrial planes still have an awful safety record. Most stats also exclude 'unrelated' deaths which happen during a flight (even though there is a good chance the changes in air pressure, stress, lack of medical care, and cramped conditions at least contributed to the death). Stats also often exclude terrorist or war shootdowns of commercial planes, which are starting to become significant.
- bsder 6y ago
  > Private planes and industrial planes still have an awful safety record. I don't know about industrial, but I assume "private" is a combination of 1) private pilots suck and 2) too much catering to client. Kobe Bryant would be my unfortunate shining example of 2). The pilot either wanted to cater to Kobe or would get fired if he didn't, and so went up in weather that it was stupid to go up in. As for 1), I've seen far too many sleep-deprived, hungover, drunk, or stoned private airplane pilots. And this is on top of the fact that they probably aren't the most experienced pilots to begin with. What is it about piloting that seems to attract frat boys who never grew up?
martinald 6y ago
That was true until 737 MAX, which statistically must have been one of the most dangerous planes (or jets at least) in history. Very few miles and 2 complete hull loss incidents very close together. These bugs really do matter. You can have quite a lot of minor issues and get away with it, but when you hit a serious failure like the MAX had, even if only triggered 1 in 10,000 flights ends up with an awful lot of casualties.

LifeLiverTransp 6y ago

Naive does not even begin to describe it. You do not save money by writting software cheap. You are borrowing it from the future as tech debt & hidden bugs. All debts are owned and paid for - one way or another. Managers who claim to have overcome this - are paying one credit card with two new.

jacquesm 6y ago

You are about 100% right on the mark here. There is only one slight problem: people don't want to pay for very high quality software except in a very limited number of fields. In a way every real software improvement (not fancy language flavor 'x' of the year but entirely new ways of developing software) have always been with the main goals of writing software with fewer bugs faster. That's the whole reason we have abstractions, compilers, syntax checkers, statical analyzers and so on. In spite of all those, software still has bugs and budgets are still not sufficient to write bug free software. On another note: this problem is getting worse over time. As tools improved codebases got larger and the number of users multiplied at an astounding rate resulting in many more live instances of bugs popping up. After all, software that contains bugs but that is never run is harmless, only when you run buggy software many times does the price of those bugs really add up. Somewhere we took a wrong turn and we decided that more of the same is a better way to compete than to have one of each that is perfected and honed until the bugs have been (mostly...) ironed out.

cryptonector 6y ago

This is not about saving money! You can't simply shutdown manufacturing of Intel or other chips that have Spectre/Meltdown issues because that would leave us with essentially no usable CPUs for new computers! The Spectre/Meltdown issues are deep and architectural, not simple to fix. It's not just a batch of CPUs that's the problem, but all of them. Besides, if a CPU ships with a bug that can be fixed via a microcode patch, then it would be a tremendous economic waste for all humanity to throw those CPUs out. Even when new CPUs come out that can be shown not to have Spectre/Meltdown issues, it will take a long time to replace the installed base of those that do because it's not a matter of a little bit of money, but a matter of a great deal of money and opportunity costs. So microcode patches and software mitigations is all there is. Absolutist attitudes don't help.

Ididntdothis 6y ago

“But the first time you see airplane software malfunction, that means you need to change the way the software is written and released so that the whole class of issues will not ever happen again. “ This sounds pretty good in theory but in practice you will just trade the current set of issues against new issues. In reality, Systems and their interactions are so complex that there is no amount of software design that can avoid bugs and fixing them. We sure can improve but it would be naive to think you can design 100% reliability into something like an airplane.

Veserv 6y ago

You are correct, but airplane companies already do that for the most part and much much more. The difference in reliability between normal software and airplane software is so vast that "best practices" from normal software can not be applied to airplane software since that would be gross criminal negligence. To explain, in the 10 years prior to the 737-MAX problems there were 50,000,000 flights and software was not implicated in a single passenger air fatality. The average flight is ~5,000 KM which is ~4-5 hours. So, in ~250,000,000 flight-hours, there were two crashes due to software. A plane takes ~3 minutes to fall from cruising altitude, so we can model this as a downtime of 6 minutes per 250,000,000 hours which gives us an downtime of 1 in 2,500,000,000 or a 99.99999996% uptime (yes, that is 9 9s). In contrast, I think most software people would agree that AWS is high quality. The AWS SLA specifies a 99.99% uptime (1 in 10,000 downtime). So, by this metric, airplane software is 250,000x more reliable than normal high quality software. The point of this is that the standard for airplanes is almost inconceivably high compared to normal software. To think that they are incompetent or suggest that all they need to do is adopt X or Y common-sense/best-practice is a gross misunderstanding of what is being done and what needs to be done to improve. It would be like someone trying to tell a civil engineer making a 50-story skyscraper that they really need to adopt high quality wood construction techniques from makers of doghouses. To actually improve it, you need to consider practices 250,000x better than "best practices" and go from there. To put it another way, the solutions are actually really really good, unfortunately the problems are really really really really hard.

jfim 6y ago
Not to detract from your point that aeronautical industry software is reliable (it is), but the 737 MAXes that crashed were all new planes. There wasn't even 24 months between the first delivery of a MAX to the model being grounded. The issues with the MAX were also clearly preventable and there were multiple failures of the systems (regulators, internal reviews, etc.) that were in place to catch these kinds of issues. But as you point out, the aeronautical industry has an excellent track record for software reliability, if you evaluate reliability by hull losses. By other metrics, it's a bit more debatable (eg. the integer overflow for Dreamliners such that they need to be restarted at least every 248 days), but still keeps people moving safely.

sokoloff 6y ago

> If you have to hire mathematicians to formally prove the critical paths of the software, you do that. If it costs 10x more to release bug-free software, oh well, you do that. If you’re trying to keep planes from crashing at all costs, sure. If you’re trying to reduce deaths from travel, that’s a terrible plan. Every family that you price out of commercial air travel and convert over to private auto travel instead has been placed at significantly higher risk as a result of the excessive pursuit of safety. It’s the reason the FAA allows lap infants under 2 years old. Not because that’s “safe” in absolute terms, but because it’s safer than the likely alternative.