3 ms·

Critical software infrastructure should be regulated the way critical physical infrastructure is. We don't trust the people who make buildings and bridges to "d

Critical software infrastructure should be regulated the way critical physical infrastructure is. We don't trust the people who make buildings and bridges to "do the right thing" - we mandate it with regulations and inspections. (When your software not working strands millions of people around the globe, it's critical) And this was just a regular old "accident"; imagine the future, when a war has threat actors trying to knock things out.

theideaofcoffee 2y ago
"We can't regulate the industry because then the US loses to China" or "regulation will kill the US competitive advantage!" responses I've had to suggesting the same and I just can't. But I agree with you 100%. If it's safety critical, it should be under even more scrutiny than other things, it shouldn't be left to self-regulating QA-like processes in profit seeking companies and has to have a bit more scrutiny before the big button gets pressed. Edit: Disclaimer: The quotes aren't mine, just retorts I've received from others when I suggest the R-word.
- janalsncm 2y ago
  > then the US loses to China Yeah it makes no sense. Was the US not losing to China when we own-goaled the biggest cybersecurity incident in history?
  - worik 2y ago
    > then the US loses to China Such a silly meme, too. Economics 101 China and USA would both benefit by halting the conflict and trading with each other
- Zigurd 2y ago
  Not to mention humans going extinct because regulators are to blame for there being no city on Mars. Because that's definitely the reason there's no city on Mars.
owl57 2y ago
Did you notice that the piece of software in question was apparently installed mostly in companies where regulations and inspections already override sysadmins' common sense? Are you sure the answer is simply more of the same?
- acdha 2y ago
  It’s not true that “common sense” is being overridden: most companies and sysadmins do need that baseline to avoid “forgetting” about things which aren’t trivial to implement (if you didn’t work in the field 10+ years ago, it was common to see systems getting patched annually or worse, people opening up SSH/Remote Desktop to the internet for convenience, shared/short passwords even for privileged accounts, vendors would require horribly insecure configuration because they didn’t want to hire anyone who knew how to do things better, etc.). There are drawbacks to compliance security but it has been useful for flushing all of that mess out. Even if it wasn’t wrong, that’s still the wrong reaction. We’re in this situation because so many companies were negligent in the past and the status quo was obviously untenable. If there is a problem with a given standard the solution is to make a better system (e.g. like Apple did) rather than to say one of the most important industries in the world can’t be improved because that’d require a small fraction of its budget.
- sitkack 2y ago
  I sure noticed how much snark you packed into two sentences!
- 0xbadcafebee 2y ago
  I've worked in these enterprise organizations for a long time. They don't run on common sense, or even what one might consider "business sense". Their existing incentives create bizarre behavior. For example, you might think "if a big security exploit happens, the stock price might tank". So if they value the stock price, they'll focus on security, right?. In reality what they do is focus on burying the evidence of security exploits. Because if nobody finds out, the stock price won't tank. Much easier than doing the work of actually securing things. And apparently it's often legal. And when it's not a bizarre incentive, often people just ignore risks, or even low-level failures, until it's too late. Four-way intersections can pile up accidents for years until a school bus full of kids gets T-boned by a dump truck. We can't expect people to do the right thing even if they notice a problem. Something has to force the right thing. The only thing I have ever seen force an executive to do the right thing is a law that says they will be held liable if they don't. That's still not a guarantee it will actually happen correctly, course. But they will put pressure on their underlings to at least try to make it happen. On top of that, I would have standards that they are required to follow, the way building codes specify the standard tolerances, sizes, engineering diagrams, etc that need to be followed and inspected before someone is allowed into the building. This would enforce the quality control (and someone impartial to check it) that was lacking recently. This will have similar results as building codes - increased bureaucracy, cost, complexity, time... but also, more safety. I think for critical things, we really do need it. Industrial controls, like those used for water, power (nuclear...), gas, etc, need it. Tanker and container ships, trains/subways, airlines, elevators, fire suppressants, military/defense, etc. The few, but very, very important, systems. If somebody else has better ideas, believe me, I am happy to hear them....
tedk-42 2y ago
Like everything, cheap, quick or good rule applies (pick 2). Software is pretty much always made cheaply and quickly. Even NASA will have b software blunders and have rockets explode mid flight.
TiredOfLife 2y ago
The regulations were the reason the companies were running Crowdstrike in the first place.
- 0xbadcafebee 2y ago
  I'm saying that a (different) regulation, standard, and inspection, should apply to the whole software bill of materials, as it relates to the critical-ness of the product. Like, if security is important, the security-critical components should be inspected/tested. That's how you build a building safely: the nails are built to a certain specification and the nail vendor signs off on that.

theideaofcoffee 2y ago

"We can't regulate the industry because then the US loses to China" or "regulation will kill the US competitive advantage!" responses I've had to suggesting the same and I just can't. But I agree with you 100%. If it's safety critical, it should be under even more scrutiny than other things, it shouldn't be left to self-regulating QA-like processes in profit seeking companies and has to have a bit more scrutiny before the big button gets pressed. Edit: Disclaimer: The quotes aren't mine, just retorts I've received from others when I suggest the R-word.

janalsncm 2y ago
> then the US loses to China Yeah it makes no sense. Was the US not losing to China when we own-goaled the biggest cybersecurity incident in history?
- worik 2y ago
  > then the US loses to China Such a silly meme, too. Economics 101 China and USA would both benefit by halting the conflict and trading with each other
Zigurd 2y ago
Not to mention humans going extinct because regulators are to blame for there being no city on Mars. Because that's definitely the reason there's no city on Mars.

owl57 2y ago

Did you notice that the piece of software in question was apparently installed mostly in companies where regulations and inspections already override sysadmins' common sense? Are you sure the answer is simply more of the same?

acdha 2y ago
It’s not true that “common sense” is being overridden: most companies and sysadmins do need that baseline to avoid “forgetting” about things which aren’t trivial to implement (if you didn’t work in the field 10+ years ago, it was common to see systems getting patched annually or worse, people opening up SSH/Remote Desktop to the internet for convenience, shared/short passwords even for privileged accounts, vendors would require horribly insecure configuration because they didn’t want to hire anyone who knew how to do things better, etc.). There are drawbacks to compliance security but it has been useful for flushing all of that mess out. Even if it wasn’t wrong, that’s still the wrong reaction. We’re in this situation because so many companies were negligent in the past and the status quo was obviously untenable. If there is a problem with a given standard the solution is to make a better system (e.g. like Apple did) rather than to say one of the most important industries in the world can’t be improved because that’d require a small fraction of its budget.
sitkack 2y ago
I sure noticed how much snark you packed into two sentences!
0xbadcafebee 2y ago
I've worked in these enterprise organizations for a long time. They don't run on common sense, or even what one might consider "business sense". Their existing incentives create bizarre behavior. For example, you might think "if a big security exploit happens, the stock price might tank". So if they value the stock price, they'll focus on security, right?. In reality what they do is focus on burying the evidence of security exploits. Because if nobody finds out, the stock price won't tank. Much easier than doing the work of actually securing things. And apparently it's often legal. And when it's not a bizarre incentive, often people just ignore risks, or even low-level failures, until it's too late. Four-way intersections can pile up accidents for years until a school bus full of kids gets T-boned by a dump truck. We can't expect people to do the right thing even if they notice a problem. Something has to force the right thing. The only thing I have ever seen force an executive to do the right thing is a law that says they will be held liable if they don't. That's still not a guarantee it will actually happen correctly, course. But they will put pressure on their underlings to at least try to make it happen. On top of that, I would have standards that they are required to follow, the way building codes specify the standard tolerances, sizes, engineering diagrams, etc that need to be followed and inspected before someone is allowed into the building. This would enforce the quality control (and someone impartial to check it) that was lacking recently. This will have similar results as building codes - increased bureaucracy, cost, complexity, time... but also, more safety. I think for critical things, we really do need it. Industrial controls, like those used for water, power (nuclear...), gas, etc, need it. Tanker and container ships, trains/subways, airlines, elevators, fire suppressants, military/defense, etc. The few, but very, very important, systems. If somebody else has better ideas, believe me, I am happy to hear them....

tedk-42 2y ago

Like everything, cheap, quick or good rule applies (pick 2). Software is pretty much always made cheaply and quickly. Even NASA will have b software blunders and have rockets explode mid flight.

TiredOfLife 2y ago

The regulations were the reason the companies were running Crowdstrike in the first place.

0xbadcafebee 2y ago
I'm saying that a (different) regulation, standard, and inspection, should apply to the whole software bill of materials, as it relates to the critical-ness of the product. Like, if security is important, the security-critical components should be inspected/tested. That's how you build a building safely: the nails are built to a certain specification and the nail vendor signs off on that.