5 ms·

There's no such thing as harmless software any more. Our devices are too interconnected and we are too reliant upon them. Failure modes can cascade in unpredict

There's no such thing as harmless software any more. Our devices are too interconnected and we are too reliant upon them. Failure modes can cascade in unpredictable ways. "Harmless" systems often contain potentially harmful data; highly sensitive systems often share network resources with IoT junk. Some semi-hypothetical scenarios: A data leak in an appointment management app allows an abusive ex-partner to track down and kill their victim. Mobile malware spread by an in-game advertising network causes a massive DDoS of the cellular network. Some areas are left without cell coverage for several hours while operators scramble to reconfigure their base station equipment; thousands of 911 calls go unanswered until service is resumed. Poor encryption in a chat app allows an oppressive regime to identify, surveil and "disappear" human rights activists. A bug in an integrated logistics management system causes some parcels to be tagged with the wrong attributes. A consignment of temperature-sensitive medicine is inadvertently shipped in an unrefrigerated container; the mistake goes unnoticed until patients start dying, because the "smart" temperature sensor inside the package thought that the shipment was safe to transport at ambient temperatures. A disreputable dating app is hacked and the names of the users published, leading to several suicides. A wifi-enabled printer is connected to the secure side of a hospital network by an unwitting member of staff. The printer is pwned, bringing down the entire network for 36 hours. Two ICU patients die as a result of preventable errors, because their medical records were unavailable. Several models of aftermarket in-car entertainment units are pwned due to an unpatched vulnerability in Android. The units are connected to the CAN-bus in order to use the media controls on the steering wheel. The poorly-written malware intermittently floods the CAN-bus with malformed packets, causing sudden failures of the braking system in several cars. I could go on ad nauseum.

tener 10y ago
You are missing the big picture here. The key phrase is: "risk management". Anything that your are doing (or NOT doing) have inherent risks. You should manage those risks. If risk realises you will be losing money (in various ways, including litigation) or other assets (such as reputation, talented people, market share etc.). You can lower your exposure to the risk generally by spending money on it. You can hire more devs, testers, enact better policies, increase auditing etc. You can even buy insurance. What you are advocating for is myopic. You should decrease your risk exposure in a way that is most efficient. It may be with better software, but often this isn't the case. What the public can and should do is to work on the other part of the equation: make the failure more costly. You should "make them pay" - literally. If you could go to out of business or to prison because the software you have delivered is faulty, you will really make sure it isn't. Simple as that. EDIT: a few recent banking regulations are good examples. In general various regulators have demanded that the international banks setup demonstrably independent local companies with appointed heads that are criminally responsible for misdeeds done those local companies. Putting in prisons few developers isn't goint to fix anything - the company will simply hire another few. Threatening the people in power is the way to go.
- alistairSH 10y ago
  If you could go to out of business or to prison because the software you have delivered is faulty And that's the problem. If a civil engineer designed a bridge that maimed hundreds, he would lose his license. There is no license to be lost for software developers. What will be the cost to the company that built this shoddy justice system software? Will anybody go to jail or be personally held accountable? I doubt it (I can't think of cases where a developer has been personally responsible).
  - tener 10y ago
    If you are going to put to jail a single developer, you may as well put to jail the entire company. That in turn wouldn't be practical or fair. It isn't the developer who has signed the contract to deliver the faulty piece of software that started this whole discussion.
    - alistairSH 10y ago
      It isn't the developer who has signed the contract Yes, and that's the problem. The engineer didn't sign the contract either, but he DID sign his name on the design. There should be some equivalent in software. And just as not all "engineers" are professional engineers (licensed), not all software developers should need to be licensed. But, for something like descried in the original article, there damn sure ought to be somebody to hold accountable.
      - tener 10y ago
        > But, for something like descried in the original article, there damn sure ought to be somebody to hold accountable. Yes. Currently you can sue the company that delivered the stuff. Stop blaming the wrong people please.
        alistairSH 10y ago
        I could say the same about the civil engineer, or lawyers, or MDs. Are you really arguing that professional licensing isn't needed because we can always seek remedy through the court system?
noir_lord 10y ago
Here in the UK we had a large NHS trust (regional sub department of our nationalised health service) shut down for 3 days, operations cancelled, delayed, surgeons stood down over malware. http://www.computing.co.uk/ctg/news/2475950/lincolnshire-nhs-trust-cancels-operations-following-computer-virus-outbreak http://www.computing.co.uk/ctg/news/2475950/lincolnshire-nhs...
treehau5 10y ago
I could go on ad nauseum about all the potential non-software ways that could go a long way in helping someone intent on breaking the law break the law, but the common thread in all your hypotheticals is people who want to break the law will find ways to break the law. If you want to help fix our current societal issues that drive people to want to break the law, that's another story. We have armed guards, large vaults, and cameras in banking centers, but that doesn't stop people from trying to rob the bank. We have federal laws and regulations protecting people's mail, but that doesn't stop identity thieves from going into your mailbox or digging through your trash for banking statements. If a creepy stalker spots their ex's car driving down the road, what's stopping them from following her/him home? "But how would they find out what car they are driving?" you may rebute, well who decided to post what brand new car they drive on facebook? At some point, people have to take responsibility for the information they put out there about themselves. I wouldn't call into a radio show and blurt out my full name and address for everyone to hear before I gave my opinion. I think this leads to a different issue -- why is appointment management software asking for personally identifiable information? Maybe we could make a distinction -- if you deal with PII, then these standards should be placed upon you.

tener 10y ago

You are missing the big picture here. The key phrase is: "risk management". Anything that your are doing (or NOT doing) have inherent risks. You should manage those risks. If risk realises you will be losing money (in various ways, including litigation) or other assets (such as reputation, talented people, market share etc.). You can lower your exposure to the risk generally by spending money on it. You can hire more devs, testers, enact better policies, increase auditing etc. You can even buy insurance. What you are advocating for is myopic. You should decrease your risk exposure in a way that is most efficient. It may be with better software, but often this isn't the case. What the public can and should do is to work on the other part of the equation: make the failure more costly. You should "make them pay" - literally. If you could go to out of business or to prison because the software you have delivered is faulty, you will really make sure it isn't. Simple as that. EDIT: a few recent banking regulations are good examples. In general various regulators have demanded that the international banks setup demonstrably independent local companies with appointed heads that are criminally responsible for misdeeds done those local companies. Putting in prisons few developers isn't goint to fix anything - the company will simply hire another few. Threatening the people in power is the way to go.

alistairSH 10y ago
If you could go to out of business or to prison because the software you have delivered is faulty And that's the problem. If a civil engineer designed a bridge that maimed hundreds, he would lose his license. There is no license to be lost for software developers. What will be the cost to the company that built this shoddy justice system software? Will anybody go to jail or be personally held accountable? I doubt it (I can't think of cases where a developer has been personally responsible).
- tener 10y ago
  If you are going to put to jail a single developer, you may as well put to jail the entire company. That in turn wouldn't be practical or fair. It isn't the developer who has signed the contract to deliver the faulty piece of software that started this whole discussion.
  - alistairSH 10y ago
    It isn't the developer who has signed the contract Yes, and that's the problem. The engineer didn't sign the contract either, but he DID sign his name on the design. There should be some equivalent in software. And just as not all "engineers" are professional engineers (licensed), not all software developers should need to be licensed. But, for something like descried in the original article, there damn sure ought to be somebody to hold accountable.
    - tener 10y ago
      > But, for something like descried in the original article, there damn sure ought to be somebody to hold accountable. Yes. Currently you can sue the company that delivered the stuff. Stop blaming the wrong people please.
      - alistairSH 10y ago
        I could say the same about the civil engineer, or lawyers, or MDs. Are you really arguing that professional licensing isn't needed because we can always seek remedy through the court system?

noir_lord 10y ago

Here in the UK we had a large NHS trust (regional sub department of our nationalised health service) shut down for 3 days, operations cancelled, delayed, surgeons stood down over malware. http://www.computing.co.uk/ctg/news/2475950/lincolnshire-nhs-trust-cancels-operations-following-computer-virus-outbreak http://www.computing.co.uk/ctg/news/2475950/lincolnshire-nhs...

treehau5 10y ago

I could go on ad nauseum about all the potential non-software ways that could go a long way in helping someone intent on breaking the law break the law, but the common thread in all your hypotheticals is people who want to break the law will find ways to break the law. If you want to help fix our current societal issues that drive people to want to break the law, that's another story. We have armed guards, large vaults, and cameras in banking centers, but that doesn't stop people from trying to rob the bank. We have federal laws and regulations protecting people's mail, but that doesn't stop identity thieves from going into your mailbox or digging through your trash for banking statements. If a creepy stalker spots their ex's car driving down the road, what's stopping them from following her/him home? "But how would they find out what car they are driving?" you may rebute, well who decided to post what brand new car they drive on facebook? At some point, people have to take responsibility for the information they put out there about themselves. I wouldn't call into a radio show and blurt out my full name and address for everyone to hear before I gave my opinion. I think this leads to a different issue -- why is appointment management software asking for personally identifiable information? Maybe we could make a distinction -- if you deal with PII, then these standards should be placed upon you.