5 ms·

My core question is, why? I understand that security can be difficult, but why is infrastructure that is able to operate effectively for many decades before mi

My core question is, why? I understand that security can be difficult, but why is infrastructure that is able to operate effectively for many decades before micro controllers were even a thing vulnerable to remote attacks. I get having monitoring systems for it that are accessible in a way they could be hacked and disrupted, but why is the core operational infrastructure that way? Command and control should be isolated and be using 50-70 pneumatic tech to control it. Building in such a way to allow it to be disrupted remotely is the core problem here. Just because you can, doesn't mean you should.

RandomThoughts3 2y ago
> I get having monitoring systems for it that are accessible in a way they could be hacked and disrupted Actually it’s very easy to isolate that part. One way network equipments with physical isolation have existed for decades. An optic fibre with only an emitter on one side and only a receiver on the other will do the trick.
- elmerfud 2y ago
  Fiber still relies on electronic circuits. While they can be isolated network wise they are not immune to attacks in the way a pneumatic system is.
  - RandomThoughts3 2y ago
    That’s irrelevant. Pneumatic systems don’t expose telemetry towards the network. If you want to do that - and you do because having someone next to your equipment all the time to monitor things is both a waste of time and money and very error prone - you will need a data diode.
Veserv 2y ago
It is cheaper, your product takes fewer people to operate, you can outsource the operations, if you deliver IoT solutions you get to call yourself a tech company which gets you valued at 30x earnings instead of 10x earnings, getting hacked does not affect your stock price, and the actual effect of getting hacked is actually minor because you get hacked by the functional equivalent of Dr. Evil who takes down water for millions of people or cripples a billion dollar business, then asks for the staggering sum of 1… million dollars.
- scrps 2y ago
  https://www.wired.com/story/oldsmar-florida-water-utility-hack/ https://www.wired.com/story/oldsmar-florida-water-utility-ha...
jcranmer 2y ago
A water treatment plant would need about 2 people to a shift (and 4 sets of people) to have 24/7 monitoring (one to watch the control screens, and one to handle tasks like running tests on water, handling deliveries, etc., that takes you away from the screens), and that basically doesn't change if you're a small facility making 10KGD of water or a large facility making 100MGD of water. There is serious economy of scale going on here. If you're a small facility servicing a few thousand people, you can't afford to have that kind of monitoring, and so you have to economize in various ways. One of the popular ways is pooling together with other small facilities so that you have one person doing that monitoring for several sites at once, which requires some form of remote operation. Furthermore, when I worked at a large water company, all of our network, even the telemetry to the various pumping stations dotted around the service area, was on a private network airgapped from the internet. But there's also economy of scale here; a large company servicing 1.5 million people in a large metropolitan area can afford to do custom fiber backhaul in a way that even a bunch of small companies in the rural Midwest cannot, and so the control systems end up being Internet-accessible because it's too expensive for them not to be.
- joe_the_user 2y ago
  Yeah, Small cities are quite willing to economize even if it means X risk some attacker will muck with the system. But the thing I'd take issue is the "can't afford it" part. Of course these cities could afford onsite worker. As gp pointed out, these districts operated long before the Internet and they could provide water then. But the neoliberal paradigm has appeared and suddenly the constant claim is no organization "can afford" not to do any given automation measure, no matter how illogical or dangerous. And so a key thing calling organizations on this baloney.
- elmerfud 2y ago
  I understand that monitoring and status is reasonable to optimize for cost and this comes with some tradeoffs. Loss of monitoring should not equal a loss if service not unless that monitoring is off line for an extended period. The actual functional control for ensuring a critical service like water is working should remain as an analog computer with something like pneumatics, or other such technology. These are robust and can continue to operate even when electronic circuits have failed. Loss of visibility, and loss of service should be separated. This should be the same even for power stations as well.
9x39 2y ago
As others have said in more detail: cost. So they enable remote control to cut costs. They don't want to pay a 24/7 on-site ops center. They take their chances and bolt-on security, and that's how the incentives work today.

RandomThoughts3 2y ago

> I get having monitoring systems for it that are accessible in a way they could be hacked and disrupted Actually it’s very easy to isolate that part. One way network equipments with physical isolation have existed for decades. An optic fibre with only an emitter on one side and only a receiver on the other will do the trick.

elmerfud 2y ago
Fiber still relies on electronic circuits. While they can be isolated network wise they are not immune to attacks in the way a pneumatic system is.
- RandomThoughts3 2y ago
  That’s irrelevant. Pneumatic systems don’t expose telemetry towards the network. If you want to do that - and you do because having someone next to your equipment all the time to monitor things is both a waste of time and money and very error prone - you will need a data diode.

Veserv 2y ago

It is cheaper, your product takes fewer people to operate, you can outsource the operations, if you deliver IoT solutions you get to call yourself a tech company which gets you valued at 30x earnings instead of 10x earnings, getting hacked does not affect your stock price, and the actual effect of getting hacked is actually minor because you get hacked by the functional equivalent of Dr. Evil who takes down water for millions of people or cripples a billion dollar business, then asks for the staggering sum of 1… million dollars.

scrps 2y ago
https://www.wired.com/story/oldsmar-florida-water-utility-hack/ https://www.wired.com/story/oldsmar-florida-water-utility-ha...

jcranmer 2y ago

A water treatment plant would need about 2 people to a shift (and 4 sets of people) to have 24/7 monitoring (one to watch the control screens, and one to handle tasks like running tests on water, handling deliveries, etc., that takes you away from the screens), and that basically doesn't change if you're a small facility making 10KGD of water or a large facility making 100MGD of water. There is serious economy of scale going on here. If you're a small facility servicing a few thousand people, you can't afford to have that kind of monitoring, and so you have to economize in various ways. One of the popular ways is pooling together with other small facilities so that you have one person doing that monitoring for several sites at once, which requires some form of remote operation. Furthermore, when I worked at a large water company, all of our network, even the telemetry to the various pumping stations dotted around the service area, was on a private network airgapped from the internet. But there's also economy of scale here; a large company servicing 1.5 million people in a large metropolitan area can afford to do custom fiber backhaul in a way that even a bunch of small companies in the rural Midwest cannot, and so the control systems end up being Internet-accessible because it's too expensive for them not to be.

joe_the_user 2y ago
Yeah, Small cities are quite willing to economize even if it means X risk some attacker will muck with the system. But the thing I'd take issue is the "can't afford it" part. Of course these cities could afford onsite worker. As gp pointed out, these districts operated long before the Internet and they could provide water then. But the neoliberal paradigm has appeared and suddenly the constant claim is no organization "can afford" not to do any given automation measure, no matter how illogical or dangerous. And so a key thing calling organizations on this baloney.
elmerfud 2y ago
I understand that monitoring and status is reasonable to optimize for cost and this comes with some tradeoffs. Loss of monitoring should not equal a loss if service not unless that monitoring is off line for an extended period. The actual functional control for ensuring a critical service like water is working should remain as an analog computer with something like pneumatics, or other such technology. These are robust and can continue to operate even when electronic circuits have failed. Loss of visibility, and loss of service should be separated. This should be the same even for power stations as well.

9x39 2y ago

As others have said in more detail: cost. So they enable remote control to cut costs. They don't want to pay a 24/7 on-site ops center. They take their chances and bolt-on security, and that's how the incentives work today.