8 ms·

I never understand why people need to connect industrial plants to the Internet. Do they actually need to control them over the Internet instead of on-site? An

by ArchD 12y ago

I never understand why people need to connect industrial plants to the Internet. Do they actually need to control them over the Internet instead of on-site? And, if they need to use the Internet on-site, can't they make an air gap and segregate the computers that can access the Internet from computers that can access the plant machinery?

ArchD 12y ago
OK, granted they may want to monitor the plant remotely. Then they could have a plant-connected machine dump UDP monitoring packets to an Internet-connected machine, and have the plant-connected machine block all incoming packets from the Internet-connected machine.
- gear54rus 12y ago
  It seems that there are many ways this could be done right (and does not seem a particularly hard challenge), it's just that people in charge probably were pretty much inept at that task. You know how it is, no one cared about it until it happened. It just wasn't a priority.
  - spacecowboy_lon 12y ago
    I get the impression from dealing with german companies that they tend to be very good at traditional "engineering" but when it comes to it/computers they are 10 or 15 years behind. I also think that in germany its considred that the good engineers and asociate profesionalas want go and work for firms like Audi.
    - fidotron 12y ago
      It's not just Germans. Anyone that isn't primarily in software has this phenomenon. Mobile phone makers, for example, are a disaster, and it's only having a whip wielded by a software company with some power over them that prevents it becoming a complete train wreck. In my experience the most dangerous are engineers in other domains that learned just enough programming to get the job done but can't understand the giant holes they've created and not run into.
      - spacecowboy_lon 12y ago
        Testify (Brother or Sister) having worked for a big telco we regarded the mobile side as grade inflated "amatuers" I still recall one of my colegues (working on the core IP network) being amused that one UK mobile provider was still using NT4 in their core network. Ill be nice and not write what we thought about the US cariers
- spacecowboy_lon 12y ago
  Well use private curcuits or go old school use modems with dialback ie you call the modem it disconects and calls you back on a hard coded number
HCIdivision17 12y ago
I remember a rule a controls engineer once told me: never connect the plant to the Internet. Nothing clever, no humorous quip, no deep insight. Just don't do it. If you do need to get data out to the living world, and you will, then you carefully set up individual firewall rules just for the data system - which very much can not drive the process system. This is where I begin ranting on the topic. Why? Because plants aren't secure. They're meant to run all the time by people who may or may not know how to properly use a mouse. There will be passwords taped to monitors, systems that automatically log in to prevent start up delays, and authentication of the order that it-better-just-work-by-default. Under no circumstances should that damn system ever, ever be exposed to the outside world. Not by Ethernet, wireless, or flash drive. It's a young innocent facing the cruel, brutal internet; it's going to get hurt. New plants are fancy and have highly trained workers with brilliant industrial IP wireless systems with state of the art VM servers and oh god did that guy just plug his phone charger into the damn wrapper HMI and now Windows media player has popped up and no one can acknowledge alarms (this partly why PCs tend to be in large metal boxes, that and dirt/water). No imagine that sort of silliness, but driven by the less savory from outside the intranet. Just don't risk it. You'll never have a problem, and no one is ever going to care enough to hurt the plant... Except but for that one time when suddenly all the convenience and hubris won't bring back the machine that just slagged itself from some malicious command from outside.
- superuser2 12y ago
  The terrifying part of this is that computers with Windows Media Player installed are running critical infrastructure. Shouldn't that be a stripped-down Linux machine with perfectly understood characteristics and close-to-zero attack surface? Yes, your scenario is bad I can totally see it happening, but the problem is not that an employee plugged in his phone, it's that you are using a desktop OS for office workers for controls that really non-optionally need to always just work. Maybe it's not a Windows Media Player launch, but what happens if Java shows up in the taskbar wanting an update, or your anti-virus software (yikes) wants to bug you about updating its definitions, or a modal dialog from the OS comes up? The fact that those are even things that can happen is pretty scary.
  - 12y ago
superuser2 12y ago
They do not see themselves as targets. Their systems are likely bespoke, or at the very least obscure. And, more importantly, they have bigger problems to worry about, like operating their businesses. As it becomes more clear that yes, someone will go to that trouble and it will have catastrophic consequences, you would hope that these things would get better. More likely, someone will pay a "security consulting" company $100 million to run a Nessus scan and tell them to turn on Automatic Updates on their Windows infrastructure.
- noonespecial 12y ago
  You don't pay them the 100mil to run the scan. You pay them the big bucks to park themselves at the top of the lawsuit list when things go wrong. Its more like insurance at that level than technology.
HCIdivision17 12y ago
A better, more serious answer: always have two networks. Preferably physically separated (though I hear virtual networks are pretty ok with the right router equipment). The machines holler on one, and the administrative support on the other. I've seen what happens when it's even just tried to put both on the same layer, and it's inevitably some form of minor disaster. Not just because of security, mind, but because you really don't want your file transfer to a network drive to even slightly lag a sensor yelling back to the PLC about an interlock's state. If you need to get on the process network, use a VPN, and only open to a machine that can't actually run equipment. A programming terminal may be made available to save costs so an integrator doesn't need to fly in for every support call, but these access points tend to require a VPN through at least two firewalls. (And even then, often you would still insist on them coming in person, for all manner of other reasons.)
- lstamour 12y ago
  See also: https://www.tofinosecurity.com/blog/why-vlan-security-isnt-scada-security-all https://www.tofinosecurity.com/blog/why-vlan-security-isnt-s... (and its comments, which echo different points)
mweatherill 12y ago
It is very unlikely that the process control network is connected to the Internet. However it is almost certainly connected to the corporate Intranet. Think about all of the metric data available on the process control network - that is needed by engineers for analysis, ERP systems for financials, asset management systems for maintenance etc. With an air gap, you can't do any of that in real-time.
- testrun 12y ago
  The solution is to run a historian in the DMZ, only the historian can read data from the DCS, and the corporate systems (ERP, BI etc) read data from the historian. And nothing from outside can update the DCS.

by ArchD 12y ago

ArchD 12y ago

OK, granted they may want to monitor the plant remotely. Then they could have a plant-connected machine dump UDP monitoring packets to an Internet-connected machine, and have the plant-connected machine block all incoming packets from the Internet-connected machine.

gear54rus 12y ago
It seems that there are many ways this could be done right (and does not seem a particularly hard challenge), it's just that people in charge probably were pretty much inept at that task. You know how it is, no one cared about it until it happened. It just wasn't a priority.
- spacecowboy_lon 12y ago
  I get the impression from dealing with german companies that they tend to be very good at traditional "engineering" but when it comes to it/computers they are 10 or 15 years behind. I also think that in germany its considred that the good engineers and asociate profesionalas want go and work for firms like Audi.
  - fidotron 12y ago
    It's not just Germans. Anyone that isn't primarily in software has this phenomenon. Mobile phone makers, for example, are a disaster, and it's only having a whip wielded by a software company with some power over them that prevents it becoming a complete train wreck. In my experience the most dangerous are engineers in other domains that learned just enough programming to get the job done but can't understand the giant holes they've created and not run into.
    - spacecowboy_lon 12y ago
      Testify (Brother or Sister) having worked for a big telco we regarded the mobile side as grade inflated "amatuers" I still recall one of my colegues (working on the core IP network) being amused that one UK mobile provider was still using NT4 in their core network. Ill be nice and not write what we thought about the US cariers
spacecowboy_lon 12y ago
Well use private curcuits or go old school use modems with dialback ie you call the modem it disconects and calls you back on a hard coded number

HCIdivision17 12y ago

I remember a rule a controls engineer once told me: never connect the plant to the Internet. Nothing clever, no humorous quip, no deep insight. Just don't do it. If you do need to get data out to the living world, and you will, then you carefully set up individual firewall rules just for the data system - which very much can not drive the process system. This is where I begin ranting on the topic. Why? Because plants aren't secure. They're meant to run all the time by people who may or may not know how to properly use a mouse. There will be passwords taped to monitors, systems that automatically log in to prevent start up delays, and authentication of the order that it-better-just-work-by-default. Under no circumstances should that damn system ever, ever be exposed to the outside world. Not by Ethernet, wireless, or flash drive. It's a young innocent facing the cruel, brutal internet; it's going to get hurt. New plants are fancy and have highly trained workers with brilliant industrial IP wireless systems with state of the art VM servers and oh god did that guy just plug his phone charger into the damn wrapper HMI and now Windows media player has popped up and no one can acknowledge alarms (this partly why PCs tend to be in large metal boxes, that and dirt/water). No imagine that sort of silliness, but driven by the less savory from outside the intranet. Just don't risk it. You'll never have a problem, and no one is ever going to care enough to hurt the plant... Except but for that one time when suddenly all the convenience and hubris won't bring back the machine that just slagged itself from some malicious command from outside.

superuser2 12y ago
The terrifying part of this is that computers with Windows Media Player installed are running critical infrastructure. Shouldn't that be a stripped-down Linux machine with perfectly understood characteristics and close-to-zero attack surface? Yes, your scenario is bad I can totally see it happening, but the problem is not that an employee plugged in his phone, it's that you are using a desktop OS for office workers for controls that really non-optionally need to always just work. Maybe it's not a Windows Media Player launch, but what happens if Java shows up in the taskbar wanting an update, or your anti-virus software (yikes) wants to bug you about updating its definitions, or a modal dialog from the OS comes up? The fact that those are even things that can happen is pretty scary.
- 12y ago

superuser2 12y ago

They do not see themselves as targets. Their systems are likely bespoke, or at the very least obscure. And, more importantly, they have bigger problems to worry about, like operating their businesses. As it becomes more clear that yes, someone will go to that trouble and it will have catastrophic consequences, you would hope that these things would get better. More likely, someone will pay a "security consulting" company $100 million to run a Nessus scan and tell them to turn on Automatic Updates on their Windows infrastructure.

noonespecial 12y ago
You don't pay them the 100mil to run the scan. You pay them the big bucks to park themselves at the top of the lawsuit list when things go wrong. Its more like insurance at that level than technology.

HCIdivision17 12y ago

A better, more serious answer: always have two networks. Preferably physically separated (though I hear virtual networks are pretty ok with the right router equipment). The machines holler on one, and the administrative support on the other. I've seen what happens when it's even just tried to put both on the same layer, and it's inevitably some form of minor disaster. Not just because of security, mind, but because you really don't want your file transfer to a network drive to even slightly lag a sensor yelling back to the PLC about an interlock's state. If you need to get on the process network, use a VPN, and only open to a machine that can't actually run equipment. A programming terminal may be made available to save costs so an integrator doesn't need to fly in for every support call, but these access points tend to require a VPN through at least two firewalls. (And even then, often you would still insist on them coming in person, for all manner of other reasons.)

lstamour 12y ago
See also: https://www.tofinosecurity.com/blog/why-vlan-security-isnt-scada-security-all https://www.tofinosecurity.com/blog/why-vlan-security-isnt-s... (and its comments, which echo different points)

mweatherill 12y ago

It is very unlikely that the process control network is connected to the Internet. However it is almost certainly connected to the corporate Intranet. Think about all of the metric data available on the process control network - that is needed by engineers for analysis, ERP systems for financials, asset management systems for maintenance etc. With an air gap, you can't do any of that in real-time.

testrun 12y ago
The solution is to run a historian in the DMZ, only the historian can read data from the DCS, and the corporate systems (ERP, BI etc) read data from the historian. And nothing from outside can update the DCS.