3 ms·

A couple years ago I worked on a service that had to communicate with a Siemens S7-1500 PLC. Based on my experience with that project, none of what I’ve read re

A couple years ago I worked on a service that had to communicate with a Siemens S7-1500 PLC. Based on my experience with that project, none of what I’ve read recently about unsecured industrial PLCs is surprising. I opened Siemens TIA Portal and PLCSIM for the first time and thought “wow, I didn’t think the Windows 95 GUI library was still supported.” None of the PLC contractors we had hired knew how to enable TLS on the thing (user/pass eg admin/admin was their usual). Anecdote: I once spent hours reading the docs and clicking around trying to get it to accept an SSL certificate signed by a real CA and it wouldn’t go, but it accepted one I self-signed in openssl. In all fairness, the people who are experts in the field of Siemens PLC programming are usually mechanical-ish engineers and security is not in their skill set or on their mind.

jordanb 1mo ago
My mind was blown when I realized that the way tftp works is that as the machine is booting it asks the network if anyone has some software for it to run.
- Joker_vD 1mo ago
  Well, what else can it do, really? It has to boot with pretty much zero knowledge about the external world (maybe except asking the user for the current date and time). Sure, you can hardcode an outdated list of CAs (it's always outdated because the system can be booted 10/20/100 years after it was made) in but that just opens you to unexpected obsolescence, and you usually can't put too much stuff in the bootloader anyway. Not really dissimilar from the human upbringing: leave a baby with "bad" guardians, and it will grow up corrupted. That's a feature, not a bug: if you knew what behaviour exactly you wanted (other than "whatever Simon says"), you would just bake it in in the first place, right?
  - deleted 1mo ago
    [deleted]
  - Dylan16807 1mo ago
    If we pretend we're revising TFTP boot in 1995, let's have it get up to 20 boot options from the server and their md5 hashes, and if it's not set to auto it waits for the user to pick one. It then verifies the hash as it downloads. Also it uses TCP for the download.
    - Joker_vD 1mo ago
      Nah, you won't sell the cow like that. You need to add more reliance on the public CA infrastructure and third-party code signing. Also, "ask user"? Ask the TPM instead — I mean, why would the computer's owner trust the computer's user, right?
sidewndr46 1mo ago
Isn't this the industry expectation in that kind of equipment? If it was signed by a real CA the cert. could expire and render the equipment unable to communicate.
- stephbook 1mo ago
  also you can't pin the user/pw to the machine with a note, because someone might need remote access. better stick with admin/admin
- peterabbitcook 1mo ago
  That’s a tough question. If your PLC is on an airgapped LAN, admin/admin is not great security hygiene but you’ve reduced most of the risk by airgapping. On my project the service I wrote was doing bidirectional communication with the PLC over OPCUA. The server running this pod was connected to the internet, so it was critical to have proper TLS for the OPCUA client/server. Rotating LetsEncrypt certs on the system every 45 days is a lot of toil, but using a self-signed cert that expires in 2040 from some dev laptop doesn’t pass muster in most organizations either. That’s just the OPCUA path.. In these projects You also typically see WinCC HMIs that can talk to the PLC from anywhere on the network without TLS. And also SIMATIC Web Server pages - minimally secured by default, and the amount of info you can grok about the system in Chrome Dev Tools is troubling.
  - dylan604 1mo ago
    > Rotating LetsEncrypt certs on the system every 45 days is a lot of toil What is unique about your system using LetsEncrypt that you can't automate certbot to handle this task as it was designed and intended to be done?
    - gmueckl 1mo ago
      On an airgapped system that is is turned on once and needs to keep running for many, many years? Industrial equipment is a world of its own and internet best practices just don't transfer directly. Some PLCs run extremely expensive machines. Some machines can't afford to have their control systems stutter or fail because that can lead to physical damage and production outages of enormous proportions. A PLC that stops communicating because a certificate just expired is absolutely not acceptable in some plants.
bugbull 1mo ago
[dead]
katzenversteher 1mo ago
Most factories I know do not allow their PLC be accessed from the internet. They are usually on a separate Network. However, the "engineering" station (the computer running e.g. TIA Portal) sometimes is. The PLC engineers I had contact with usually had an electrical engineering background. That's why they like PLCs in the first place with the ladder logic programming languages, grafcet and if they feel fancy a bit of structured text (assembly like) or structured control language (pascal like). They indeed did not know much about software security but a great deal about machine safety. A real security nightmare are older OPC servers (OPC-DA) which is super reliant on DCOM. OPC is quite important to connect the PLCs to SCADA systems or 3rd party devices.
- lowbloodsugar 1mo ago
  >They are usually on a separate Network. Then someone plugs in a cable because boss wants something "over there" and there's already a network that runs "over there". Or optimizes to a smart switch with vlans, and then someone else optimizes to a single vlan. It's not hard to not give a shit, or not understand, network security.
  - lenerdenator 1mo ago
    That someone can be brought into an office and shown a small diagram of the approved network topology. Then they can be shown a small diagram of the current network topology. Next, they can be asked if they're the same. If they're not, they can finally be asked if they're aware that deviating from the approved network topology without consulting infosec is grounds for termination of their employment.
    - edoceo 1mo ago
      Bunch of assumptions about operational excellence in there. Doesn't match my experience but, it does match my desire.
      - lenerdenator 1mo ago
lenerdenator 1mo ago
> In all fairness, the people who are experts in the field of Siemens PLC programming are usually mechanical-ish engineers and security is not in their skill set or on their mind. Stuxnet was over a decade ago. There should be a simple rule that everyone with the ability to understand things like PLCs should be able to grasp: your equipment does not touch the internet or external storage, period. Those who can't grasp this concept should be shown the door with a recommendation that they find a less mentally-taxing line of work.
- crote 1mo ago
  The Stuxnet PLCs weren't touching either, though. The worm was designed to jump across network air gaps via USB flash drives, and spread across isolated networks to other hosts. Finally, it targeted what was likely going to be a service technician laptop, which had to connect to the PLC via a data cable to update and adjust its configuration. An attack like this can only be avoided if you never transfer any data from the outside to the inside. But that means you won't ever be able to install any form of software update or upload new config files without manually typing them in - or even reinstall the OS on any machine...
- Prickle 1mo ago
  Oh I wish that was possible. Fact is if the customer wants to put their worksite management on AWS, you inevitably expose the OPC server and/or PLC to a Intranet. This is then inevitably connected to the internet in some way. The customer may or may not put barriers between them, but that's not up to us to decide. This becomes especially a problem if they have multiple sites across a country or countries. We have a customer that runs a factory on god forsaken WIFI, then dares to complain about a bad connection to said equipment. Thankfully, they have their own internal IT department.
floro 1mo ago
I work with logistics automation software. Most PLCs I've worked with are legacy and don't even support TLS. Doesn't surprise me at all. I honestly am baffled how old some of these sites are and never got a security upgrade. Now with the EU cyber resiliency act things are moving but I think it's infeasible for many sites to meet modern security standards, because of both hardware and software.

jordanb 1mo ago

My mind was blown when I realized that the way tftp works is that as the machine is booting it asks the network if anyone has some software for it to run.

Joker_vD 1mo ago
Well, what else can it do, really? It has to boot with pretty much zero knowledge about the external world (maybe except asking the user for the current date and time). Sure, you can hardcode an outdated list of CAs (it's always outdated because the system can be booted 10/20/100 years after it was made) in but that just opens you to unexpected obsolescence, and you usually can't put too much stuff in the bootloader anyway. Not really dissimilar from the human upbringing: leave a baby with "bad" guardians, and it will grow up corrupted. That's a feature, not a bug: if you knew what behaviour exactly you wanted (other than "whatever Simon says"), you would just bake it in in the first place, right?
- deleted 1mo ago
  [deleted]
- Dylan16807 1mo ago
  If we pretend we're revising TFTP boot in 1995, let's have it get up to 20 boot options from the server and their md5 hashes, and if it's not set to auto it waits for the user to pick one. It then verifies the hash as it downloads. Also it uses TCP for the download.
  - Joker_vD 1mo ago
    Nah, you won't sell the cow like that. You need to add more reliance on the public CA infrastructure and third-party code signing. Also, "ask user"? Ask the TPM instead — I mean, why would the computer's owner trust the computer's user, right?

sidewndr46 1mo ago

Isn't this the industry expectation in that kind of equipment? If it was signed by a real CA the cert. could expire and render the equipment unable to communicate.

stephbook 1mo ago
also you can't pin the user/pw to the machine with a note, because someone might need remote access. better stick with admin/admin
peterabbitcook 1mo ago
That’s a tough question. If your PLC is on an airgapped LAN, admin/admin is not great security hygiene but you’ve reduced most of the risk by airgapping. On my project the service I wrote was doing bidirectional communication with the PLC over OPCUA. The server running this pod was connected to the internet, so it was critical to have proper TLS for the OPCUA client/server. Rotating LetsEncrypt certs on the system every 45 days is a lot of toil, but using a self-signed cert that expires in 2040 from some dev laptop doesn’t pass muster in most organizations either. That’s just the OPCUA path.. In these projects You also typically see WinCC HMIs that can talk to the PLC from anywhere on the network without TLS. And also SIMATIC Web Server pages - minimally secured by default, and the amount of info you can grok about the system in Chrome Dev Tools is troubling.
- dylan604 1mo ago
  > Rotating LetsEncrypt certs on the system every 45 days is a lot of toil What is unique about your system using LetsEncrypt that you can't automate certbot to handle this task as it was designed and intended to be done?
  - gmueckl 1mo ago
    On an airgapped system that is is turned on once and needs to keep running for many, many years? Industrial equipment is a world of its own and internet best practices just don't transfer directly. Some PLCs run extremely expensive machines. Some machines can't afford to have their control systems stutter or fail because that can lead to physical damage and production outages of enormous proportions. A PLC that stops communicating because a certificate just expired is absolutely not acceptable in some plants.

bugbull 1mo ago

[dead]

katzenversteher 1mo ago

Most factories I know do not allow their PLC be accessed from the internet. They are usually on a separate Network. However, the "engineering" station (the computer running e.g. TIA Portal) sometimes is. The PLC engineers I had contact with usually had an electrical engineering background. That's why they like PLCs in the first place with the ladder logic programming languages, grafcet and if they feel fancy a bit of structured text (assembly like) or structured control language (pascal like). They indeed did not know much about software security but a great deal about machine safety. A real security nightmare are older OPC servers (OPC-DA) which is super reliant on DCOM. OPC is quite important to connect the PLCs to SCADA systems or 3rd party devices.

lowbloodsugar 1mo ago
>They are usually on a separate Network. Then someone plugs in a cable because boss wants something "over there" and there's already a network that runs "over there". Or optimizes to a smart switch with vlans, and then someone else optimizes to a single vlan. It's not hard to not give a shit, or not understand, network security.
- lenerdenator 1mo ago
  That someone can be brought into an office and shown a small diagram of the approved network topology. Then they can be shown a small diagram of the current network topology. Next, they can be asked if they're the same. If they're not, they can finally be asked if they're aware that deviating from the approved network topology without consulting infosec is grounds for termination of their employment.
  - edoceo 1mo ago
    Bunch of assumptions about operational excellence in there. Doesn't match my experience but, it does match my desire.
    - lenerdenator 1mo ago

lenerdenator 1mo ago

> In all fairness, the people who are experts in the field of Siemens PLC programming are usually mechanical-ish engineers and security is not in their skill set or on their mind. Stuxnet was over a decade ago. There should be a simple rule that everyone with the ability to understand things like PLCs should be able to grasp: your equipment does not touch the internet or external storage, period. Those who can't grasp this concept should be shown the door with a recommendation that they find a less mentally-taxing line of work.

crote 1mo ago
The Stuxnet PLCs weren't touching either, though. The worm was designed to jump across network air gaps via USB flash drives, and spread across isolated networks to other hosts. Finally, it targeted what was likely going to be a service technician laptop, which had to connect to the PLC via a data cable to update and adjust its configuration. An attack like this can only be avoided if you never transfer any data from the outside to the inside. But that means you won't ever be able to install any form of software update or upload new config files without manually typing them in - or even reinstall the OS on any machine...
Prickle 1mo ago
Oh I wish that was possible. Fact is if the customer wants to put their worksite management on AWS, you inevitably expose the OPC server and/or PLC to a Intranet. This is then inevitably connected to the internet in some way. The customer may or may not put barriers between them, but that's not up to us to decide. This becomes especially a problem if they have multiple sites across a country or countries. We have a customer that runs a factory on god forsaken WIFI, then dares to complain about a bad connection to said equipment. Thankfully, they have their own internal IT department.

floro 1mo ago

I work with logistics automation software. Most PLCs I've worked with are legacy and don't even support TLS. Doesn't surprise me at all. I honestly am baffled how old some of these sites are and never got a security upgrade. Now with the EU cyber resiliency act things are moving but I think it's infeasible for many sites to meet modern security standards, because of both hardware and software.