5 ms·

It's surprising to many, but these tools run primarily on Windows, and, without anti-virus protection because it can interfere with critical timing actions that

It's surprising to many, but these tools run primarily on Windows, and, without anti-virus protection because it can interfere with critical timing actions that the tools need to make. The IT systems are locked down like a fortress, but tool technicians can still bring viruses in and transmit when they connect their laptops for diagnostics and maintenance. I can easily see a virus running amok.

rectang 8y ago
We've seen this before many times. It's an inherent weakness of proprietary software solutions: you're at the mercy of the vendor as far as porting it to a new platform. The weakness becomes more urgent as the platform grows old/unsupported/insecure/etc. It may have been Windows this time, but it could have been DEC or Solaris or what have you. Open Source Software may or may not be easy or economically realistic to port, but at least the users have a say.
- analognoise 8y ago
  Software doesn't exist in a vacuum, it needs support and maintenance, which somebody has to pay for. Can we stop repeating this as a strength of OSS? It definitely is not. "Users" have a problem they want solved, they don't really want that problem and to learn how to patch software.
  - mort96 8y ago
    If the "user" is the world's largest independent foundry, I'm sure they could find someone to patch the software they're using.
  - rectang 8y ago
    With OSS, it is at least possible for a new vendor to step in with competing support and maintenance if the old vendor is failing. It may also be possible for customers to participate in supporting and maintaining the software to varying degrees, for example by hiring the core maintainers; whether that is realistic depends on the size and complexity of the software, the tech culture of the customer, etc.
- robbyt 8y ago
  This isn't caused by lack of open source software, this is caused by lack of management, partitioning, and containment. I assume the issue was caused by a 100% flat network, with office worker's laptops being on the same subnet as million dollar fab machine computers.
  - rectang 8y ago
    Certainly those are also factors, but the central security principle of defense in depth argues for minimizing all possible factors. Having nodes that are highly vulnerable to viruses is a weakness and a contributing factor towards a breach.
  - jdietrich 8y ago
    Even an airgapped system can be breached, as we saw with Stuxnet. There's no perfect solution for protecting vulnerable systems.
    - kirubakaran 8y ago
      There is benefit in making a negative outcome less probable, like there's benefit in wearing a bullet-proof vest though you can still be shot in the face.
    - delfinom 8y ago
      Remove all data ports? Upload instructions via punch cards
      - MertsA 8y ago
        "Hey guys I found these punch cards in the parking lot"
- delfinom 8y ago
  You are kidding right? The machines at TSMC would be at 100% utilization on million dollar jobs, management would never accept downtime for maintenance for software updates that could introduce bugs. Its one thing for consumer devices to receive updates, its a completely different ballgame with industrial. Downtime due to anything can be millions of dollars per hour in losses. So it comes down to what could cause more monetary loss, downtime due to hacks or downtime due to bugs introduced. "If it works why break it". Its the same reason why there are production lines where physically, hardware could be replaced and improved to reduce wear, reliability, power usage, etc (even things like motors) but they arent, for decades. Only when things start costing money outside the original design do engineers perform upgrades.
  - MarkSweep 8y ago
    Exactly. You can say "oh, just take the machine out of production for a bit do software patching". However that reduces the uptime for your tool which will require buying more of them to get the same throughput. More tools is not only capex, but also space on the factory floor. This is extra true for my company. We sell sorters, which don't actually do anything useful in terms of processing the wafers. The tools just shuffle the wafers between FOUPs. While these tools help customers optimize processes by changing batch sizes and such, they really want to have as few non-process tools as possible. So uptime is really important.
ddeck 8y ago
Seem likely. The monitor shown at 1:09, 1:58, and 7:25[1] on the "TSMC Fab Tour" video on this page[2] looks suspiciously like Windows 95, but could be Windows Embedded I guess. [1] https://imgur.com/a/J4EhzPC https://imgur.com/a/J4EhzPC [2] http://www.tsmc.com/english/newsEvents/dc_video.htm# http://www.tsmc.com/english/newsEvents/dc_video.htm#
- CompuHacker 8y ago
  Looks like XP Embedded.
- sandworm101 8y ago
  On a factory floor? It could be win 3.1. Because something is high tech doesnt meant the machine building it hasnt been repurposed a dozen times. Dos is still out there.
  - _trampeltier 8y ago
    In our factory, is at least still one OS/2 PC running. I'm not proud for this.
    - avs733 8y ago
      last time I was in a 300mm fab I definitely saw OS/2, command lines being used by technicians, and a LOT of windows embedded 3.1. The main defense is a computer at the entrance where technicians are supposed to scan their USB drives. It's a devils choice really...do you put the tools on an externally accessible network? If not how do engineers get data off of tools? how does the factory talk to itself and schedule things? I don't envy FAB IT. Utterly thankless.
    - dylan604 8y ago
      It's not necessarily something to not be proud of, though. NASA's space shuttles ran on something like a 386 CPU. That's what was available when the system was designed. Probably one of the best "if it ain't broke, don't fix it" examples. Swapping out to newer components would have meant replacing so much of the system, that it wasn't worth it. Especially if it was doing everything that needed to be done. Same thing with your OS/2 system. As long as it's working, there's really no need to change it. The fun begins when it no longer works and needs to be replaced. How much of the factory will then need to be replaced/updated just because the controlling computer can no longer operate the older equipment? That's the type of scenarios that give managers nightmares.
JumpCrisscross 8y ago
> these tools run primarily on Windows, and, without anti-virus protection Back when I made rocket things, we used an RTOS. Are real-time operating systems (like QNX) not widely used?
- jacquesm 8y ago
  They should be (and are in some industries, apparently not in this one). But QnX real-time capabilities are just one nice part of the mix, the user land drivers and impossible to crash kernel are equal partners in giving it the stability that it has. Process isolation for drivers should be mandatory.

rectang 8y ago

We've seen this before many times. It's an inherent weakness of proprietary software solutions: you're at the mercy of the vendor as far as porting it to a new platform. The weakness becomes more urgent as the platform grows old/unsupported/insecure/etc. It may have been Windows this time, but it could have been DEC or Solaris or what have you. Open Source Software may or may not be easy or economically realistic to port, but at least the users have a say.

analognoise 8y ago
Software doesn't exist in a vacuum, it needs support and maintenance, which somebody has to pay for. Can we stop repeating this as a strength of OSS? It definitely is not. "Users" have a problem they want solved, they don't really want that problem and to learn how to patch software.
- mort96 8y ago
  If the "user" is the world's largest independent foundry, I'm sure they could find someone to patch the software they're using.
- rectang 8y ago
  With OSS, it is at least possible for a new vendor to step in with competing support and maintenance if the old vendor is failing. It may also be possible for customers to participate in supporting and maintaining the software to varying degrees, for example by hiring the core maintainers; whether that is realistic depends on the size and complexity of the software, the tech culture of the customer, etc.
robbyt 8y ago
This isn't caused by lack of open source software, this is caused by lack of management, partitioning, and containment. I assume the issue was caused by a 100% flat network, with office worker's laptops being on the same subnet as million dollar fab machine computers.
- rectang 8y ago
  Certainly those are also factors, but the central security principle of defense in depth argues for minimizing all possible factors. Having nodes that are highly vulnerable to viruses is a weakness and a contributing factor towards a breach.
- jdietrich 8y ago
  Even an airgapped system can be breached, as we saw with Stuxnet. There's no perfect solution for protecting vulnerable systems.
  - kirubakaran 8y ago
    There is benefit in making a negative outcome less probable, like there's benefit in wearing a bullet-proof vest though you can still be shot in the face.
  - delfinom 8y ago
    Remove all data ports? Upload instructions via punch cards
    - MertsA 8y ago
      "Hey guys I found these punch cards in the parking lot"
delfinom 8y ago
You are kidding right? The machines at TSMC would be at 100% utilization on million dollar jobs, management would never accept downtime for maintenance for software updates that could introduce bugs. Its one thing for consumer devices to receive updates, its a completely different ballgame with industrial. Downtime due to anything can be millions of dollars per hour in losses. So it comes down to what could cause more monetary loss, downtime due to hacks or downtime due to bugs introduced. "If it works why break it". Its the same reason why there are production lines where physically, hardware could be replaced and improved to reduce wear, reliability, power usage, etc (even things like motors) but they arent, for decades. Only when things start costing money outside the original design do engineers perform upgrades.
- MarkSweep 8y ago
  Exactly. You can say "oh, just take the machine out of production for a bit do software patching". However that reduces the uptime for your tool which will require buying more of them to get the same throughput. More tools is not only capex, but also space on the factory floor. This is extra true for my company. We sell sorters, which don't actually do anything useful in terms of processing the wafers. The tools just shuffle the wafers between FOUPs. While these tools help customers optimize processes by changing batch sizes and such, they really want to have as few non-process tools as possible. So uptime is really important.

ddeck 8y ago

Seem likely. The monitor shown at 1:09, 1:58, and 7:25[1] on the "TSMC Fab Tour" video on this page[2] looks suspiciously like Windows 95, but could be Windows Embedded I guess. [1] https://imgur.com/a/J4EhzPC https://imgur.com/a/J4EhzPC [2] http://www.tsmc.com/english/newsEvents/dc_video.htm# http://www.tsmc.com/english/newsEvents/dc_video.htm#

CompuHacker 8y ago
Looks like XP Embedded.
sandworm101 8y ago
On a factory floor? It could be win 3.1. Because something is high tech doesnt meant the machine building it hasnt been repurposed a dozen times. Dos is still out there.
- _trampeltier 8y ago
  In our factory, is at least still one OS/2 PC running. I'm not proud for this.
  - avs733 8y ago
    last time I was in a 300mm fab I definitely saw OS/2, command lines being used by technicians, and a LOT of windows embedded 3.1. The main defense is a computer at the entrance where technicians are supposed to scan their USB drives. It's a devils choice really...do you put the tools on an externally accessible network? If not how do engineers get data off of tools? how does the factory talk to itself and schedule things? I don't envy FAB IT. Utterly thankless.
  - dylan604 8y ago
    It's not necessarily something to not be proud of, though. NASA's space shuttles ran on something like a 386 CPU. That's what was available when the system was designed. Probably one of the best "if it ain't broke, don't fix it" examples. Swapping out to newer components would have meant replacing so much of the system, that it wasn't worth it. Especially if it was doing everything that needed to be done. Same thing with your OS/2 system. As long as it's working, there's really no need to change it. The fun begins when it no longer works and needs to be replaced. How much of the factory will then need to be replaced/updated just because the controlling computer can no longer operate the older equipment? That's the type of scenarios that give managers nightmares.

JumpCrisscross 8y ago

> these tools run primarily on Windows, and, without anti-virus protection Back when I made rocket things, we used an RTOS. Are real-time operating systems (like QNX) not widely used?

jacquesm 8y ago
They should be (and are in some industries, apparently not in this one). But QnX real-time capabilities are just one nice part of the mix, the user land drivers and impossible to crash kernel are equal partners in giving it the stability that it has. Process isolation for drivers should be mandatory.