6 ms·

TSMC's Photoresist Material Incident: $550M Loss

116 points by Itsdijital 8y ago

xoa 8y ago
If you haven't watched it, there was a fantastic 2009 talk (given during HOPE09) on what goes into fabrication called "Indistinguishable From Magic: Manufacturing Modern Computer Chips" [1]. Keeping in mind things have advanced even farther over the subsequent decade, even at that point it was mind blowing what goes into the hardware at the core of what we all do. The level of purity and consistency needed for a lot of basic inputs is measured in atoms, really digging into the details it's amazing some of this works at all let alone with high yields. Helps give some context of how an incident like this can happen and how easily the cost could rapidly rise to eye watering levels. I think Andor has posted on HN since too with some other high level talk, though not further updates per se since unsurprisingly details around fabrications are very heavily guarded trade secrets. I still rewatch it every year or two, it remains a real source of wonder to me what we've pulled off there. ---- 1: https://www.youtube.com/watch?v=NGFhc8R_uO4 https://www.youtube.com/watch?v=NGFhc8R_uO4
- downer52 8y ago
  The best part is the brief addendum about some of the bunny suit inhabitants of the chip fabs (~20 minute mark), and how they are weird, but that the gender distribution of their children is, in fact, "normal" or at last not specifically a product of their occupation.
- Joona 8y ago
  Definitely one of my favorite talks. I believe this is a higher quality version of the same talk a few years later: https://www.youtube.com/watch?v=KL-I3-C-KBk https://www.youtube.com/watch?v=KL-I3-C-KBk
- drbawb 8y ago
  Thank you so much for posting that video. I watched it a long time ago and have been wanting to revisit it, but was never able to find it again.
- tecleandor 8y ago
  Hi, Andor here. I'm the uploader of the video. I'm not the Todd Fernandez, the speaker for that HOPE09 talk. I happened to give a talk at the same congress (Digital Security in Healthcare Institutions, pretty outdated as of today), watched Todd talk, and loved it. My upload is published every once in a while in HN or Reddit, but I'm not the author, just a, ahem, freeloader. I uploaded the talk with intention of subtitle it for my deaf teacher in particular and Spanish speaking people in general, but I lost control of the account, I think it was related of some sort of Google+ problem. So here are my explanations
sevensor 8y ago
I'm a little surprised this ended up being resist. Photolithography usually involves quite a lot of looking at SEM imagery of test patterns. (These usually go in the scribe lines.) The problem must have been pretty subtle for it not to get flagged. Maybe (since they mention unexpected polymerization) it left an unexpected residue after cleans? Post-photo clean usually starts with ashing in an oxygen plasma, so it would have to be some pretty serious gunk to survive that and the subsequent wet clean.
masto 8y ago
Imagine a state actor with the resources and motivation to engage in serious sabotage (a-la stuxnet). These processes are so sensitive and complex, I bet they could come up with a really devious way to introduce a contaminant that isn't detected until it's too late -- maybe even when the parts start failing after delivery. Pretty wild to think about having that kind of target on your back and what you'd have to do to even try to defend against it.
- xenadu02 8y ago
  Supply chains are so interconnected it would be impossible to limit the damage to your intended targets. Probably similar to GPS "selective availability": turns out it would hurt the US Military more than anything else these days so the military has essentially abandoned the idea.
  - pas 8y ago
    Is there a tl;dr on why and how it would hurt the US Mil the most?
    - skissane 8y ago
      Shortages of the special military GPS receivers (which support decrypting the encrypted signal) resulted in the US military having to use ordinary civilian GPS receivers instead in many cases. So, limiting the accuracy of the civilian signal was having a negative impact on the US military's own operations.
- deleted 8y ago
  [deleted]
- theandrewbailey 8y ago
  It's probably easier than that: TSMC got hit with WannaCry a while ago. https://www.theregister.co.uk/2018/08/06/tsmc_malware/ https://www.theregister.co.uk/2018/08/06/tsmc_malware/
rbanffy 8y ago
What is the size of the market for framed almost-complete wafers? Say you can sell a framed wafer for $50, of which $10 is materials and $10 is labor, you still make $30 x ~20,000 or ~$600,000 for the whole batch, minus taxes.
ljoshua 8y ago
Of the $550 million, I wonder how much of it is in materials cost and how much of it is in penalties paid to renegotiate delivery dates for the affected customers.
- lefty2 8y ago
  it's $550 million lost revenue in 1Q, which is recognised in 2Q instead. The loss for the whole year is $66M
baybal2 8y ago
Dow Chem must be owing TSMC some good explanation...

116 points by Itsdijital 8y ago

xoa 8y ago

If you haven't watched it, there was a fantastic 2009 talk (given during HOPE09) on what goes into fabrication called "Indistinguishable From Magic: Manufacturing Modern Computer Chips" [1]. Keeping in mind things have advanced even farther over the subsequent decade, even at that point it was mind blowing what goes into the hardware at the core of what we all do. The level of purity and consistency needed for a lot of basic inputs is measured in atoms, really digging into the details it's amazing some of this works at all let alone with high yields. Helps give some context of how an incident like this can happen and how easily the cost could rapidly rise to eye watering levels. I think Andor has posted on HN since too with some other high level talk, though not further updates per se since unsurprisingly details around fabrications are very heavily guarded trade secrets. I still rewatch it every year or two, it remains a real source of wonder to me what we've pulled off there. ---- 1: https://www.youtube.com/watch?v=NGFhc8R_uO4 https://www.youtube.com/watch?v=NGFhc8R_uO4

downer52 8y ago
The best part is the brief addendum about some of the bunny suit inhabitants of the chip fabs (~20 minute mark), and how they are weird, but that the gender distribution of their children is, in fact, "normal" or at last not specifically a product of their occupation.
Joona 8y ago
Definitely one of my favorite talks. I believe this is a higher quality version of the same talk a few years later: https://www.youtube.com/watch?v=KL-I3-C-KBk https://www.youtube.com/watch?v=KL-I3-C-KBk
drbawb 8y ago
Thank you so much for posting that video. I watched it a long time ago and have been wanting to revisit it, but was never able to find it again.
tecleandor 8y ago
Hi, Andor here. I'm the uploader of the video. I'm not the Todd Fernandez, the speaker for that HOPE09 talk. I happened to give a talk at the same congress (Digital Security in Healthcare Institutions, pretty outdated as of today), watched Todd talk, and loved it. My upload is published every once in a while in HN or Reddit, but I'm not the author, just a, ahem, freeloader. I uploaded the talk with intention of subtitle it for my deaf teacher in particular and Spanish speaking people in general, but I lost control of the account, I think it was related of some sort of Google+ problem. So here are my explanations

sevensor 8y ago

I'm a little surprised this ended up being resist. Photolithography usually involves quite a lot of looking at SEM imagery of test patterns. (These usually go in the scribe lines.) The problem must have been pretty subtle for it not to get flagged. Maybe (since they mention unexpected polymerization) it left an unexpected residue after cleans? Post-photo clean usually starts with ashing in an oxygen plasma, so it would have to be some pretty serious gunk to survive that and the subsequent wet clean.

masto 8y ago

Imagine a state actor with the resources and motivation to engage in serious sabotage (a-la stuxnet). These processes are so sensitive and complex, I bet they could come up with a really devious way to introduce a contaminant that isn't detected until it's too late -- maybe even when the parts start failing after delivery. Pretty wild to think about having that kind of target on your back and what you'd have to do to even try to defend against it.

xenadu02 8y ago
Supply chains are so interconnected it would be impossible to limit the damage to your intended targets. Probably similar to GPS "selective availability": turns out it would hurt the US Military more than anything else these days so the military has essentially abandoned the idea.
- pas 8y ago
  Is there a tl;dr on why and how it would hurt the US Mil the most?
  - skissane 8y ago
    Shortages of the special military GPS receivers (which support decrypting the encrypted signal) resulted in the US military having to use ordinary civilian GPS receivers instead in many cases. So, limiting the accuracy of the civilian signal was having a negative impact on the US military's own operations.
deleted 8y ago
[deleted]
theandrewbailey 8y ago
It's probably easier than that: TSMC got hit with WannaCry a while ago. https://www.theregister.co.uk/2018/08/06/tsmc_malware/ https://www.theregister.co.uk/2018/08/06/tsmc_malware/

rbanffy 8y ago

What is the size of the market for framed almost-complete wafers? Say you can sell a framed wafer for $50, of which $10 is materials and $10 is labor, you still make $30 x ~20,000 or ~$600,000 for the whole batch, minus taxes.

ljoshua 8y ago

Of the $550 million, I wonder how much of it is in materials cost and how much of it is in penalties paid to renegotiate delivery dates for the affected customers.

lefty2 8y ago
it's $550 million lost revenue in 1Q, which is recognised in 2Q instead. The loss for the whole year is $66M

baybal2 8y ago

Dow Chem must be owing TSMC some good explanation...