7 ms·

Since it sounds like they didn’t catch it until near the end, it’s probably north of $65k/wafer. Probably single digit billions would be an accurate guess for t

Since it sounds like they didn’t catch it until near the end, it’s probably north of $65k/wafer. Probably single digit billions would be an accurate guess for the total loss to TSMC.

rubbingalcohol 8y ago
Worth mentioning that this is TSMC's fuckup, not Nvidia or the other companies using their fab. This is so incredible it makes me wonder if we're looking at industrial sabotage. I can't believe TSMC wouldn't have safeguards in place to prevent contamination under normal circumstances, much less at this scale.
- jannes 8y ago
  Who benefits from this, though? The first one that comes to mind is Intel... Although I am not suggesting that they did it. It's just a hypothetical question.
  - wmf 8y ago
    In theory Samsung or AMD could benefit from shortages of Qualcomm and NVidia products although in practice the magnitude of such substitution would probably be small.
  - zamalek 8y ago
    Anyone that doesn't use the Nanke 14 factory specifically (or uses the factory only for some of their products).
    - JudasGoat 8y ago
      Or perhaps even a vendor that had contracted more capacity(in Nanke 14) than the present market demands, could benefit from a delay in production.
- iamdead 8y ago
  In confidence, I’ve heard stories of simple cost-saving measures at fabs that have resulted in 9+ figures of damages. Honestly, I am impressed at how well these companies keep it out of the news. It’s a shame because the stories are really quite good. You can have safeguards against contamination but these safeguards aren’t 100% reliable. The article reports “substandard” chemicals and that’s an umbrella term that includes contamination and many other problems. Speaking as nothing more than a hobbyist, I can tell you that analog photography suffers from many of the same problems you might see in semiconductor manufacturing, only on a much smaller scale. I used to mix my own photochemicals from raw reagents and it’s a complex subject, to say the least. Exposure to air and minerals in the water have all sorts of effects, and the standard way to test your process is just to run film through it. I’m sure that fabs have better testing equipment than I do, but at the end of the day, it’s not feasible to test everything and I’m not surprised that a bad batch of chemicals made it through, ruining many batches of wafers due to the sheer depth of the manufacturing pipeline. With photochemicals, a small change in the developer formulation can result in what is more or less a completely black and unworkable negative, or possibly a blank negative. I expect semiconductor manufacturing to be similar, since both processes rely so heavily on knowing reaction rates. Kinetics is complicated, to say the least. For photochemistry I rely heavily on using known developer / film concentrations and being borderline religious when it comes to temperature and time.
  - avs733 8y ago
    >In confidence, I’ve heard stories of simple cost-saving measures at fabs that have resulted in 9+ figures of damages. Honestly, I am impressed at how well these companies keep it out of the news. It’s a shame because the stories are really quite good. Dude at vendor switched a label on two boxes. Parts inside looked identical, and were installed. No way to tell until ~week later when chips hit end of line test and didn't work. An average process tool (not even a complicated one) probably has a dozen different gasses and chemicals running into it. In a fab it's hundreds likely thousands. Things happen. They all come from vendors around the world, made in batches that vary and rely on negotiated compliance standards and qualifications. Systems are everything. Wafers travel in boxes 10,000x cleaner than an OR inside a building 1,000x cleaner than an OR...everything matters.
reitzensteinm 8y ago
Surely the cost per wafer is more or less the same no matter where you catch it, since you'll end up with a bubble in your manufacturing pipeline of one unit?
- ehntoo 8y ago
  If it's an issue in an early stage of the production pipeline and the same chemical contamination is in place for more than one lot, you eat up to an entire pipeline full of product.
wmf 8y ago
Wafer cost is supposed to be more like $4K although there may be penalties in this case.
- metildaa 8y ago
  $65k is probably with how much lithography and work has gone into the wafer.
  - wmf 8y ago
    No, I mean a finished wafer is $4K.
  - extrapickles 8y ago
    Its looking like my $65k figure might be the retail value of the ics in the wafer, the silicon at TSMC is about $4k like wmk said. Source: https://www.semiwiki.com/forum/attachments/content/attachments/18722d1481580862-tsmc-cost.pdf https://www.semiwiki.com/forum/attachments/content/attachmen...
01100011 8y ago
I wonder how deep the pipeline is between inputs and failing tests on the output? I'd guess they'd have quite a bit of inventory in flight before they realized there was a problem with the output, but hell, I don't know.
- metildaa 8y ago
  3 months of inventory are "in flight" being manufactured at any given time, that is the minimum amount of affected inventory. Huge feedback loop, most chip designers can only squeeze in 2 revisions a year due to this.

Since it sounds like they didn’t catch it until near the end, it’s probably north of $65k/wafer. Probably single digit billions would be an accurate guess for the total loss to TSMC.

rubbingalcohol 8y ago

Worth mentioning that this is TSMC's fuckup, not Nvidia or the other companies using their fab. This is so incredible it makes me wonder if we're looking at industrial sabotage. I can't believe TSMC wouldn't have safeguards in place to prevent contamination under normal circumstances, much less at this scale.

jannes 8y ago
Who benefits from this, though? The first one that comes to mind is Intel... Although I am not suggesting that they did it. It's just a hypothetical question.
- wmf 8y ago
  In theory Samsung or AMD could benefit from shortages of Qualcomm and NVidia products although in practice the magnitude of such substitution would probably be small.
- zamalek 8y ago
  Anyone that doesn't use the Nanke 14 factory specifically (or uses the factory only for some of their products).
  - JudasGoat 8y ago
    Or perhaps even a vendor that had contracted more capacity(in Nanke 14) than the present market demands, could benefit from a delay in production.
iamdead 8y ago
In confidence, I’ve heard stories of simple cost-saving measures at fabs that have resulted in 9+ figures of damages. Honestly, I am impressed at how well these companies keep it out of the news. It’s a shame because the stories are really quite good. You can have safeguards against contamination but these safeguards aren’t 100% reliable. The article reports “substandard” chemicals and that’s an umbrella term that includes contamination and many other problems. Speaking as nothing more than a hobbyist, I can tell you that analog photography suffers from many of the same problems you might see in semiconductor manufacturing, only on a much smaller scale. I used to mix my own photochemicals from raw reagents and it’s a complex subject, to say the least. Exposure to air and minerals in the water have all sorts of effects, and the standard way to test your process is just to run film through it. I’m sure that fabs have better testing equipment than I do, but at the end of the day, it’s not feasible to test everything and I’m not surprised that a bad batch of chemicals made it through, ruining many batches of wafers due to the sheer depth of the manufacturing pipeline. With photochemicals, a small change in the developer formulation can result in what is more or less a completely black and unworkable negative, or possibly a blank negative. I expect semiconductor manufacturing to be similar, since both processes rely so heavily on knowing reaction rates. Kinetics is complicated, to say the least. For photochemistry I rely heavily on using known developer / film concentrations and being borderline religious when it comes to temperature and time.
- avs733 8y ago
  >In confidence, I’ve heard stories of simple cost-saving measures at fabs that have resulted in 9+ figures of damages. Honestly, I am impressed at how well these companies keep it out of the news. It’s a shame because the stories are really quite good. Dude at vendor switched a label on two boxes. Parts inside looked identical, and were installed. No way to tell until ~week later when chips hit end of line test and didn't work. An average process tool (not even a complicated one) probably has a dozen different gasses and chemicals running into it. In a fab it's hundreds likely thousands. Things happen. They all come from vendors around the world, made in batches that vary and rely on negotiated compliance standards and qualifications. Systems are everything. Wafers travel in boxes 10,000x cleaner than an OR inside a building 1,000x cleaner than an OR...everything matters.

reitzensteinm 8y ago

Surely the cost per wafer is more or less the same no matter where you catch it, since you'll end up with a bubble in your manufacturing pipeline of one unit?

ehntoo 8y ago
If it's an issue in an early stage of the production pipeline and the same chemical contamination is in place for more than one lot, you eat up to an entire pipeline full of product.

wmf 8y ago

Wafer cost is supposed to be more like $4K although there may be penalties in this case.

metildaa 8y ago
$65k is probably with how much lithography and work has gone into the wafer.
- wmf 8y ago
  No, I mean a finished wafer is $4K.
- extrapickles 8y ago
  Its looking like my $65k figure might be the retail value of the ics in the wafer, the silicon at TSMC is about $4k like wmk said. Source: https://www.semiwiki.com/forum/attachments/content/attachments/18722d1481580862-tsmc-cost.pdf https://www.semiwiki.com/forum/attachments/content/attachmen...

01100011 8y ago

I wonder how deep the pipeline is between inputs and failing tests on the output? I'd guess they'd have quite a bit of inventory in flight before they realized there was a problem with the output, but hell, I don't know.

metildaa 8y ago
3 months of inventory are "in flight" being manufactured at any given time, that is the minimum amount of affected inventory. Huge feedback loop, most chip designers can only squeeze in 2 revisions a year due to this.