9 ms·
TSMC Nanke 14 Factory Production Interruption Could Affect NVIDIA and Others
- olliej 8y agoOof, I hope they can absorb the cost :-/
- extrapickles 8y agoSince it sounds like they didn’t catch it until near the end, it’s probably north of $65k/wafer. Probably single digit billions would be an accurate guess for the total loss to TSMC.
- rubbingalcohol 8y agoWorth mentioning that this is TSMC's fuckup, not Nvidia or the other companies using their fab. This is so incredible it makes me wonder if we're looking at industrial sabotage. I can't believe TSMC wouldn't have safeguards in place to prevent contamination under normal circumstances, much less at this scale.
- jannes 8y agoWho benefits from this, though? The first one that comes to mind is Intel... Although I am not suggesting that they did it. It's just a hypothetical question.
- wmf 8y agoIn theory Samsung or AMD could benefit from shortages of Qualcomm and NVidia products although in practice the magnitude of such substitution would probably be small.
- zamalek 8y agoAnyone that doesn't use the Nanke 14 factory specifically (or uses the factory only for some of their products).
- JudasGoat 8y agoOr perhaps even a vendor that had contracted more capacity(in Nanke 14) than the present market demands, could benefit from a delay in production.
- iamdead 8y agoIn confidence, I’ve heard stories of simple cost-saving measures at fabs that have resulted in 9+ figures of damages. Honestly, I am impressed at how well these companies keep it out of the news. It’s a shame because the stories are really quite good. You can have safeguards against contamination but these safeguards aren’t 100% reliable. The article reports “substandard” chemicals and that’s an umbrella term that includes contamination and many other problems. Speaking as nothing more than a hobbyist, I can tell you that analog photography suffers from many of the same problems you might see in semiconductor manufacturing, only on a much smaller scale. I used to mix my own photochemicals from raw reagents and it’s a complex subject, to say the least. Exposure to air and minerals in the water have all sorts of effects, and the standard way to test your process is just to run film through it. I’m sure that fabs have better testing equipment than I do, but at the end of the day, it’s not feasible to test everything and I’m not surprised that a bad batch of chemicals made it through, ruining many batches of wafers due to the sheer depth of the manufacturing pipeline. With photochemicals, a small change in the developer formulation can result in what is more or less a completely black and unworkable negative, or possibly a blank negative. I expect semiconductor manufacturing to be similar, since both processes rely so heavily on knowing reaction rates. Kinetics is complicated, to say the least. For photochemistry I rely heavily on using known developer / film concentrations and being borderline religious when it comes to temperature and time.
- avs733 8y ago>In confidence, I’ve heard stories of simple cost-saving measures at fabs that have resulted in 9+ figures of damages. Honestly, I am impressed at how well these companies keep it out of the news. It’s a shame because the stories are really quite good. Dude at vendor switched a label on two boxes. Parts inside looked identical, and were installed. No way to tell until ~week later when chips hit end of line test and didn't work. An average process tool (not even a complicated one) probably has a dozen different gasses and chemicals running into it. In a fab it's hundreds likely thousands. Things happen. They all come from vendors around the world, made in batches that vary and rely on negotiated compliance standards and qualifications. Systems are everything. Wafers travel in boxes 10,000x cleaner than an OR inside a building 1,000x cleaner than an OR...everything matters.
- reitzensteinm 8y agoSurely the cost per wafer is more or less the same no matter where you catch it, since you'll end up with a bubble in your manufacturing pipeline of one unit?
- ehntoo 8y agoIf it's an issue in an early stage of the production pipeline and the same chemical contamination is in place for more than one lot, you eat up to an entire pipeline full of product.
- wmf 8y agoWafer cost is supposed to be more like $4K although there may be penalties in this case.
- metildaa 8y ago$65k is probably with how much lithography and work has gone into the wafer.
- wmf 8y agoNo, I mean a finished wafer is $4K.
- extrapickles 8y agoIts looking like my $65k figure might be the retail value of the ics in the wafer, the silicon at TSMC is about $4k like wmk said. Source: https://www.semiwiki.com/forum/attachments/content/attachments/18722d1481580862-tsmc-cost.pdf https://www.semiwiki.com/forum/attachments/content/attachmen...
- 01100011 8y agoI wonder how deep the pipeline is between inputs and failing tests on the output? I'd guess they'd have quite a bit of inventory in flight before they realized there was a problem with the output, but hell, I don't know.
- metildaa 8y ago3 months of inventory are "in flight" being manufactured at any given time, that is the minimum amount of affected inventory. Huge feedback loop, most chip designers can only squeeze in 2 revisions a year due to this.
- slenk 8y agoWhat are "wafers"?
- jey 8y agoIt's the underlying material chips are made on: https://en.wikipedia.org/wiki/Wafer_(electronics) https://en.wikipedia.org/wiki/Wafer_(electronics)
- davidkuhta 8y ago> A wafer, also called a slice or substrate,[1] is a thin slice of semiconductor material, such as a crystalline silicon, used in electronics for the fabrication of integrated circuits and in photovoltaics for conventional, wafer-based solar cells.
- deleted 8y ago[deleted]
- obl 8y agoIf you have time : https://www.youtube.com/watch?v=NGFhc8R_uO4 https://www.youtube.com/watch?v=NGFhc8R_uO4 It's great.
- avs733 8y agoAlways flattered when this gets posted. Always wish I could go back and re-record this talking about 50% as fast.
- taxicabjesus 8y ago> Always wish I could go back and re-record this talking about 50% as fast. I'm listening to this at 1.5x, which is my usual listening speed for non-musical videos. Your speed is fine. Toastmasters [0] has helped me reduced my use of filler words: ahh, uhm, and, okay, so, you know, like, well, [pregnant pauses], [double clutches]. It's hard for me to listen to politicians anymore, because they're always gumming up their speech with linguistic crutches. I didn't used to notice. [0] https://www.toastmasters.org/ https://www.toastmasters.org/ "When we find ourselves rattled while speaking — whether we’re nervous, distracted, or at a loss for what comes next — it’s easy to lean on filler words. These may give us a moment to collect our thoughts before we press on, and in some cases, they may be useful indicators that the audience should pay special attention to what comes next. But when we start to overuse them, they become crutches — academics call them disfluencies — that diminish our credibility and distract from our message." [1] [1] https://hbr.org/2018/08/how-to-stop-saying-um-ah-and-you-know https://hbr.org/2018/08/how-to-stop-saying-um-ah-and-you-kno...
- gjsman-1000 8y agoOh, that is absolutely disastrous if true. With their tight margins... oof. What can I say? That is ridiculously bad.
- nihil75 8y agoDon't worry, they have tons of old stock to sell..
- ChuckMcM 8y agoTheir margins aren't as tight as you might expect. Their last quarter report[1] puts their gross margin at nearly 48% and their operating margin at 37%. That is pretty healthy and suggests they can easily withstand this sort of event. [1] https://www.tsmc.com/english/investorRelations/quarterly_results.htm https://www.tsmc.com/english/investorRelations/quarterly_res...
- IMTDb 8y agoThis does not bode well as well for Nvidia who already had a hard Q4 2018. They expecs to report $2.2 billion in fourth quarter revenue versus the previous guidance of $2.7 billion.
- lumberingjack 8y agoTHey can't move the new products according to this morning talks, so they might benefit from a shortage and they have a history of shady tactics like GPP to muzzle bad reviews.
- rubbingalcohol 8y agoBad reviews aren't their problem now, it's that people won't pay an extra $400 for incrementally better performance and a missing-in-action feature set.
- chx 8y agoAnd much higher wattage, across the entire lineup.
- duaoebg 8y agoI would pay it, and I did pay it. I need a bunch for on-prem Deep Learning. My test rig of 4 RTX 2080tis artifacted after about a week of use. I'd like to buy them by the handful but I've not been able to source a reliable supply. Given this new it looks like I may have to wait a lot longer.
- ericd 8y agoI'm in the middle of a similar build, and I'm curious how you got 4 of those to work well right next to each other. Are they FE cards, or some other variant?
- duaoebg 8y agoI planned on water cooling but I never made it that far. They were FE cards. I do miss blower style but I understand they're hitting limits with it.
- echofavor 8y agoCost absurbing is good.
- maxander 8y agoEvery once in awhile, one hears about some accident like this one at a single, physical factory, followed by a laundry list of tech companies that are adversely effected. It powerfully demonstrates how awfully centralized microchip production is; individual plants can account for substantial fractions (or even majorities) of entire product categories, which means that individual incidents can have outsized effects. I don't know if that sort of risk has been fully internalized by the tech sector. For that matter, how specialized is the market for whichever high-purity chemicals were behind this incident? What portion of the semiconductor industry was purchasing from the same supplier? We may not have heard the last of this.
- rburhum 8y agoYou mean like AWS being down?
- YjSe2GMQ 8y agoOr a bad BGP route being broadcasted, whether intentionally or not: https://bgpmon.net/bgp-leak-causing-internet-outages-in-japan-and-beyond/ https://bgpmon.net/bgp-leak-causing-internet-outages-in-japa...
- maxander 8y agoNot even- I don’t know the relevant numbers, but I’d bet there are many more AWS regions than factories making the same kind of chips as the one in the article.
- jandrese 8y agoThis is right on the back of nVidia's stock taking a beating after admitting that the global market for GPUs has become soft from the collapse of the crypto bubble.
- nomel 8y ago> I don't know if that sort of risk has been fully internalized by the tech sector. It’s known and understood completely within any silicon company. If you can afford it, and it’s technically possible (no special deals or tech), you’ll require at least some functionality at a second fab. What you cannot do is immediately get a second fab up and running. There’s a surprising amount of process tuning, sometimes accommodating new device models, and design rule differences (they will not guarantee or sometimes even start if they’re not meet). It’s in no way turn-key if you’re anywhere near the limits of what the fab can do.
- libertine 8y agoThis could be good for AMD.
- SteveNuts 8y agoI thought AMD was using TSMC now?
- nonbel 8y agoThis is about TSMC's "16/12nm process". AMD is using TSMC's 7nm process and Global Foundries' 12/14nm processes: https://www.extremetech.com/computing/276169-amd-moves-all-7nm-cpu-gpu-production-to-tsmc https://www.extremetech.com/computing/276169-amd-moves-all-7...
- deleted 8y ago[deleted]
- mackal 8y agoAMD apparently uses the PS4 Pro and Xbox One X CPU/GPU are made in this fab according to some other articles about this.
- mahgnous 8y agoLet the price gouging commence.
- dragosmocrii 8y agoShould we expect an increased cost as consumers? I remember when Hynix foundry had a fire, ram prices went up a lot
- sydney6 8y agoI believe so, yes. Supply and Demand.
- ohithereyou 8y agoOut of the GPU mining fire and into the incompetent fab frying pan.
- dooglius 8y agoDo TSMC or their customers have insurance for this sort of thing? It would seem prudent given the stakes.
- sevensor 8y agoLet me tell you a story about sulfuric acid. When I was working process integration at a fab, I had a long series of meetings about qualifying a new source of sulfuric acid. Not a new supplier even, nothing that risky. Just a new source, meaning the iso (tanker truck) full of sulfuric comes from a different plant owned by the same chemical company. What it comes down to is that it's basically impossible from a process integration perspective. Sure, you can take assays, and you do. But what about the things you aren't looking for in your assays? All you can do is plug it in, cross your fingers, and look at the yield trends. Not only does sulfuric mix in the supply lines, it gets used throughout the entire process. So if you do have some kind of strange contaminant that you didn't test for, maybe it only actually hurts you pre-gate. And the damage ramps up slowly and unevenly. But if your yield tanks, then you have to scramble to figure out what happened, and sulfuric isn't even your first suspect. (Because there are thousands of things like this that can tank your yields.) What's worse, you're literally over a barrel. Sometimes Plant A has to shut down and can't send you another iso. Do you take an iso from (unqualified) Plant B? Do you crack open a barrel of sulfuric? You have a stock of qualified sulfuric in barrels, for situations like this, but in practice barrels are worse than isos. Metal everywhere. I'm not saying it's sulfuric in this case. Lots of chemicals have a story like this one, but it would also not surprise me one little bit.
- parimm 8y agoWouldn't using multiple Spectrometry/Spectroscopy methods help in finding out the impurities in the process chemicals? ICP-MS and NMR should cover most elemental impurities.
- Const-me 8y agoI don't have any experience working in that industry. But based on what I know, modern ICs require exceptional levels of purity of the materials. A single atom can ruin a transistor, and therefore the complete chip. AFAIK the methods you mentioned aren't sensitive enough to detect individual atoms in barrels of stuff. E.g. ICP-MS detects 1E-15 concentrations, but in absolute numbers, 1E15 molecules is only 1.66e-9 mole, e.g. for iron (55.8g/mole), it translates to 1E+10 defects per kg of stuff. Way too many.
- fizixer 8y agoI love how Moore's law is a complete marketing gimmick at this point: - Intel not only hasn't released 10nm chips yet, they even stopped publicizing transistor count of their 14nm, the node from five years ago. - Not to mention their i7 chips node over node being marginal improvements for the past 10 years or so. - And all of a sudden, Intel's competitors seem to be right on track with 7nm, with the Moore's law giant itself, and with most resources, is struggling at 10nm.
- ggm 8y agoPrimo Levi "the periodic table" has a couple of chemist stories from his post WWII experiences, which go to this problem. unknown adulterants, effects on process chemistry, the wierd moments in "DONT CHANGE IT" culture. Older times, but similer problem? And, beautifully written.