14 ms·

I get that this is a potential big problem, but I had to stop reading when the dog washing company blamed needing to respin a new board on a chip being out of s

I get that this is a potential big problem, but I had to stop reading when the dog washing company blamed needing to respin a new board on a chip being out of stock due to this. I'm sorry, but if you aren't buying enough chips to build all the boards you want to build of a revision, this is just what happens, chip shortage or no. Stuff gets obsoleted all the time, it doesn't take a chip shortage. Blaming a black swan shortage instead of your own poor planning for something as commonplace as needing to respin one board to make more new products is embarrassing.

syshum 5y ago
Never let a crisis go to waste Companies always piggy back on the current crisis to make changes or blame issues on that crisis that have nothing at all to do with the actual crisis... come to think about it, governments also do this.
- dahart 5y ago
  > come to think about it, governments also do this This was my thought watching the video in the article. It went on and on about supply shortages, China, and the Biden administration’s talks to prevent shortage as well as Chinese competition, and then near the end of the video, two little details slip by so fast you could miss them: auto makers cancelled their own orders, now they want their place in line back, and the biggest chip maker (TSMC) is in Taiwan, but it’s really close to China, so it still counts. Yes, preventing supply shortages and paying attention to national security are good things. But neither of those would address the specific “shortage” behind this article & video.
- TeMPOraL 5y ago
  And so do individuals. There's plenty of tragic or difficult moments that come unexpected in one's life, and it's easy to make them cover for unrelated issues, because questioning the explanation would be greatly untactful. A somewhat stereotypical example - a student that excuses their lack of homework and unpreparedness by mentioning their aunt died, even though they weren't really close with their aunt and they learned about the death an hour before the class started - it's an easy way out that nobody will dare question.
ndiddy 5y ago
Chips don't suddenly get obsoleted though, they get put in "not recommended for new designs" for at least a year or two beforehand, and when the chip maker plans to phase the chip out they typically give a "last time buy" notice at least a month or two before it gets discontinued.
- varjag 5y ago
  The normal courtesy for LTB is a year, often 18 months.
  - teclordphrack2 5y ago
    And if you have a contract to buy a certain amount from a supplier then they will hold that much stock in their own inventory to cover your contract.
- neltnerb 5y ago
  Apologies, I truly intended "out of stock" rather than "obsoleted" and agree 100% that there is a big difference.
crote 5y ago
I'm very loosely involved in the highly customized keyboard business, think runs of a couple hundred units. One vendor is having trouble finding any chip to design for. The software stack supports a couple of dozen STM32 chip series, but none of them are even remotely available. I've seen a lead time of over 11 months. Before COVID, pretty much every single chip was in stock almost all of the time. Respinning a board isn't too difficult, but good luck doing that if there's no chip to respin it for.
- structural 5y ago
  Yep, we're hearing 52week lead times on pretty much all microcontrollers at the moment, with more esoteric parts being... somewhat more than that. A year ago you could buy 10k+ units of most common parts -- in stock -- from each of the major distributors.
- baybal2 5y ago
  > Respinning a board isn't too difficult, but good luck doing that if there's no chip to respin it for. Some people hoard toilet paper, some people hoard baby formula, some people hoard Shanghai apartments, some people hoard microchips... Got burned with same STM32s recently. A purchaser been shopping parts, and ordered a given model of MCU by muscle memory. The price has moved one zero overnight, and we ran for quite a sum.
  - freeopinion 5y ago
    Some people fail to stock toilet people, some people fail to stock chip fabs.
- neltnerb 5y ago
  I don't doubt it, in the article the dog washing company specifically said that their design firm already had a solution that just required new PCBs. That is not sympathetic, that's very normal. What you're describing is sympathetic and hard to deal with. Their example was lucky about it and still ended up somehow quoted in the story.
- stemthirtywat 5y ago
99_00 5y ago
But it's a great save your ass tactic that's used all the time. If it's a big news story people are focused on it. And humans mistakenly attribute events to what the are focused on. Trump was a great scapegoat for local, city, regional, state politicians. All kinds of things can be blamed on climate change, systematic racism, etc. Basically anything that a leader can't fix but is in the news is a great scapegoat. Not to say that these aren't real issue. They are, but once you hear how they are over attributed as causal you can't unsee it.
TomVDB 5y ago
> Stuff gets obsoleted all the time, it doesn't take a chip shortage. This is just not true. But when it happens, there is usually an announcement well ahead of time (e.g. 1 year), as well as a last time buy option. Historically, there has never been a need to buy the chips of all the boards of a revision. If you are in the position of being able to buy all the components of all the boards you'll ever make for a certain revision, you're probably just above hobbyist level.
- neltnerb 5y ago
  I meant the parts that are not common enough to have pin-compatible substitutes from multiple vendors. I have worked with people way above hobbyist level and it is those obnoxious non-standard parts that are the ones that suddenly go out of stock everywhere even if they aren't actually obsoleted. Sometimes they just haven't done a run manufacturing that part number in a while and the stock ran out. I've seen this happen with everyone from ST to Microchip to Atmel (when it existed) to Qualcomm to Analog Devices (though interestingly never Texas Instruments so far); this is just something that happens... Thank you for the clarification, my initial comment was flippant and needed like four paragraphs of caveats.
  - TomVDB 5y ago
    “The parts that don’t have a pin compatible substitute” is usually 80% of your total BOM cost. It’s not a helpful clarification. Let’s take something totally ordinary like an STM32 MCU, and let’s ignore for a moment that there are now some clones on the market, with questionable compatibility at times. Even though they’re unique parts, there is no way anybody sane would stock up on those for the lifetime of a product revision. It was never needed and it’s ridiculous to plan for a black swan event like the one we’re experiencing now. And here’s why: even if you had planned for a sudden shortage of an STM32, you’d still be screwed on some generic components. Because I’ve seen people get stuck recently because they couldn’t source certain generic diodes. It makes no financial sense to always plan for the worst possible case. The whole reason distributors exist is because they are the buffer that moderates spikes in supply and demand. The system has worked very well for decades. It’s much better to be right or wrong along with everybody else than to be wrong 99% of the time (and waste margins compared to the competition), and being able to say “I told you so” to the rest once. And that 99% is not hyperbole.
robomartin 5y ago
> I had to stop reading when the dog washing company blamed needing to respin a new board on a chip being out of stock due to this ... > if you aren't buying enough chips to build all the boards you want to build of a revision Friendly advice: If you are going to be a consultant to industry, don't post comments like this. As someone who has been manufacturing tech products for over thirty years, my first reaction to your comment was "this guy doesn't have a clue". Then I looked at your site and was absolutely floored. My guess is you have lived in what I like to call the "SBIR distortion field", which is a domain that is very, very far from the realities of, say, a dog washing company. Not just because of usually just having to make one or a few of something (rather than 10,000), but also because of the financial dynamics of these programs --I have experience in that domain as well. Your vision of how this dog washing-machine company should operate does not align with the realities of a business outside of the "SBIR distortion field". Companies don't have cash reserves to fill the warehouse to the brim with components and product, weather a storm, keep the business afloat and everyone employed simultaneously. On top of that, manufacturing at any non-trivial scale is such a cash intensive endeavor that cash must be managed very carefully. If you buy too much inventory you can end-up in financial dire straits. The phase lag between spending money to manufacture a product and getting a return on that investment can be in the order of months, and that assumes a "linear" market. If you include R&D in that equation, it's even worse, years. I experienced this personally back in 2008. I did precisely what you suggested above and filled the warehouse with some two million dollars in components and assemblies to get ready for sales of our new product. We had demand. In fact, the purchase of the components and assemblies was triggered by receiving a purchase order for five million dollars of this product. And that was just one customer. I didn't know better. I thought it was perfectly sensible to place large PO's for critical components that would cover us for at least a year and tool-up. We even bought a bunch of brand new CNC equipment to bring manufacturing of heat sinks and other mechanical components in-house in order to reduce our cost basis. In fact, interestingly enough given some of what you have on your site, I made the single largest purchase to date (at that time) from Osram's high power LED division. No company in the world had ordered that many high power LEDs from them. And then the music stopped. The economy came to a grinding halt. Sales went to ZERO. The five million dollar purchase order? They went insolvent when their bank cut-down and eventually cancelled their line of credit. Other orders from major companies were put on hold (we had a PO but were told they were not going to accept deliveries, so, don't ship). We went from having tens of millions of dollars in orders for that product and that year to, effectively, zero. What was the end result? It was very rough. All of our cash was in the warehouse, on shelves, as components and assemblies we could not sell. We couldn't even get a loan to weather the storm. Nobody was buying anything, not at scale anyhow. We had to sell some of our component inventory for ten or twenty cents on the dollar just to bring in cash. It was worthless. I had to take a second mortgage on my home and use credit cards to make payroll (big mistakes, both of them). We survived for two years on bread crumbs. And then I had to shut down the company. It too me years to even be able to talk about this episode of my life to anyone. It was horrible. The two millions dollars I spent on "buying enough chips to build all the boards", as you put it (it was more than chips, but the example fits) was the single biggest mistake I have made in my business career. And this one cost me a business I built over a decade, starting in my garage with $5,000 to receiving a $30MM acquisition offer just as the economy took a shit (the offer was rescinded). So, please, pretty please, with sugar on top, if you want to be a consultant, don't say anything unless you really understand it. In this case, you clearly do not. To someone like me --who has actually lived through many ups and downs in life and business-- such comments result in what I will call "less-than-favorable conclusions" about the author. This isn't good for a consultant, unless the consulting is in a domain that does not necessarily align with reality outside of something like the SBIR/academic domains. It took years to recover, both mentally and financially. I eventually launched a new business, also in tech. Today we are facing having to manufacture 10K to 20K units per month of a new product. When we started design we picked readily available components and went on to design the product over about twelve months (real product design for scale manufacturing takes time). Today, as we approach production requirements, we are being quoted anywhere from 40 to 50 weeks for some of the components. In other words, we can't even buy them. We are having to consider having multiple alternative designs to see if we can manufacture functionally equivalent versions of the hardware using different chip sets. This means all of our regulatory and safety testing --another thing you ignored-- (FCC, CE, TUV, UL, environmental, thermal, lifetime, etc.) has to be redone, not once, but likely four to six times (depending on how many versions we end-up with). It's a nightmare. And, no, buying a million chips a year ago wasn't the solution. The cash drain would have resulted in people losing their jobs and possibly even going out of business again as sales levels last years went down some 80%. You buy as close to just-in-time as you possibly can. This practice has gained acceptance over the years for a reason. Sadly, I happen to have learned the lesson the hard way. If you have to weather a storm it is far better to have cash in the bank than a warehouse full of worthless components that you can't turn into cash precisely because of the storm. "A man holding a cat by the tail learns something he can learn in no other way". --Mark Twain So true.
nostromo 5y ago
This is largely a result of just-in-time manufacturing. Companies hate holding on to inventory. Most companies now don’t hold on to any extra stock needed to manufacture their products. This is great for your finances in normal times. But don’t complain when there is a shortage and you literally have zero slack for delays and your factory sits idle — that’s the well-known drawback.
- avs733 5y ago
  I agree but with a caveat, it's largely a result of BAD just-in-time manufacturing. Zero inventory production systems (ZIPS) are what people often mean when they talk about JIT, they are different. Toyota, one of the originators and leaders of JIT/lean has not had this problem...because they don't blindly ZIPS [0]. Not holding inventory is great for a lot of reasons, but it is a calculated risk...Toyota learned from experience that if they want to do it, they need to understand risk all the way up their supply chain. They largely aren't having this problem. They de-risked certain things because they really treat TPS and everything associated with it as a philosophy not a set of heuristics that should just be implemented blindly. Manufacturing something is all tradeoffs...there is (almost) nothing with a universal upside. Sure I want to hold less inventory, but JIT is actually about manufacturing time, not inventory. If I have NO inventory and NO ability to get inventory my manufacturing time goes up while I wait with my thumb in the fertilizer pipe. I've worked with a couple of manufacturing plants (and consultants...) that treat 'JIT' inventory management as something that can simply be pushed off to vendors and then the upside of less inventory enjoyed. They have specs and forms and certification and paperwork...but nobody looks at it. Those are the companies struggling now. They outsourced without fully understanding the risk of the outsourcing. Usually this doesn't bite the world, it bites one or two companies that relied on a certain part (someone misses an EOL notice) or a certain vendor (who goes bust because the owner's grandson ran the thing into the ground). It's like my students who make choices in the first week of the semester that seem minor...and then are frustrated when it effects their grade at the end of the semester (sorry, too much grading this week) [0] https://www.autoblog.com/2021/03/09/toyota-how-it-avoided-semiconductor-shortage/ https://www.autoblog.com/2021/03/09/toyota-how-it-avoided-se...
  - 5y ago
bluGill 5y ago
Chips shouldn't go obsolete so fast. Your local auto parts store has in stock parts for cars made 50 years ago. That include aftermarket parts made in the past few years. You cannot support right to repair if you support the idea that chips go obsolete.
- baybal2 5y ago
  It's actually a "feature." Automakers like to have easily sun-settable, hard to replace parts to quickly remove old cars off the market. Now, they want to do it even quicker. DRMed autoparts are a new craze. A tsunami of them is coming in 2021+ cars. They all now want to do the the John Deere trick. They gave the industry a very bad example to follow.
  - bluGill 5y ago
    As an employee of John Deere I can assure you that is false. We are proud that machines made in the 1950s are still in regular use, and we make a ton of money providing replacement parts for machines that old. We have also spent large amount of dollars over the last decade replacing electronics that have gone obsolete, not to mention buying and storing spare parts so we can continue to replace those old electronics for customers. Our bottom line would look a lot better if we didn't have to redesign perfectly good electronics all the time. I can't go into more detail than that.
    - CamperBob2 5y ago
      Our bottom line would look a lot better if we didn't have to redesign perfectly good electronics all the time. Well, I can think of at least one easy way to avoid that problem. If you document your hardware and put a leash on your lawyers, you'll find that your customers and their independent service organizations will be glad to take on this responsibility for you.
- jlawer 5y ago
  A few things help the auto part example over consumer electronics * the model was setup while the auto makers were substantially smaller. GM / Ford couldn't hope to control the entire supply chain in the 1930s like they can now. These pre-existing businesses are able to protect their interests. * Early cars had massive defect rates compared to modern consumer electronics. You HAD to be able to have a large number of people able to repair and maintain cars. * Due to the value of cars, a higher proportion of damage is likely to be considered repairable over a CE device. * Many auto parts were simpler when the model setup. Many parts did not have extensive IP protections, and were relatively easily clone-able. (i.e. you can make a mould of a part and cast other copies). Not to mention, outside a few halo proprietary technologies, companies were less litigious and protective of IP. * Commercial operators such as Taxi and corporate fleets mean that there was a market for these 3rd party products, enough to cover R&D costs of cloning an existing part. I think we need to enforce a compulsory licensing of IP for all parts, chips, etc after a period of non-availability or obsolescence. Price should be set at a point where 3rd party companies come in and produce generic versions, paying royalties to the owner. Allow these 3rd parties to reverse engineer the products and sell their clones if there is truly a market need. What will likely happen is that the companies would rather license their designs to a chosen partner, in the hope of having more control over it
teclordphrack2 5y ago
What are you talking about. The vast majority of companies use just in time stocking. If you sell x amount of a product that is when you order x amount of each item on the BOM. It has been this way for decades for manufacturing in the usa. There are tax and other liabilities when keeping more than required stocks on hand.
baybal2 5y ago
> I get that this is a potential big problem, but I had to stop reading when the dog washing company blamed needing to respin a new board on a chip being out of stock due to this. What you are suggesting is quite impractical, unless you are a really big business with cash, which can simply direct order components. Small businesses, even in Shenzhen, a place inundated with supply chain abundance, always have to either keep running from one small wholesaler, to another searching for components, or pay n-times the price working with somebody like Arrow. In my experience, you can't safeguard yourself against such things as a small company no matter what. I worked in, and around OEM electronics since 2007, and things like having to redesign a product 4 times a year to accommodate a supply chain disruption were happening even in the best years. Nothing special with the current disruption besides the scale. This is also the reason why Asian electronics makers have such short product lifetimes. It's usually easier just to sunset a product, and make an improved, and better version with newer components, than to fight against the always evolving supply chain. And I not talking about small companies, ASUS, Acer, MSI, and such all practice this. I know few people running the Chuwi brand. What they do as a small maker is that the moment the get a good consignment of chipsets, and other parts, they spin a laptop model solely for that batch alone. Then, they live off it until they get another good parts purchase, when they usually sell their component leftovers, or do few final runs if they can find people wanting to buy them for rebadging. The entirety of small volume laptop industry spins around chipsets, and screens — hardest things to find for a small company. I want you people to take a looks on a big difference in how companies in the West, and here handle the crisis: Western brands wail, cry, and wait for component supplies to resume, while Asian brands just keep releasing new products with parts coming into their hands, and making great cash from this shortage.
- neltnerb 5y ago
  I agree and have experienced that too. Even without obsoleted parts, stuff will be impossible to find all the sudden, particularly specialty components. Heck, I've had vendors who simply had fabs that caught fire... but I always have a laundry list of substitutes that are pin compatible when humanly possible, which is for almost all the parts on the board. For the ones that aren't possible, that's a much smaller BOM to buy ahead of time if you absolutely have to keep the PCB design the same. The caveat is that because MCUs are not standardized you frequently get screwed over so my replacement part list is a collection of versions of the part with different amounts of memory that are designed to be pin-compatible. Sometimes that isn't enough, and often companies aren't big enough to negotiate a guaranteed supply. But if you're not big enough to negotiate a guaranteed supply you just have to deal with buying enough of the parts to keep you in business long enough to create a new revision before you literally run out of stuff to sell... I sympathize with the other comment berating me at length for being flippant about this, and am genuinely sorry for being flippant especially in light of their experience with the opposite issue (buying stuff and getting screwed by customers canceling orders). But I was talking about a dog washing startup. A dog washing startup vocally complaining to the news about needing to respin one board because of a supply chain interruption. Yes, there is FCC and other testing you have to do if you significantly modify a board, so I'm not sitting here oblivious to the challenges involved in a respin. My response was to them being a poster child for the people suffering from this issue. Those of you who cannot find any component that will do the job have my absolute sympathy and I apologize for not being more verbose.
simonbarker87 5y ago
Many small manufactures can’t afford to order much more than the MOQ on parts. PCBs generally have much higher MOQs than parts you can buy from digi key, I ran a manufacturing business fit years and a PCB MOQ could last me a whole year but I could say cash flow on buying the components more just in time. It’s not “poor planning” it’s working within the realities of your business and it’s limitations (and he’s saving a few hundred in a month at cash low could have been the difference between me just not taking a pay check that month or paying part of an employees wage from my own savings on top of not getting paid myself)
CamperBob2 5y ago
ROFL. Yeah, I'll just do a lifetime buy of these: https://www.digikey.com/en/products/detail/xilinx-inc/XCVU13P-2FLGA2577E/7604526 https://www.digikey.com/en/products/detail/xilinx-inc/XCVU13... ... as soon as my board hits production. If they give me any grief at DigiKey, I'll have my VP of Procurement fly up there in the company G5 to give them a piece of my mind.
jbgreer 5y ago
Even if you did order enough to produce a run, there is no guarantee you’ll get the full quantity or any at all. Suffice it to say I know this first hand. Had a vendor come back and say, “Sorry, we know we promised your order by June, but it isn’t going to happen.” Thankfully getting a break on a new part, but respin and recertification cost and time are also causing heartburn.
- dapids 5y ago
  Sounds like a contracting problem to me.
ecpottinger 5y ago
I had a customer years ago using a Vic-20 as the controller for his greenhouse. The Vic-20 at that time had already been discontinued. He bought in two units to fix to have in stock since he already had a unit running his greenhouse. I asked him why use Vic-20s when there were a number of controllers already on the market. He said the problem he never knew when the present unit may fail and the Vic-20 did the job of an $10,000 controller which he did not know how long it would take to get the present controller to work on his system and the present one works fine. It has been decades since I did the work and I would not be surprised if the system is still working today because he was prepared.
Cadwhisker 5y ago
I work at a decent-sized electronics company with several dozen products on the market at any one time. Lead times for chips were usually 12 weeks in normal times. We have forecast sales going off at least 6 months or a year away. We have known where the shortages will be for some months now. We have already re-spun PCB designs to work in alternate ICs for some of the ones that we know will become unavailable; dual-footprint in case the originals come back. This took time and effort to design and verify and all that work could have gone into the design of new products, but it had to be done. This has put us some number of months behind our original planned dates for our new products. The problem is that we can see shortages coming up which we cannot work around. We cannot buy enough parts, the lead-times are astonishing. At some point we are going to have to get lucky or halt production.
spfzero 5y ago
Sometimes, well, actually all the time, you can’t exactly predict the eventual demand for a new, unique product.
varjag 5y ago
Re-spinning a board, LOL. Dude, there's copper substrate shortage on the market, people have delays getting their already ordered PCBs delivered.
- KirillPanov 5y ago
  > there's copper substrate shortage on the market, people have delays getting their already ordered PCBs delivered. Any details on this? Media/tradejournal links? I wouldn't have believed you six months ago, but hey, everything's changed. Still, copper-clad board shortages... I mean if that really is for real we're headed back to the stone age real quick. I hope it's just a rumor.
  - xadhominemx 5y ago
    Before berating someone in the industry for “source”, why don’t you try googling it? PCB shortage, ABF shortage, etc. Scores of recent articles at your fingertips.
    - KirillPanov 5y ago
      ... and yet you post no links. Also: I wasn't berating anybody.
  - varjag 5y ago
    Very much everyone in the business is aware of it since winter. but sure https://www.linkedin.com/pulse/crisis-printed-circuit-board-supply-karuna-karan/ https://www.linkedin.com/pulse/crisis-printed-circuit-board-... https://elmatica.com/portfolio/copper-foil-shortage-persists/ https://elmatica.com/portfolio/copper-foil-shortage-persists...
    - KirillPanov 5y ago
      Thank you for your constructive reply!

syshum 5y ago

Never let a crisis go to waste Companies always piggy back on the current crisis to make changes or blame issues on that crisis that have nothing at all to do with the actual crisis... come to think about it, governments also do this.

dahart 5y ago
> come to think about it, governments also do this This was my thought watching the video in the article. It went on and on about supply shortages, China, and the Biden administration’s talks to prevent shortage as well as Chinese competition, and then near the end of the video, two little details slip by so fast you could miss them: auto makers cancelled their own orders, now they want their place in line back, and the biggest chip maker (TSMC) is in Taiwan, but it’s really close to China, so it still counts. Yes, preventing supply shortages and paying attention to national security are good things. But neither of those would address the specific “shortage” behind this article & video.
TeMPOraL 5y ago
And so do individuals. There's plenty of tragic or difficult moments that come unexpected in one's life, and it's easy to make them cover for unrelated issues, because questioning the explanation would be greatly untactful. A somewhat stereotypical example - a student that excuses their lack of homework and unpreparedness by mentioning their aunt died, even though they weren't really close with their aunt and they learned about the death an hour before the class started - it's an easy way out that nobody will dare question.

ndiddy 5y ago

Chips don't suddenly get obsoleted though, they get put in "not recommended for new designs" for at least a year or two beforehand, and when the chip maker plans to phase the chip out they typically give a "last time buy" notice at least a month or two before it gets discontinued.

varjag 5y ago
The normal courtesy for LTB is a year, often 18 months.
- teclordphrack2 5y ago
  And if you have a contract to buy a certain amount from a supplier then they will hold that much stock in their own inventory to cover your contract.
neltnerb 5y ago
Apologies, I truly intended "out of stock" rather than "obsoleted" and agree 100% that there is a big difference.

crote 5y ago

I'm very loosely involved in the highly customized keyboard business, think runs of a couple hundred units. One vendor is having trouble finding any chip to design for. The software stack supports a couple of dozen STM32 chip series, but none of them are even remotely available. I've seen a lead time of over 11 months. Before COVID, pretty much every single chip was in stock almost all of the time. Respinning a board isn't too difficult, but good luck doing that if there's no chip to respin it for.

structural 5y ago
Yep, we're hearing 52week lead times on pretty much all microcontrollers at the moment, with more esoteric parts being... somewhat more than that. A year ago you could buy 10k+ units of most common parts -- in stock -- from each of the major distributors.
baybal2 5y ago
> Respinning a board isn't too difficult, but good luck doing that if there's no chip to respin it for. Some people hoard toilet paper, some people hoard baby formula, some people hoard Shanghai apartments, some people hoard microchips... Got burned with same STM32s recently. A purchaser been shopping parts, and ordered a given model of MCU by muscle memory. The price has moved one zero overnight, and we ran for quite a sum.
- freeopinion 5y ago
  Some people fail to stock toilet people, some people fail to stock chip fabs.
neltnerb 5y ago
I don't doubt it, in the article the dog washing company specifically said that their design firm already had a solution that just required new PCBs. That is not sympathetic, that's very normal. What you're describing is sympathetic and hard to deal with. Their example was lucky about it and still ended up somehow quoted in the story.
stemthirtywat 5y ago

99_00 5y ago

But it's a great save your ass tactic that's used all the time. If it's a big news story people are focused on it. And humans mistakenly attribute events to what the are focused on. Trump was a great scapegoat for local, city, regional, state politicians. All kinds of things can be blamed on climate change, systematic racism, etc. Basically anything that a leader can't fix but is in the news is a great scapegoat. Not to say that these aren't real issue. They are, but once you hear how they are over attributed as causal you can't unsee it.

TomVDB 5y ago

> Stuff gets obsoleted all the time, it doesn't take a chip shortage. This is just not true. But when it happens, there is usually an announcement well ahead of time (e.g. 1 year), as well as a last time buy option. Historically, there has never been a need to buy the chips of all the boards of a revision. If you are in the position of being able to buy all the components of all the boards you'll ever make for a certain revision, you're probably just above hobbyist level.

neltnerb 5y ago
I meant the parts that are not common enough to have pin-compatible substitutes from multiple vendors. I have worked with people way above hobbyist level and it is those obnoxious non-standard parts that are the ones that suddenly go out of stock everywhere even if they aren't actually obsoleted. Sometimes they just haven't done a run manufacturing that part number in a while and the stock ran out. I've seen this happen with everyone from ST to Microchip to Atmel (when it existed) to Qualcomm to Analog Devices (though interestingly never Texas Instruments so far); this is just something that happens... Thank you for the clarification, my initial comment was flippant and needed like four paragraphs of caveats.
- TomVDB 5y ago
  “The parts that don’t have a pin compatible substitute” is usually 80% of your total BOM cost. It’s not a helpful clarification. Let’s take something totally ordinary like an STM32 MCU, and let’s ignore for a moment that there are now some clones on the market, with questionable compatibility at times. Even though they’re unique parts, there is no way anybody sane would stock up on those for the lifetime of a product revision. It was never needed and it’s ridiculous to plan for a black swan event like the one we’re experiencing now. And here’s why: even if you had planned for a sudden shortage of an STM32, you’d still be screwed on some generic components. Because I’ve seen people get stuck recently because they couldn’t source certain generic diodes. It makes no financial sense to always plan for the worst possible case. The whole reason distributors exist is because they are the buffer that moderates spikes in supply and demand. The system has worked very well for decades. It’s much better to be right or wrong along with everybody else than to be wrong 99% of the time (and waste margins compared to the competition), and being able to say “I told you so” to the rest once. And that 99% is not hyperbole.

robomartin 5y ago

> I had to stop reading when the dog washing company blamed needing to respin a new board on a chip being out of stock due to this ... > if you aren't buying enough chips to build all the boards you want to build of a revision Friendly advice: If you are going to be a consultant to industry, don't post comments like this. As someone who has been manufacturing tech products for over thirty years, my first reaction to your comment was "this guy doesn't have a clue". Then I looked at your site and was absolutely floored. My guess is you have lived in what I like to call the "SBIR distortion field", which is a domain that is very, very far from the realities of, say, a dog washing company. Not just because of usually just having to make one or a few of something (rather than 10,000), but also because of the financial dynamics of these programs --I have experience in that domain as well. Your vision of how this dog washing-machine company should operate does not align with the realities of a business outside of the "SBIR distortion field". Companies don't have cash reserves to fill the warehouse to the brim with components and product, weather a storm, keep the business afloat and everyone employed simultaneously. On top of that, manufacturing at any non-trivial scale is such a cash intensive endeavor that cash must be managed very carefully. If you buy too much inventory you can end-up in financial dire straits. The phase lag between spending money to manufacture a product and getting a return on that investment can be in the order of months, and that assumes a "linear" market. If you include R&D in that equation, it's even worse, years. I experienced this personally back in 2008. I did precisely what you suggested above and filled the warehouse with some two million dollars in components and assemblies to get ready for sales of our new product. We had demand. In fact, the purchase of the components and assemblies was triggered by receiving a purchase order for five million dollars of this product. And that was just one customer. I didn't know better. I thought it was perfectly sensible to place large PO's for critical components that would cover us for at least a year and tool-up. We even bought a bunch of brand new CNC equipment to bring manufacturing of heat sinks and other mechanical components in-house in order to reduce our cost basis. In fact, interestingly enough given some of what you have on your site, I made the single largest purchase to date (at that time) from Osram's high power LED division. No company in the world had ordered that many high power LEDs from them. And then the music stopped. The economy came to a grinding halt. Sales went to ZERO. The five million dollar purchase order? They went insolvent when their bank cut-down and eventually cancelled their line of credit. Other orders from major companies were put on hold (we had a PO but were told they were not going to accept deliveries, so, don't ship). We went from having tens of millions of dollars in orders for that product and that year to, effectively, zero. What was the end result? It was very rough. All of our cash was in the warehouse, on shelves, as components and assemblies we could not sell. We couldn't even get a loan to weather the storm. Nobody was buying anything, not at scale anyhow. We had to sell some of our component inventory for ten or twenty cents on the dollar just to bring in cash. It was worthless. I had to take a second mortgage on my home and use credit cards to make payroll (big mistakes, both of them). We survived for two years on bread crumbs. And then I had to shut down the company. It too me years to even be able to talk about this episode of my life to anyone. It was horrible. The two millions dollars I spent on "buying enough chips to build all the boards", as you put it (it was more than chips, but the example fits) was the single biggest mistake I have made in my business career. And this one cost me a business I built over a decade, starting in my garage with $5,000 to receiving a $30MM acquisition offer just as the economy took a shit (the offer was rescinded). So, please, pretty please, with sugar on top, if you want to be a consultant, don't say anything unless you really understand it. In this case, you clearly do not. To someone like me --who has actually lived through many ups and downs in life and business-- such comments result in what I will call "less-than-favorable conclusions" about the author. This isn't good for a consultant, unless the consulting is in a domain that does not necessarily align with reality outside of something like the SBIR/academic domains. It took years to recover, both mentally and financially. I eventually launched a new business, also in tech. Today we are facing having to manufacture 10K to 20K units per month of a new product. When we started design we picked readily available components and went on to design the product over about twelve months (real product design for scale manufacturing takes time). Today, as we approach production requirements, we are being quoted anywhere from 40 to 50 weeks for some of the components. In other words, we can't even buy them. We are having to consider having multiple alternative designs to see if we can manufacture functionally equivalent versions of the hardware using different chip sets. This means all of our regulatory and safety testing --another thing you ignored-- (FCC, CE, TUV, UL, environmental, thermal, lifetime, etc.) has to be redone, not once, but likely four to six times (depending on how many versions we end-up with). It's a nightmare. And, no, buying a million chips a year ago wasn't the solution. The cash drain would have resulted in people losing their jobs and possibly even going out of business again as sales levels last years went down some 80%. You buy as close to just-in-time as you possibly can. This practice has gained acceptance over the years for a reason. Sadly, I happen to have learned the lesson the hard way. If you have to weather a storm it is far better to have cash in the bank than a warehouse full of worthless components that you can't turn into cash precisely because of the storm. "A man holding a cat by the tail learns something he can learn in no other way". --Mark Twain So true.

nostromo 5y ago

This is largely a result of just-in-time manufacturing. Companies hate holding on to inventory. Most companies now don’t hold on to any extra stock needed to manufacture their products. This is great for your finances in normal times. But don’t complain when there is a shortage and you literally have zero slack for delays and your factory sits idle — that’s the well-known drawback.

avs733 5y ago
I agree but with a caveat, it's largely a result of BAD just-in-time manufacturing. Zero inventory production systems (ZIPS) are what people often mean when they talk about JIT, they are different. Toyota, one of the originators and leaders of JIT/lean has not had this problem...because they don't blindly ZIPS [0]. Not holding inventory is great for a lot of reasons, but it is a calculated risk...Toyota learned from experience that if they want to do it, they need to understand risk all the way up their supply chain. They largely aren't having this problem. They de-risked certain things because they really treat TPS and everything associated with it as a philosophy not a set of heuristics that should just be implemented blindly. Manufacturing something is all tradeoffs...there is (almost) nothing with a universal upside. Sure I want to hold less inventory, but JIT is actually about manufacturing time, not inventory. If I have NO inventory and NO ability to get inventory my manufacturing time goes up while I wait with my thumb in the fertilizer pipe. I've worked with a couple of manufacturing plants (and consultants...) that treat 'JIT' inventory management as something that can simply be pushed off to vendors and then the upside of less inventory enjoyed. They have specs and forms and certification and paperwork...but nobody looks at it. Those are the companies struggling now. They outsourced without fully understanding the risk of the outsourcing. Usually this doesn't bite the world, it bites one or two companies that relied on a certain part (someone misses an EOL notice) or a certain vendor (who goes bust because the owner's grandson ran the thing into the ground). It's like my students who make choices in the first week of the semester that seem minor...and then are frustrated when it effects their grade at the end of the semester (sorry, too much grading this week) [0] https://www.autoblog.com/2021/03/09/toyota-how-it-avoided-semiconductor-shortage/ https://www.autoblog.com/2021/03/09/toyota-how-it-avoided-se...
- 5y ago

bluGill 5y ago

Chips shouldn't go obsolete so fast. Your local auto parts store has in stock parts for cars made 50 years ago. That include aftermarket parts made in the past few years. You cannot support right to repair if you support the idea that chips go obsolete.

baybal2 5y ago
It's actually a "feature." Automakers like to have easily sun-settable, hard to replace parts to quickly remove old cars off the market. Now, they want to do it even quicker. DRMed autoparts are a new craze. A tsunami of them is coming in 2021+ cars. They all now want to do the the John Deere trick. They gave the industry a very bad example to follow.
- bluGill 5y ago
  As an employee of John Deere I can assure you that is false. We are proud that machines made in the 1950s are still in regular use, and we make a ton of money providing replacement parts for machines that old. We have also spent large amount of dollars over the last decade replacing electronics that have gone obsolete, not to mention buying and storing spare parts so we can continue to replace those old electronics for customers. Our bottom line would look a lot better if we didn't have to redesign perfectly good electronics all the time. I can't go into more detail than that.
  - CamperBob2 5y ago
    Our bottom line would look a lot better if we didn't have to redesign perfectly good electronics all the time. Well, I can think of at least one easy way to avoid that problem. If you document your hardware and put a leash on your lawyers, you'll find that your customers and their independent service organizations will be glad to take on this responsibility for you.
jlawer 5y ago
A few things help the auto part example over consumer electronics * the model was setup while the auto makers were substantially smaller. GM / Ford couldn't hope to control the entire supply chain in the 1930s like they can now. These pre-existing businesses are able to protect their interests. * Early cars had massive defect rates compared to modern consumer electronics. You HAD to be able to have a large number of people able to repair and maintain cars. * Due to the value of cars, a higher proportion of damage is likely to be considered repairable over a CE device. * Many auto parts were simpler when the model setup. Many parts did not have extensive IP protections, and were relatively easily clone-able. (i.e. you can make a mould of a part and cast other copies). Not to mention, outside a few halo proprietary technologies, companies were less litigious and protective of IP. * Commercial operators such as Taxi and corporate fleets mean that there was a market for these 3rd party products, enough to cover R&D costs of cloning an existing part. I think we need to enforce a compulsory licensing of IP for all parts, chips, etc after a period of non-availability or obsolescence. Price should be set at a point where 3rd party companies come in and produce generic versions, paying royalties to the owner. Allow these 3rd parties to reverse engineer the products and sell their clones if there is truly a market need. What will likely happen is that the companies would rather license their designs to a chosen partner, in the hope of having more control over it

teclordphrack2 5y ago

What are you talking about. The vast majority of companies use just in time stocking. If you sell x amount of a product that is when you order x amount of each item on the BOM. It has been this way for decades for manufacturing in the usa. There are tax and other liabilities when keeping more than required stocks on hand.

baybal2 5y ago

> I get that this is a potential big problem, but I had to stop reading when the dog washing company blamed needing to respin a new board on a chip being out of stock due to this. What you are suggesting is quite impractical, unless you are a really big business with cash, which can simply direct order components. Small businesses, even in Shenzhen, a place inundated with supply chain abundance, always have to either keep running from one small wholesaler, to another searching for components, or pay n-times the price working with somebody like Arrow. In my experience, you can't safeguard yourself against such things as a small company no matter what. I worked in, and around OEM electronics since 2007, and things like having to redesign a product 4 times a year to accommodate a supply chain disruption were happening even in the best years. Nothing special with the current disruption besides the scale. This is also the reason why Asian electronics makers have such short product lifetimes. It's usually easier just to sunset a product, and make an improved, and better version with newer components, than to fight against the always evolving supply chain. And I not talking about small companies, ASUS, Acer, MSI, and such all practice this. I know few people running the Chuwi brand. What they do as a small maker is that the moment the get a good consignment of chipsets, and other parts, they spin a laptop model solely for that batch alone. Then, they live off it until they get another good parts purchase, when they usually sell their component leftovers, or do few final runs if they can find people wanting to buy them for rebadging. The entirety of small volume laptop industry spins around chipsets, and screens — hardest things to find for a small company. I want you people to take a looks on a big difference in how companies in the West, and here handle the crisis: Western brands wail, cry, and wait for component supplies to resume, while Asian brands just keep releasing new products with parts coming into their hands, and making great cash from this shortage.

neltnerb 5y ago
I agree and have experienced that too. Even without obsoleted parts, stuff will be impossible to find all the sudden, particularly specialty components. Heck, I've had vendors who simply had fabs that caught fire... but I always have a laundry list of substitutes that are pin compatible when humanly possible, which is for almost all the parts on the board. For the ones that aren't possible, that's a much smaller BOM to buy ahead of time if you absolutely have to keep the PCB design the same. The caveat is that because MCUs are not standardized you frequently get screwed over so my replacement part list is a collection of versions of the part with different amounts of memory that are designed to be pin-compatible. Sometimes that isn't enough, and often companies aren't big enough to negotiate a guaranteed supply. But if you're not big enough to negotiate a guaranteed supply you just have to deal with buying enough of the parts to keep you in business long enough to create a new revision before you literally run out of stuff to sell... I sympathize with the other comment berating me at length for being flippant about this, and am genuinely sorry for being flippant especially in light of their experience with the opposite issue (buying stuff and getting screwed by customers canceling orders). But I was talking about a dog washing startup. A dog washing startup vocally complaining to the news about needing to respin one board because of a supply chain interruption. Yes, there is FCC and other testing you have to do if you significantly modify a board, so I'm not sitting here oblivious to the challenges involved in a respin. My response was to them being a poster child for the people suffering from this issue. Those of you who cannot find any component that will do the job have my absolute sympathy and I apologize for not being more verbose.

simonbarker87 5y ago

Many small manufactures can’t afford to order much more than the MOQ on parts. PCBs generally have much higher MOQs than parts you can buy from digi key, I ran a manufacturing business fit years and a PCB MOQ could last me a whole year but I could say cash flow on buying the components more just in time. It’s not “poor planning” it’s working within the realities of your business and it’s limitations (and he’s saving a few hundred in a month at cash low could have been the difference between me just not taking a pay check that month or paying part of an employees wage from my own savings on top of not getting paid myself)

CamperBob2 5y ago

ROFL. Yeah, I'll just do a lifetime buy of these: https://www.digikey.com/en/products/detail/xilinx-inc/XCVU13P-2FLGA2577E/7604526 https://www.digikey.com/en/products/detail/xilinx-inc/XCVU13... ... as soon as my board hits production. If they give me any grief at DigiKey, I'll have my VP of Procurement fly up there in the company G5 to give them a piece of my mind.

jbgreer 5y ago

Even if you did order enough to produce a run, there is no guarantee you’ll get the full quantity or any at all. Suffice it to say I know this first hand. Had a vendor come back and say, “Sorry, we know we promised your order by June, but it isn’t going to happen.” Thankfully getting a break on a new part, but respin and recertification cost and time are also causing heartburn.

dapids 5y ago
Sounds like a contracting problem to me.

ecpottinger 5y ago

I had a customer years ago using a Vic-20 as the controller for his greenhouse. The Vic-20 at that time had already been discontinued. He bought in two units to fix to have in stock since he already had a unit running his greenhouse. I asked him why use Vic-20s when there were a number of controllers already on the market. He said the problem he never knew when the present unit may fail and the Vic-20 did the job of an $10,000 controller which he did not know how long it would take to get the present controller to work on his system and the present one works fine. It has been decades since I did the work and I would not be surprised if the system is still working today because he was prepared.

Cadwhisker 5y ago

I work at a decent-sized electronics company with several dozen products on the market at any one time. Lead times for chips were usually 12 weeks in normal times. We have forecast sales going off at least 6 months or a year away. We have known where the shortages will be for some months now. We have already re-spun PCB designs to work in alternate ICs for some of the ones that we know will become unavailable; dual-footprint in case the originals come back. This took time and effort to design and verify and all that work could have gone into the design of new products, but it had to be done. This has put us some number of months behind our original planned dates for our new products. The problem is that we can see shortages coming up which we cannot work around. We cannot buy enough parts, the lead-times are astonishing. At some point we are going to have to get lucky or halt production.

spfzero 5y ago

Sometimes, well, actually all the time, you can’t exactly predict the eventual demand for a new, unique product.

varjag 5y ago

Re-spinning a board, LOL. Dude, there's copper substrate shortage on the market, people have delays getting their already ordered PCBs delivered.

KirillPanov 5y ago
> there's copper substrate shortage on the market, people have delays getting their already ordered PCBs delivered. Any details on this? Media/tradejournal links? I wouldn't have believed you six months ago, but hey, everything's changed. Still, copper-clad board shortages... I mean if that really is for real we're headed back to the stone age real quick. I hope it's just a rumor.
- xadhominemx 5y ago
  Before berating someone in the industry for “source”, why don’t you try googling it? PCB shortage, ABF shortage, etc. Scores of recent articles at your fingertips.
  - KirillPanov 5y ago
    ... and yet you post no links. Also: I wasn't berating anybody.
- varjag 5y ago
  Very much everyone in the business is aware of it since winter. but sure https://www.linkedin.com/pulse/crisis-printed-circuit-board-supply-karuna-karan/ https://www.linkedin.com/pulse/crisis-printed-circuit-board-... https://elmatica.com/portfolio/copper-foil-shortage-persists/ https://elmatica.com/portfolio/copper-foil-shortage-persists...
  - KirillPanov 5y ago
    Thank you for your constructive reply!