13 ms·

> “We were able to substitute alternative chips, and then write the firmware in a matter of weeks,” Musk said. “It’s not just a matter of swapping out a chip; y

• baybal2 5y ago
  ARM M family cores can run each other's code more or less. Only the libraries for I/O need to be rewritten. In some cases, there are drop in replacements (Gigadevices STM32 clones)
  • im_down_w_otp 5y ago
    Are there even any ARM Cortex-M chips that are ASIL-B rated or better?
    • detaro 5y ago
      yes. ARM sells appropriate designs, and e.g. NXP offers some chips.
    • TaylorPhebillo 5y ago
      I think the Cortex-R chips are more targeted here- I think the V8-R chips are designed specifically for automotive use, so I assume they have appropriate ratings.
      • baybal2 5y ago
        There are very lazy car companies which simply specify "automotive certified" car chips for everything, including MCUs running window switches. So, don't be surprised $20 ARM-R based MCUs being used to do something trivial like driving window motor. I seen a few dashboard where there is an STM32 sitting connected to a single button, whose only duty is to register a key press, and burp something on the CanBus in response.
        • coryrc 5y ago
          I guarantee you no one is running $20 MCUs in trivial applications (in normal times). They're very cost-conscious.
        • hulitu 5y ago
          Every component ( resistors, transistors, microcontrollers, etc ) must be Automotive qualified (AEC-Q ). That's how you have a minimum standard. A button cannot communicate on the CAN bus that's why you need the microcontroller.
        • kortex 5y ago
          A) I doubt you need a $20 component to register a key press and do some canbus IO B) If you did have a $20 microcontroller for Canbus control of a window motor driver but it saved $19 in extra wiring/labor (driver controls usually route to all windows), provides reliable debounce, automated one-push-to-open, and allows things like holding keyless lock to close all windows, it's totally worth it.
        • sbierwagen 5y ago
          A big chunk of this crisis was caused by car manufacturers being insanely focused on cost cutting. Doing obviously dumb things with BOM isn't exactly a calling card of the automotive industry.
    • im_down_w_otp 5y ago
      Looks like the Cortex-R52 and Cortex-R5, a CPU and MCU, respectively both have been ASIL-D (the highest rating) qualified. Which is fantastic frankly. There's a bunch of legacy PPC, MIPS, and TriCore tech that's just miserable to work with because of their ancient and/or bespoke & proprietary toolchains. So, really the automotive industry lacks excuses that I would buy for why they're not moving forward onto new platforms other than the institutional inertia and supply-chain entanglements that its prioritizing instead.
  • legulere 5y ago
    That relies on I/O being written as replaceable units of code and not sprinkled throughout all of the code.
  • pwg 5y ago
    Except that in some cases, the automakers are not using a CPU with external interface hardware, they are using microcontrollers similar to the PIC series, example here: https://ww1.microchip.com/downloads/en/DeviceDoc/40300C.pdf https://ww1.microchip.com/downloads/en/DeviceDoc/40300C.pdf (Note, I'm not suggesting they use these exact chips, this is just an example of the level of onboard integration that is available for micro-controller chips). Single chip system, with built in analog comparators, timers, signal capture and PWM generation module, serial UART, etc. I.e., a whole "system on a chip" with the external world interfaces already built into the chip. Changing one of these for an ARM CPU with external analog comparators, timers, PWM modules, serial UART's, etc., is a redesign effort, not just a "change the chip" effort. If the clock speed possible via the internal clock oscillator is sufficient, then this chip needs only +5V and ground (plus programming) to be able to interface to and control some external analog or digital equipment.
• JohnJamesRambo 5y ago
  I feel like GM and Ford are at US Government and NASA size where getting any change through takes so long and so much middle management and red tape that it stretches to infinity. I’m sure Tesla is more agile than that. For better or worse sometimes…
  • mathattack 5y ago
    The bigger the company, the more effort that’s needed to coordinate the parts. At some point one person can’t keep the whole in the head and has to lean on process. Tesla is small enough, and has a CEO that can keep everything in his head.
    • ulfw 5y ago
      Tesla has over 70,000 employees. I know people think Elon is god, but come on, he can't have "everything in his head". Tesla isn't some mom and pop shop.
• jbverschoor 5y ago
  Weeks? This has been going on for almost 100 weeks
  • mkr-hn 5y ago
    I assume the alternative chips are still in short supply, but less so than their first choice. Everyone else is looking for alternatives too.
• LennyWhiteJr 5y ago
  To elaborate on this, the feature sets on these low level microcontrollers are no standardized. There is no common API for them to implement. Even chips coming from the same manufacturer will have different hardware capabilities, and although low-level drivers can abstract that way to a certain extent, there will always be differences. The biggest challenge is when you need to update your firmware to use a microcontroller from a different manufacturer. Often times these chips are specifically chosen due to the set of hardware functionality they offer, and the firmware is written to take advantage of that from the start. The two are coupled. Now you are forced to use a different chip, and the firmware that was written for a specific set of hardware now has to be modified for a new chip that may have a different feature set. Things fine print on how things like Analog to Digital converters becomes extremely important.
  • Espressosaurus 5y ago
    Even a single hardware iteration intended to be a drop-in replacement in some cases won't be due to relying on implementation-defined behavior that is not guaranteed by the datasheet and wasn't considered a constraint by the people designing the silicon. Sometimes it's a bug, sometimes it's just "didn't think that was important". Either way, you're left with something where the saturation behavior changed, or it's no longer possible to read out a value without risk of corruption since they assumed it can be treated as write-only, or some other hard to find and debug problem. Swapping out chips is an exercise in testing and risk management, and never should be done without care, even before we start talking about safety critical applications.
  • rapjr9 5y ago
    I see this as a failing of microprocessor design. I/O should be standardized and there should be high level API's that do not change across chips or companies. MCU's are intentionally fragile in this respect to try to lock buyers into one MCU product family. It is absolutely stupid that I/O ports work differently for different MCU's and don't use a common API. It means every designer always has to reinvent the wheel when moving to a new MCU. This must have had a huge negative effect on product reliability and innovation. It's also why many products have moved to using Arduino's. There is also a huge lack of improvement in MCU designs. Why doesn't my SPI bus autonegotiate everything? Why can't the serial port do the same? This should all be available in hardware as a standard feature of the ports. What a waste of decades of engineering talent to have to repeatedly troubleshoot everything at the bit level.
• AtlasBarfed 5y ago
  I get that what Musk quoted is not a trivial task, and if accurate, is probably much quicker than OEMs and Big Auto could do since Tesla is presumably much better at software. The article quotes Intel at 16nm, which is, what, 10 years from cutting edge process node? At this point mature OEMs and Big Auto should have been on a refresh process to auto-migrate forward the chips. The semiconductor industry is 40-50 years old now. And new generations produce better chips (cost, power use, performance). As other comments say, it smacks of laziness and lack of forecasting. Tesla has other advantages, since they are so much more vertically integrated, they can probably manage migrations a lot more easily and centrally. Big Auto is more like "Big assemble OEM parts". OEMs make all the components, Big Auto just wants to screw them into place and wire them together. It's part of why they suck at software that integrates things: They don't make any of the components, and a dozen different departments order them from a hundred suppliers, so getting interfaces/protocols/specs is a lot more time consuming than for Tesla.
• 01100011 5y ago
  As far as I can tell, Tesla plays fast and loose, treating their product like a manufacturer of consumer electronics and not a manufacturer of a dangerous and durable good. That obviously allows them to out-compete other auto manufacturers who are more aware of things like product liability. See the Toyota accelerator-gate lawsuits. From https://en.wikipedia.org/wiki/2009%E2%80%932011_Toyota_vehicle_recalls https://en.wikipedia.org/wiki/2009%E2%80%932011_Toyota_vehic... : > However, on October 24, 2013, a jury ruled against Toyota and found that unintended acceleration could have been caused due to deficiencies in the drive-by-wire throttle system or Electronic Throttle Control System (ETCS). Michael Barr of the Barr Group testified[30] that NASA had not been able to complete its examination of Toyota's ETCS and that Toyota did not follow best practices for real time life critical software, and that a single bit flip which can be caused by cosmic rays could cause unintended acceleration. As well, the run-time stack of the real-time operating system was not large enough and that it was possible for the stack to grow large enough to overwrite data that could cause unintended acceleration.[31][32] As a result, Toyota has entered into settlement talks with its plaintiffs. Not following software best practices(i.e. ISO 26262) can expose your customers to unnecessary risk and leave your company vulnerable to lawsuits. Tesla may learn a very expensive lesson one day, or they may get lucky. Time will tell. --- Depending on the design of your system and software stack, switching SoCs can be anywhere from a massive effort to a simple recompilation. Even switching architectures could be trivial for some components(i.e. a UI written in HTML5 making REST API calls) or a nightmare for others(ECU, ABS or other real-time algorithm written to use bare metal modules on the SoC without an OS providing abstraction).
  • baybal2 5y ago
    It is very troubling development when doing something so trivial even requires a computer. That's a matter readily solved by a PID controller made of a few dozen 74XXX parts. Possibly made n-times redundant. Even a more fancy LP solver probably wouldn't need a fully fledged computer.
    • SkyPuncher 5y ago
      It's really easy to over react because of the silicone shortage, but in many cars these chips are replacing physically manufactured parts or significantly increasing efficiency in a way that isn't possible with physical based components. Electronic fuel injection is a _great_ example. Much of modern fuel efficiency stems from the fact that an engine can accurately control fuel in the cylinder based on a bunch of factors. Not only does this system require physically fewer materials, it uses less gas (in turn saving production effort).
• echelon 5y ago
  Why did car companies allow themselves to make such a strategic mistake? Shouldn't they be on a standard microcontroller architecture? Why not even use ARM, X86, etc? It'd save them money at this point. And there wouldn't be a supply risk.
  • im_down_w_otp 5y ago
    Tier 1 suppliers have an incredible amount of influence in that ecosystem, and Tier 1 suppliers have an incentive to not be easily replaceable. That'd be my guess. I mean, it's an entire MASSIVE industry that's comprised almost entirely of truly commodity technology, and so you get all the perverse outcomes that happen when trying to "differentiate" despite being commodity by inventing ways to be "special" to drive lock-in. Interoperability only really benefits the OEM, and the OEM isn't the only part of the chain.
  • clairity 5y ago
    i'd guess adding an abstraction layer (between a general purpose microcontroller and the custom interfaces it needs to support) is likely not worth the added complexity (and bugs), since cars are on development cycles of many years anyway (for the benefit of simpler software development, and the risk reduction of being able to swap out the microcontroller if need be).
    • im_down_w_otp 5y ago
      They actually theoretically have a thing for this already. It's called AUTOSAR (now AUTOSAR Classic). It's supposed to be a component architecture to write down the software against a framework that can be backed by different MCUs. Of course, in practice much of the underlying esoteria of any given MCU/board configuration bled up through the abstractions and/or produced custom extensions.
      • RealityVoid 5y ago
        And AUTOSAR is a nightmare of a framework where it's nigh impossible to understand what is going on because of the crazy abstractions.
• jillesvangurp 5y ago
  Tesla is much better at writing software than most of their competitors. That's how they can adapt so quickly. They also do a lot more in house than their competitors. That means they can optimize their development processes, and align with chip suppliers across different components. If you have to work with a multitude of suppliers that each ship their own hardware and software, life is a lot more complicated. Changes take years in such an environment. Even a simple thing such as over the air updates to software is still science fiction in a large part of the industry. VW famously struggled with doing that for the ID.3 only managing their first updates fairly recently. Of course, Tesla is still affected by supply issues as well. They can't switch suppliers every quarter.
  • IshKebab 5y ago
    Are they better at writing software or do they just care less about how safety critical it is?
    • filoleg 5y ago
      It isn’t that their engineers are inherently better at writing software. The parent comment explained exactly why their organizational and supply chain structure allows them to be more agile and better at writing software and delivering it.
      • hef19898 5y ago
        I personally would rank Tesla's Supply Chain to high, so.
    • coryrc 5y ago
      I think you underestimate how poorly software development is done in traditional industries. Practice makes perfect and they mostly practice writing reports not software.
      • hulitu 5y ago
        It depends. If you want to sleep well at night you take care. Of course if you are VW you take risks.
• kube-system 5y ago
  CPUs aren’t the only silicon out of stock right now. Most silicon doesn’t even run software.
• jowday 5y ago
  I mean they also just dropped certain feature, like radar on autopilot.
• hulitu 5y ago
  If this is true, stay away from anything Tesla does. I work in the industry. 2 examples: 1) you have to change a transistor with one from a different supplier, same characteristics. You need to do some HW engineering tests. If the transistor is part of a safety relevant function, you need to do also system, SW and safety tests. And then environmental and EMC tests. Of course you can tailor but you need a very good justification. 2) You change the microcontroller. You need new SW, new tools (emulators - takes some time to be delivered), new sourcing, new HW design etc. It is from development perspective a new product so you need testing (HW, SW, System, Safety) and design validation (environmental tests, EMC) , maybe more than one loop.And when everything is passed and sourcing is done (i.e. you can buy the component) you can start production.
• stefan_ 5y ago
  What software can turn a MSP430 or 68 knockoff into a highly efficient power transistor? Take everything Musk says with some caution..
• rootusrootus 5y ago
  That’s a fancy way of portraying big tech debt. They will pay for these decisions later.
• GeorgeTirebiter 5y ago
  So much speculation here. Let me set the record straight(er). (I worked in Tesla FW, left 5 years ago.) Tesla does use industry best practices. All code must pass MISRA checkers, code is modular, and reused when possible. We used all sorts of chips (that were auto qual) of many different architectures. We had extensive tools for e.g. stack monitoring, diagnostic readouts, and much more. If you move from, say, one ST ARM chip to another one in a similar family, the peripherals may be a little different, but generally work the same way (with maybe a few more or fewer features). So reworking the I/O drivers is work, but given the excellent layering of Tesla "body control" FW, it's really quite straightforward to pick a different processor from a given family. It's for sure true that if you went from ST ARM to Freescale/NXP ARM the peripherals are diff, so yes, more time would be needed to write the I/O drivers. But the effort ongoing when I left was exactly to make the appropriate SW generalizations so it would be possible to do exactly this.