7 ms·

As far as I can tell, Tesla plays fast and loose, treating their product like a manufacturer of consumer electronics and not a manufacturer of a dangerous and d

As far as I can tell, Tesla plays fast and loose, treating their product like a manufacturer of consumer electronics and not a manufacturer of a dangerous and durable good. That obviously allows them to out-compete other auto manufacturers who are more aware of things like product liability. See the Toyota accelerator-gate lawsuits. From https://en.wikipedia.org/wiki/2009%E2%80%932011_Toyota_vehicle_recalls https://en.wikipedia.org/wiki/2009%E2%80%932011_Toyota_vehic... : > However, on October 24, 2013, a jury ruled against Toyota and found that unintended acceleration could have been caused due to deficiencies in the drive-by-wire throttle system or Electronic Throttle Control System (ETCS). Michael Barr of the Barr Group testified[30] that NASA had not been able to complete its examination of Toyota's ETCS and that Toyota did not follow best practices for real time life critical software, and that a single bit flip which can be caused by cosmic rays could cause unintended acceleration. As well, the run-time stack of the real-time operating system was not large enough and that it was possible for the stack to grow large enough to overwrite data that could cause unintended acceleration.[31][32] As a result, Toyota has entered into settlement talks with its plaintiffs. Not following software best practices(i.e. ISO 26262) can expose your customers to unnecessary risk and leave your company vulnerable to lawsuits. Tesla may learn a very expensive lesson one day, or they may get lucky. Time will tell. --- Depending on the design of your system and software stack, switching SoCs can be anywhere from a massive effort to a simple recompilation. Even switching architectures could be trivial for some components(i.e. a UI written in HTML5 making REST API calls) or a nightmare for others(ECU, ABS or other real-time algorithm written to use bare metal modules on the SoC without an OS providing abstraction).

baybal2 5y ago
It is very troubling development when doing something so trivial even requires a computer. That's a matter readily solved by a PID controller made of a few dozen 74XXX parts. Possibly made n-times redundant. Even a more fancy LP solver probably wouldn't need a fully fledged computer.
- SkyPuncher 5y ago
  It's really easy to over react because of the silicone shortage, but in many cars these chips are replacing physically manufactured parts or significantly increasing efficiency in a way that isn't possible with physical based components. Electronic fuel injection is a _great_ example. Much of modern fuel efficiency stems from the fact that an engine can accurately control fuel in the cylinder based on a bunch of factors. Not only does this system require physically fewer materials, it uses less gas (in turn saving production effort).
  - baybal2 5y ago
    I think you are misunderstanding me. What I mean under a computer is a Turing complete machine. A circuit just doing some LP solving, or PID is not necessarily a computer.
    - SkyPuncher 5y ago
      By that explanation, I actually 100% confident I'm understanding you correctly. Most of the functionality of these chips can be represented in some form of hardware. However, that hardware is often much more expensive, significantly less flexible, and likely significantly bulkier.
    - rcxdude 5y ago
      Very little applications strictly need a turing complete machine. However microcontrollers are ubiquitous because they are so flexible. One chip does the job of many, and without needing to be specialised, allowing all applications to take advantage of the economies of scale in semiconductor manufacturing. (Also as pointed out here you drastically underestimate the complexity of a modern ECU)
- avianlyric 5y ago
  There’s nothing trivial about a modern internal combustion engine. The throttle control system is doing quite a bit more work than just opening and closing a physical throttle valve.
- dharmab 5y ago
  ECUs are simpler to build than the mechanical components they replaced and do things impossible to do mechanically (given the space and weight limits of a car). Jadon Cammisa has a great series on this stuff. Here's just one episode on one topic, drive by wire throttle control: https://youtu.be/gKsCHx5NOMM https://youtu.be/gKsCHx5NOMM
  - baybal2 5y ago
    Most drive by wire systems I seen are nothing, but a simplest closed loop PIDs. It's very close to what you are taught in the "How to write a PID controller" class without much extra creativity.
    - dharmab 5y ago
      Have you looked at any vehicle from the past 3-5 years? I own one with drive by wire and you can feel and hear it reacting to variables when you push it to the limits (either in inclement terrain or on a closed course). The linked video has specific video examples from production vehicles as well.
    - kevin_thibedeau 5y ago
      Deciding how much fuel will be injected each cycle is not just a simple PID. The throttle body control may just be a basic servo (although it probably isn't) but the system controlling it is more complicated.
      - baybal2 5y ago
        A separate circuit to map fuel/gas ratio for a given sensor input is also not a rocket science.
  - joking 5y ago
    Simpler to build, and even simpler to overbill for them. Is insane to charge over 1000$ for some replacement logic board.
- HeyLaughingBoy 5y ago
  Let's start by assuming that the engineers are at least basically competent, shall we? I mean, I could implement a PID controller without any 74xxx logic at all: a basic op-amp, a handful of resistors and capacitors and boom I'm done! But there's a reason that engineering doesn't do that these days and it's not that engineers don't know how. My PID control built from an opamp? In 2021 that's almost always a stupid idea. We use computers because doing it digitally has many advantages. We don't use 74xx (or any other logic family) for this because microcontrollers are far more flexible, allow for behavior changes without reworking hardware, allow inventories to scale (the same component can be used in hundreds of products) and more reasons I won't detail. The reality is that in 2021, often the simplest, most robust and cost-effective way to do something trivial is by using a computer to do it.
  - baybal2 5y ago
    > The reality is that in 2021, often the simplest, most robust and cost-effective way to do something trivial is by using a computer to do it. Very much not. Complicated MCUs are a very risky single vendor supply, and as we see now, people are now paying for an engineering overkill. 74xxx are available in every shape, and form, from dozens of suppliers.
- tziki 5y ago
  Ah the classic HN "just use this things I just came up with". https://xkcd.com/793/ https://xkcd.com/793/
  - cycomanic 5y ago
    I actually believe programmers/software engineers are the new physicists in this regard.
- kortex 5y ago
  Ok, lets assume for a moment that throttle can be modelled by a PID plant (it can't) - is the circuit temperature stable? You don't want the characteristics changing as the engine. Now you need a compensator circuit. What about power supply stability and noise? Car power busses are hella noisy, due to alternator and spark coil. Now you need some serious conditioning and filtering. What about knocking? Modern ECUs adjust parameters on knock detection to reduce the damage from pre-ignition. Gotta put in a control system for that. Repeat for manifold air temp, exhaust temp, emissions, rev limiter, etc., and as you mention, make that 3x for redundancy. Now you have hundreds to thousands of components, in a high noise, vibration, and temperature environment, and you have something which is about as efficient as an 80s car. And you can't as easily simulate it cause it's analog. Or just use some digital chips and simulate all the tuning.
- KZerda 5y ago
  Okay, so you've designed it. Now, make sure it works with the dirty electrical signal of a typical car's power bus. Then make sure it fits in the space requirements of under the car's hood, or in the door, or any of the other tight places that these embedded systems go. Then make sure that it's verified to act over several years of vehicle life. Then, make sure your suppliers will guarantee you that they won't discontinue the part for at least 10-15 years because of legal requirements for spare parts. Just throwing discrete logic at the problem doesn't always help.
- deleted 5y ago
  [deleted]
shiftpgdn 5y ago
Toyota accelerator-gate is largely a farce. Almost all of the people involved with Toyota unintended acceleration were over the age of 65 and had the poorly designed loose floor mats in their cars. Even if they had perfect readable John Carmack tier coding they still would have lost.
- 01100011 5y ago
  Sure but that's not related to the point I'm making. The point is that not following software design best practices can leave you open to liability, or at least put you in such a weak legal position that you are compelled to settle. If/when Tesla gets hauled into court over its software flaws, any lack of adherence to best practices will make the company appear negligent.
  - thoughtstheseus 5y ago
    This is a risk adverse culture then. Tesla is spending gobs of money specifically on building safer if not the safest cars out there.
    - kevingadd 5y ago
      How do you square that with them removing sensors from their cars and going all-in on doing self-driving exclusively using vision? If they were so passionate about spending money to make the safest cars, they'd be doing sensor fusion.
      - matz1 5y ago
        They specifically mention because camera is better than radar. It gives more information than radar. https://twitter.com/elonmusk/status/1380796939151704071?s=19 https://twitter.com/elonmusk/status/1380796939151704071?s=19
      - tantony 5y ago
        > If they were so passionate about spending money to make the safest cars, they'd be doing sensor fusion. That's your opinion. They have shown data where the information from radar was of a lower quality than what their vision stack was providing (e.g. lack of vertical resolution). Watch their AI day webcast for examples. Their current vision-only stack has been validated with LIDAR ground-truth data.
rectang 5y ago
From earlier discussions here (https://news.ycombinator.com/item?id=9643551 https://news.ycombinator.com/item?id=9643551 "10,000 global variables"), Toyota's conservative, arguably antiquated approach to software architecture wouldn't necessarily prove reliable as that software evolves. Trying to assess whether thousands of global variables are still playing nice during a major rewrite to accommodate a new chip would be definitely be difficult and time consuming! Not that a fast-and-loose approach is ideal, either. Writing reliable software in the context of an organization is hard.
- finnh 5y ago
  10K global variables sounds insane, until i realize "oh, huh, we have that too, we just call it _config_". Almost every place we would have a constant (tuning parameters, etc) we instead have a configurable value with the likely default declared in code, but all overridable in config. Managing config can be a hassle, but the number of times we've merely had to tweak a value rather than roll a new build pays for itself every day. We have thousands of such "global variables"... of course they are read-only, so aren't used to share state. If Toyota is doing that: nbd. If they are using them to share state ... god have mercy on their souls.
  - tremon 5y ago
    I always interpreted the "10K global variables" to mean 10K directly accessible, heap-allocated symbols. It's a code reek. If you use a configuration object or other abstraction, the number of accessible variables may not change, but their scope and access method does. It's a matter of interpretation I guess, but I would not classify a wrapper object (like .Net's ConfigurationManager singleton) as "10K global variables" even if the accompanying config contained 10K items, or if the ConfigurationManager backing store was preallocated in the data section of the binary.
    - delfinom 5y ago
      >I always interpreted the "10K global variables" to mean 10K directly accessible, heap-allocated symbols. It's a code reek. Which to me on an old _accelerator_ design is suspect because that's alot of heap for the micros that would have been used back in the day. This isn't a Linux system. It was at best a micro with like 8KB of RAM. (I'm not actually sure what it is but there's no way it was impressive).
pokerhobo 5y ago
See https://www.youtube.com/watch?v=w5c5KzpamiM https://www.youtube.com/watch?v=w5c5KzpamiM by someone who worked for a number of auto OEMs including Tesla where he explains how Tesla does agile hardware
- ngrilly 5y ago
  I watched the video. Thanks for sharing. Powerful approach. But wondering how close it is from reality or not. Would be interested to know more.
  - panick21 5y ago
    I would suggest you watch the Munro Live videos where they disassemble and analyses cars. This company has been doing this for decades and recently started making some videos. The changes in Tesla vehicles are absurdly fast. They disassembled original Model 3 and then China Model 3, and since then the refresh 3 and the Y. Recently they showed the new Model 3 LFP (Lithium Iron Phosphate) and the electronics and how it was integrated is already different again.
    - 01100011 5y ago
      How does that work out for maintainability? Is Tesla going to maintain parts inventory long enough to support owners for the lifetime of the vehicle(15-20 years?)?
agumonkey 5y ago
> That obviously allows them to out-compete other auto manufacturers who are more aware of things like product liability A lot of this era comes down to this. You have a legacy industry with tons of regulations, then a new guy comes up, steps on all the lines, and convinces people they're smarter for doing more for less. (up until regulations come back in the equation).
- panick21 5y ago
  Its very easy to claim that everybody else is not following regulation and that the only reason you are losing is because you do. Its much harder to prove. If anything, GM had more problems and failures of their EVs so far. So the claim that Tesla is ignoring liability and doing much higher risk development is literally just an accusation based on nothing. In actual fact, GM is spending 10s of billions buying back every single Bolt and they have been forced to shut down the line. At the same time Tesla had no such issues. So, how about these people actually prove or substantiate in some way that the new comer ignores regulation or quality. Specially when, they themselves have a worse record.
  - agumonkey 5y ago
    Oh in that case I was more thinking about Uber and the likes. In Tesla case, the only glaring problem is 'autopilot' the rest I have nothing to say.
ahepp 5y ago
Isn't the fact that Toyota lost this case damning to traditional automotive software development practices, rather than evidence of their importance? The traditional players are already doing a bad job trying to write safety critical software the way things are now.

baybal2 5y ago

It is very troubling development when doing something so trivial even requires a computer. That's a matter readily solved by a PID controller made of a few dozen 74XXX parts. Possibly made n-times redundant. Even a more fancy LP solver probably wouldn't need a fully fledged computer.

SkyPuncher 5y ago
It's really easy to over react because of the silicone shortage, but in many cars these chips are replacing physically manufactured parts or significantly increasing efficiency in a way that isn't possible with physical based components. Electronic fuel injection is a _great_ example. Much of modern fuel efficiency stems from the fact that an engine can accurately control fuel in the cylinder based on a bunch of factors. Not only does this system require physically fewer materials, it uses less gas (in turn saving production effort).
- baybal2 5y ago
  I think you are misunderstanding me. What I mean under a computer is a Turing complete machine. A circuit just doing some LP solving, or PID is not necessarily a computer.
  - SkyPuncher 5y ago
    By that explanation, I actually 100% confident I'm understanding you correctly. Most of the functionality of these chips can be represented in some form of hardware. However, that hardware is often much more expensive, significantly less flexible, and likely significantly bulkier.
  - rcxdude 5y ago
    Very little applications strictly need a turing complete machine. However microcontrollers are ubiquitous because they are so flexible. One chip does the job of many, and without needing to be specialised, allowing all applications to take advantage of the economies of scale in semiconductor manufacturing. (Also as pointed out here you drastically underestimate the complexity of a modern ECU)
avianlyric 5y ago
There’s nothing trivial about a modern internal combustion engine. The throttle control system is doing quite a bit more work than just opening and closing a physical throttle valve.
dharmab 5y ago
ECUs are simpler to build than the mechanical components they replaced and do things impossible to do mechanically (given the space and weight limits of a car). Jadon Cammisa has a great series on this stuff. Here's just one episode on one topic, drive by wire throttle control: https://youtu.be/gKsCHx5NOMM https://youtu.be/gKsCHx5NOMM
- baybal2 5y ago
  Most drive by wire systems I seen are nothing, but a simplest closed loop PIDs. It's very close to what you are taught in the "How to write a PID controller" class without much extra creativity.
  - dharmab 5y ago
    Have you looked at any vehicle from the past 3-5 years? I own one with drive by wire and you can feel and hear it reacting to variables when you push it to the limits (either in inclement terrain or on a closed course). The linked video has specific video examples from production vehicles as well.
  - kevin_thibedeau 5y ago
    Deciding how much fuel will be injected each cycle is not just a simple PID. The throttle body control may just be a basic servo (although it probably isn't) but the system controlling it is more complicated.
    - baybal2 5y ago
      A separate circuit to map fuel/gas ratio for a given sensor input is also not a rocket science.
- joking 5y ago
  Simpler to build, and even simpler to overbill for them. Is insane to charge over 1000$ for some replacement logic board.
HeyLaughingBoy 5y ago
Let's start by assuming that the engineers are at least basically competent, shall we? I mean, I could implement a PID controller without any 74xxx logic at all: a basic op-amp, a handful of resistors and capacitors and boom I'm done! But there's a reason that engineering doesn't do that these days and it's not that engineers don't know how. My PID control built from an opamp? In 2021 that's almost always a stupid idea. We use computers because doing it digitally has many advantages. We don't use 74xx (or any other logic family) for this because microcontrollers are far more flexible, allow for behavior changes without reworking hardware, allow inventories to scale (the same component can be used in hundreds of products) and more reasons I won't detail. The reality is that in 2021, often the simplest, most robust and cost-effective way to do something trivial is by using a computer to do it.
- baybal2 5y ago
  > The reality is that in 2021, often the simplest, most robust and cost-effective way to do something trivial is by using a computer to do it. Very much not. Complicated MCUs are a very risky single vendor supply, and as we see now, people are now paying for an engineering overkill. 74xxx are available in every shape, and form, from dozens of suppliers.
tziki 5y ago
Ah the classic HN "just use this things I just came up with". https://xkcd.com/793/ https://xkcd.com/793/
- cycomanic 5y ago
  I actually believe programmers/software engineers are the new physicists in this regard.
kortex 5y ago
Ok, lets assume for a moment that throttle can be modelled by a PID plant (it can't) - is the circuit temperature stable? You don't want the characteristics changing as the engine. Now you need a compensator circuit. What about power supply stability and noise? Car power busses are hella noisy, due to alternator and spark coil. Now you need some serious conditioning and filtering. What about knocking? Modern ECUs adjust parameters on knock detection to reduce the damage from pre-ignition. Gotta put in a control system for that. Repeat for manifold air temp, exhaust temp, emissions, rev limiter, etc., and as you mention, make that 3x for redundancy. Now you have hundreds to thousands of components, in a high noise, vibration, and temperature environment, and you have something which is about as efficient as an 80s car. And you can't as easily simulate it cause it's analog. Or just use some digital chips and simulate all the tuning.
KZerda 5y ago
Okay, so you've designed it. Now, make sure it works with the dirty electrical signal of a typical car's power bus. Then make sure it fits in the space requirements of under the car's hood, or in the door, or any of the other tight places that these embedded systems go. Then make sure that it's verified to act over several years of vehicle life. Then, make sure your suppliers will guarantee you that they won't discontinue the part for at least 10-15 years because of legal requirements for spare parts. Just throwing discrete logic at the problem doesn't always help.
deleted 5y ago
[deleted]

shiftpgdn 5y ago

Toyota accelerator-gate is largely a farce. Almost all of the people involved with Toyota unintended acceleration were over the age of 65 and had the poorly designed loose floor mats in their cars. Even if they had perfect readable John Carmack tier coding they still would have lost.

01100011 5y ago
Sure but that's not related to the point I'm making. The point is that not following software design best practices can leave you open to liability, or at least put you in such a weak legal position that you are compelled to settle. If/when Tesla gets hauled into court over its software flaws, any lack of adherence to best practices will make the company appear negligent.
- thoughtstheseus 5y ago
  This is a risk adverse culture then. Tesla is spending gobs of money specifically on building safer if not the safest cars out there.
  - kevingadd 5y ago
    How do you square that with them removing sensors from their cars and going all-in on doing self-driving exclusively using vision? If they were so passionate about spending money to make the safest cars, they'd be doing sensor fusion.
    - matz1 5y ago
      They specifically mention because camera is better than radar. It gives more information than radar. https://twitter.com/elonmusk/status/1380796939151704071?s=19 https://twitter.com/elonmusk/status/1380796939151704071?s=19
    - tantony 5y ago
      > If they were so passionate about spending money to make the safest cars, they'd be doing sensor fusion. That's your opinion. They have shown data where the information from radar was of a lower quality than what their vision stack was providing (e.g. lack of vertical resolution). Watch their AI day webcast for examples. Their current vision-only stack has been validated with LIDAR ground-truth data.

rectang 5y ago

From earlier discussions here (https://news.ycombinator.com/item?id=9643551 https://news.ycombinator.com/item?id=9643551 "10,000 global variables"), Toyota's conservative, arguably antiquated approach to software architecture wouldn't necessarily prove reliable as that software evolves. Trying to assess whether thousands of global variables are still playing nice during a major rewrite to accommodate a new chip would be definitely be difficult and time consuming! Not that a fast-and-loose approach is ideal, either. Writing reliable software in the context of an organization is hard.

finnh 5y ago
10K global variables sounds insane, until i realize "oh, huh, we have that too, we just call it _config_". Almost every place we would have a constant (tuning parameters, etc) we instead have a configurable value with the likely default declared in code, but all overridable in config. Managing config can be a hassle, but the number of times we've merely had to tweak a value rather than roll a new build pays for itself every day. We have thousands of such "global variables"... of course they are read-only, so aren't used to share state. If Toyota is doing that: nbd. If they are using them to share state ... god have mercy on their souls.
- tremon 5y ago
  I always interpreted the "10K global variables" to mean 10K directly accessible, heap-allocated symbols. It's a code reek. If you use a configuration object or other abstraction, the number of accessible variables may not change, but their scope and access method does. It's a matter of interpretation I guess, but I would not classify a wrapper object (like .Net's ConfigurationManager singleton) as "10K global variables" even if the accompanying config contained 10K items, or if the ConfigurationManager backing store was preallocated in the data section of the binary.
  - delfinom 5y ago
    >I always interpreted the "10K global variables" to mean 10K directly accessible, heap-allocated symbols. It's a code reek. Which to me on an old _accelerator_ design is suspect because that's alot of heap for the micros that would have been used back in the day. This isn't a Linux system. It was at best a micro with like 8KB of RAM. (I'm not actually sure what it is but there's no way it was impressive).

pokerhobo 5y ago

See https://www.youtube.com/watch?v=w5c5KzpamiM https://www.youtube.com/watch?v=w5c5KzpamiM by someone who worked for a number of auto OEMs including Tesla where he explains how Tesla does agile hardware

ngrilly 5y ago
I watched the video. Thanks for sharing. Powerful approach. But wondering how close it is from reality or not. Would be interested to know more.
- panick21 5y ago
  I would suggest you watch the Munro Live videos where they disassemble and analyses cars. This company has been doing this for decades and recently started making some videos. The changes in Tesla vehicles are absurdly fast. They disassembled original Model 3 and then China Model 3, and since then the refresh 3 and the Y. Recently they showed the new Model 3 LFP (Lithium Iron Phosphate) and the electronics and how it was integrated is already different again.
  - 01100011 5y ago
    How does that work out for maintainability? Is Tesla going to maintain parts inventory long enough to support owners for the lifetime of the vehicle(15-20 years?)?

agumonkey 5y ago

> That obviously allows them to out-compete other auto manufacturers who are more aware of things like product liability A lot of this era comes down to this. You have a legacy industry with tons of regulations, then a new guy comes up, steps on all the lines, and convinces people they're smarter for doing more for less. (up until regulations come back in the equation).

panick21 5y ago
Its very easy to claim that everybody else is not following regulation and that the only reason you are losing is because you do. Its much harder to prove. If anything, GM had more problems and failures of their EVs so far. So the claim that Tesla is ignoring liability and doing much higher risk development is literally just an accusation based on nothing. In actual fact, GM is spending 10s of billions buying back every single Bolt and they have been forced to shut down the line. At the same time Tesla had no such issues. So, how about these people actually prove or substantiate in some way that the new comer ignores regulation or quality. Specially when, they themselves have a worse record.
- agumonkey 5y ago
  Oh in that case I was more thinking about Uber and the likes. In Tesla case, the only glaring problem is 'autopilot' the rest I have nothing to say.

ahepp 5y ago

Isn't the fact that Toyota lost this case damning to traditional automotive software development practices, rather than evidence of their importance? The traditional players are already doing a bad job trying to write safety critical software the way things are now.