5 ms·

I'm all for open things, but that's a false equivalence. You don't use those tools on a public road around unsuspecting others. In the same way you can't just

I'm all for open things, but that's a false equivalence. You don't use those tools on a public road around unsuspecting others. In the same way you can't just merrily hack about with a plane. The FAA don't really care that much if you die in your experiment. They do care if the burning wreckage falls on someone minding their own business.

jacquesm 3y ago
And what makes you think that the current crop of automotive software written in either asm or unsafe C is going to be any better than what you or I would produce? I've had a very recent model Mercedes C-class nearly kill me twice on account of buggy software. So much for that 'stellar' (pun intended) reputation. My current car is as dumb as it possibly could be. I'd expect that if any ECU software was to be released that we'd finally realize how bad things really are and that there would be a massive amount of work done on making sure these pieces of critical software would be as safe as they could possibly be. Note that the norm is 'a subset of C deemed to be safe' but that what I've seen of such development would not pass my personal threshold for quality work. In fact, rather the opposite. On the plus side, the hardware people usually know their stuff and realize what is dangerous to pass to the software people so with some luck your vehicle will use an FPGA for any kind of really safety critical stuff (or processors embedded with the relevant hardware, such as ABS and so on).
- formerly_proven 3y ago
  > I'd expect that if any ECU software was to be released that we'd finally realize how bad things really are and that there would be a massive amount of work done on making sure these pieces of critical software would be as safe as they could possibly be. toyota/denso michael barr testimony cough edit: oh, there's even slides now, you don't even have to read the court transcript https://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUBBED.pdf https://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRU...
  - jacquesm 3y ago
    Yes, that's one of the better known cases. But I've looked at similar stuff under NDA and I'm more than a little bit uncomfortable with some of the stuff that I've seen. The scariest kind of developer to me are the ones that believe that they and only they are able to create safe software without outside scrutiny of what they produce. Of course that scrutiny would have exposed emissions cheating software right on day #1.
    - intern4tional 3y ago
      This comment chain appears to have a fundamental misconception of what constitutes safe and what does not. Automotive standards and automotive coding standards approach safety in a different way than most people think (and given your comments I would say this includes you). If you're curious, you can have a look at some rules to evaluate automotive code that are published here: https://github.com/github/codeql-coding-standards https://github.com/github/codeql-coding-standards In short, the rules do not aim to eliminate failure or crashes, but rather make the crash predictable and uniform when a crash occurs so that it can be dealt with. This is further complicated by where and how the automotive manufacture chooses to implement safety controls. It is entirely possible to have a bunch of unsafe code running somewhere on a car, and simply have a small safety shim around said code that prevents the unsafe code from impacting the safe operation of the vehicle. With that in mind, let's take the example that you use here of emissions cheating software. Emissions is likely not considered safety relevant (it might not even be QM, it just might be some code) and so no safety requirement applies to it. So, no real scrutiny would happen there regardless, at least from a safety perspective. See, validating that software passes a particular safety certification is time and money intensive and manufacturers therefore keep the amount of code that they qualify as safe to a minimum. This means as an example that the infotainment systems of many manufacturers are not safety relevant and no safety function should exist on or interact with them. A few other things to consider from other threads: - Telsa doesn't necessarily follow or adhere to safety standards. They (Telsa) are explicitly non-compliant in some cases, and this is partially why there are investigations into their practices. - Industrial robotics code is just as bad if not worse than most automotive software from what I've seen. As you note, its that these robots are not under manual control - None of this prevents the software from being open source. There are plenty of safety qualified open source projects. This simply limits who can contribute and how contributions are managed. The main reason why many things in automotive are not open source is that the ECU manufacturer isn't interested in doing so, and the Tier 1/2/3 that does the implementation is even less so.
- mikrotikker 3y ago
  I got a friend who was almost killed by traction control. Pulling out of his driveway onto a 3 lane road, and having 2 cars 1 overtaking come around the corner very fast. He tried to punch it to get out of the way but: ”computer says no”. Longest 2 seconds of his life he reckons, waiting for the gas pedal to work again. Both cars had to brake very heavily. "Safety"
  - jacquesm 3y ago
    Yes. That sort of thing. I had a 2015 model C-class Mercedes make two fairly serious attempts at killing me before I got rid of it. The first time it could have been a glitch, the second was so obviously borked that I still wonder why they released that stuff on the road. And no way to disable that 'feature'. I'd have loved to analyze what exactly went wrong there and why a bridge and an advertisement by the side of the road were classified as imminent frontal collision. My present car just does what I tell it to, no gizmos. It may be statistically less safe (I'm willing to believe that) but at least it doesn't actively try to kill me in the name of keeping me safe.
LeifCarrotson 3y ago
No, Jacques and I hand those things off to random unskilled laborers who come work at a robot cell or CNC for $15/hr without any real knowledge of the code and safety standards. "Push the green button, trust us, it's safe." FAA regulations for experimental or kit aircraft are wide open. As are automotive standards, having just helped my buddy get his sand rail DOT certified for use on the road. A tube chassis, '65 Volkswagen rear end and VIN, safety glass and manually-squeezed washer bottle and manually-swung windshield wipers (for the DOT guy, then never used again), un-boosted non-ABS brakes with a cutting brake in the loop, and it's cleared for use on the Interstate if we were crazy enough to do that. It's only the manufacturers who claim they're protecting the public by locking down their designs.
- svorakang 3y ago
  Road approval looks very differently for mass-produced vehicles. It is not an easy thing to pass.

jacquesm 3y ago

And what makes you think that the current crop of automotive software written in either asm or unsafe C is going to be any better than what you or I would produce? I've had a very recent model Mercedes C-class nearly kill me twice on account of buggy software. So much for that 'stellar' (pun intended) reputation. My current car is as dumb as it possibly could be. I'd expect that if any ECU software was to be released that we'd finally realize how bad things really are and that there would be a massive amount of work done on making sure these pieces of critical software would be as safe as they could possibly be. Note that the norm is 'a subset of C deemed to be safe' but that what I've seen of such development would not pass my personal threshold for quality work. In fact, rather the opposite. On the plus side, the hardware people usually know their stuff and realize what is dangerous to pass to the software people so with some luck your vehicle will use an FPGA for any kind of really safety critical stuff (or processors embedded with the relevant hardware, such as ABS and so on).

formerly_proven 3y ago
> I'd expect that if any ECU software was to be released that we'd finally realize how bad things really are and that there would be a massive amount of work done on making sure these pieces of critical software would be as safe as they could possibly be. toyota/denso michael barr testimony cough edit: oh, there's even slides now, you don't even have to read the court transcript https://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUBBED.pdf https://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRU...
- jacquesm 3y ago
  Yes, that's one of the better known cases. But I've looked at similar stuff under NDA and I'm more than a little bit uncomfortable with some of the stuff that I've seen. The scariest kind of developer to me are the ones that believe that they and only they are able to create safe software without outside scrutiny of what they produce. Of course that scrutiny would have exposed emissions cheating software right on day #1.
  - intern4tional 3y ago
    This comment chain appears to have a fundamental misconception of what constitutes safe and what does not. Automotive standards and automotive coding standards approach safety in a different way than most people think (and given your comments I would say this includes you). If you're curious, you can have a look at some rules to evaluate automotive code that are published here: https://github.com/github/codeql-coding-standards https://github.com/github/codeql-coding-standards In short, the rules do not aim to eliminate failure or crashes, but rather make the crash predictable and uniform when a crash occurs so that it can be dealt with. This is further complicated by where and how the automotive manufacture chooses to implement safety controls. It is entirely possible to have a bunch of unsafe code running somewhere on a car, and simply have a small safety shim around said code that prevents the unsafe code from impacting the safe operation of the vehicle. With that in mind, let's take the example that you use here of emissions cheating software. Emissions is likely not considered safety relevant (it might not even be QM, it just might be some code) and so no safety requirement applies to it. So, no real scrutiny would happen there regardless, at least from a safety perspective. See, validating that software passes a particular safety certification is time and money intensive and manufacturers therefore keep the amount of code that they qualify as safe to a minimum. This means as an example that the infotainment systems of many manufacturers are not safety relevant and no safety function should exist on or interact with them. A few other things to consider from other threads: - Telsa doesn't necessarily follow or adhere to safety standards. They (Telsa) are explicitly non-compliant in some cases, and this is partially why there are investigations into their practices. - Industrial robotics code is just as bad if not worse than most automotive software from what I've seen. As you note, its that these robots are not under manual control - None of this prevents the software from being open source. There are plenty of safety qualified open source projects. This simply limits who can contribute and how contributions are managed. The main reason why many things in automotive are not open source is that the ECU manufacturer isn't interested in doing so, and the Tier 1/2/3 that does the implementation is even less so.
mikrotikker 3y ago
I got a friend who was almost killed by traction control. Pulling out of his driveway onto a 3 lane road, and having 2 cars 1 overtaking come around the corner very fast. He tried to punch it to get out of the way but: ”computer says no”. Longest 2 seconds of his life he reckons, waiting for the gas pedal to work again. Both cars had to brake very heavily. "Safety"
- jacquesm 3y ago
  Yes. That sort of thing. I had a 2015 model C-class Mercedes make two fairly serious attempts at killing me before I got rid of it. The first time it could have been a glitch, the second was so obviously borked that I still wonder why they released that stuff on the road. And no way to disable that 'feature'. I'd have loved to analyze what exactly went wrong there and why a bridge and an advertisement by the side of the road were classified as imminent frontal collision. My present car just does what I tell it to, no gizmos. It may be statistically less safe (I'm willing to believe that) but at least it doesn't actively try to kill me in the name of keeping me safe.

LeifCarrotson 3y ago

No, Jacques and I hand those things off to random unskilled laborers who come work at a robot cell or CNC for $15/hr without any real knowledge of the code and safety standards. "Push the green button, trust us, it's safe." FAA regulations for experimental or kit aircraft are wide open. As are automotive standards, having just helped my buddy get his sand rail DOT certified for use on the road. A tube chassis, '65 Volkswagen rear end and VIN, safety glass and manually-squeezed washer bottle and manually-swung windshield wipers (for the DOT guy, then never used again), un-boosted non-ABS brakes with a cutting brake in the loop, and it's cleared for use on the Interstate if we were crazy enough to do that. It's only the manufacturers who claim they're protecting the public by locking down their designs.

svorakang 3y ago
Road approval looks very differently for mass-produced vehicles. It is not an easy thing to pass.