5 ms·

So, take this into account, and add in self driving cars. The complexity involved in these, and the coordination required, is nontrivial and its clear from the

So, take this into account, and add in self driving cars. The complexity involved in these, and the coordination required, is nontrivial and its clear from the article they do NOT have a good handle on things. I mean, most cars do seem to get around OK, so I'm partly amazed that under the hood, things could really be so scary. But we can't be lazy about adding complexity and systems, especially as more faith is placed on them. That this is not NASA grade code (or anything close) does not give me any warm fuzzies. It reeks of unprofessionalism, greed, and laziness. There needs to be a sane-software assesment. People rely on products with an ever increasing amount of source code - it would be interesting if a third party could come along and certify that a given codebase is not a big scary unmaintainable mess; not to say there won't be bugs or issues, but that these guys are at least trying to make a well designed system, and aren't doing a bunch of crazy stupid things.

deleted 11y ago
[deleted]
- meesterdude 11y ago
  Shouldn't toyota be closer to NASA than google? I mean, we're talking about systems that peoples lives depend on. I would be OK if my website was built by some IT department, if it meant that some level of NASA engineering is going into the control systems i rely on in my day to day life. That this is not the case is a little surprising to me, to be honest.
thaumaturgy 11y ago
The companies currently most actively involved in automated automobiles (hah) are pretty good at what they do. Volvo has been producing big trucks for several years now that have remarkable safety features, like automatically detecting stopped traffic ahead and bringing the fully loaded truck to a complete stop in what seems like an impossibly short distance (https://www.youtube.com/watch?v=HoCknasKdRU https://www.youtube.com/watch?v=HoCknasKdRU). There are plenty of things I dislike about Google but I can't say they're bad at software. If anybody can build a safe automated vehicle, they can. Tesla, too, who have a lot riding on their hard-earned reputation for building next-generation vehicles. I'm an old-school car guy, I haven't been a big fan of the complexity added to cars ever since the really bad emissions control systems in 80s cars, but even I have to admit that all that complexity has probably saved a lot more lives than it has cost -- antilock brakes, airbags, better pollution control, better fuel economy, way better structural safety features.
- userbinator 11y ago
  If you've ever driven a vehicle with them you'd know that air brakes are very, very powerful.
Sanddancer 11y ago
There are standards, Toyota simply chose to ignore all of them. Standards such as MISRA [1] and DO-178C [2] exist for the purpose of ensuring software quality in safety-critical situations. Most embedded software development environments even include tooling to help verify that you're not doing the things that are unsafe. The problem lies more in that automakers aren't required to use any such standard, unlike what the FAA has required for decades. [1] http://en.wikipedia.org/wiki/Motor_Industry_Software_Reliability_Association http://en.wikipedia.org/wiki/Motor_Industry_Software_Reliabi... [2] http://en.wikipedia.org/wiki/DO-178C http://en.wikipedia.org/wiki/DO-178C
- meesterdude 11y ago
  Very awesome comment! I didn't know there were standards, but it's great to see there are. However, Toyota should have followed them. I mean, did they think they knew better? I think with self driving cars, the standards will become more requirements; though I doubt before a series of deaths result from software failure.
- rainforest 11y ago
  Given that a lot of computer vision work is based on randomised algorithms, do you think that these standards would be enough? You could demonstrate 100% MC/DC coverage through a neural net implementation, but the weights are where the faults probably exist, for example.
  - mtrimpe 11y ago
    That actually leads to a very interesting point; would self driving cars be vulnerable to adversarial imagery? Neural nets are well known for being easily fooled [1] ... I wonder if you could create similar situations for self-driving cars. [1] http://www.evolvingai.org/fooling http://www.evolvingai.org/fooling
- lambdaelite 11y ago
  Toyota couldn't ignore these (voluntary) standards given that MISRA-C nor the earlier DO-178 didn't exist at the time the code was developed.
derefr 11y ago
Self-driving cars seem less scary to me. You can take unreliable components (like, say, a car with crappy firmware) and make a reliable system out of it if the software running on top is fault-tolerant. The "autopilot", unlike a human, will pull over the moment the car's brakes begin to fail, etc.

deleted 11y ago

[deleted]

meesterdude 11y ago
Shouldn't toyota be closer to NASA than google? I mean, we're talking about systems that peoples lives depend on. I would be OK if my website was built by some IT department, if it meant that some level of NASA engineering is going into the control systems i rely on in my day to day life. That this is not the case is a little surprising to me, to be honest.

thaumaturgy 11y ago

The companies currently most actively involved in automated automobiles (hah) are pretty good at what they do. Volvo has been producing big trucks for several years now that have remarkable safety features, like automatically detecting stopped traffic ahead and bringing the fully loaded truck to a complete stop in what seems like an impossibly short distance (https://www.youtube.com/watch?v=HoCknasKdRU https://www.youtube.com/watch?v=HoCknasKdRU). There are plenty of things I dislike about Google but I can't say they're bad at software. If anybody can build a safe automated vehicle, they can. Tesla, too, who have a lot riding on their hard-earned reputation for building next-generation vehicles. I'm an old-school car guy, I haven't been a big fan of the complexity added to cars ever since the really bad emissions control systems in 80s cars, but even I have to admit that all that complexity has probably saved a lot more lives than it has cost -- antilock brakes, airbags, better pollution control, better fuel economy, way better structural safety features.

userbinator 11y ago
If you've ever driven a vehicle with them you'd know that air brakes are very, very powerful.

Sanddancer 11y ago

There are standards, Toyota simply chose to ignore all of them. Standards such as MISRA [1] and DO-178C [2] exist for the purpose of ensuring software quality in safety-critical situations. Most embedded software development environments even include tooling to help verify that you're not doing the things that are unsafe. The problem lies more in that automakers aren't required to use any such standard, unlike what the FAA has required for decades. [1] http://en.wikipedia.org/wiki/Motor_Industry_Software_Reliability_Association http://en.wikipedia.org/wiki/Motor_Industry_Software_Reliabi... [2] http://en.wikipedia.org/wiki/DO-178C http://en.wikipedia.org/wiki/DO-178C

meesterdude 11y ago
Very awesome comment! I didn't know there were standards, but it's great to see there are. However, Toyota should have followed them. I mean, did they think they knew better? I think with self driving cars, the standards will become more requirements; though I doubt before a series of deaths result from software failure.
rainforest 11y ago
Given that a lot of computer vision work is based on randomised algorithms, do you think that these standards would be enough? You could demonstrate 100% MC/DC coverage through a neural net implementation, but the weights are where the faults probably exist, for example.
- mtrimpe 11y ago
  That actually leads to a very interesting point; would self driving cars be vulnerable to adversarial imagery? Neural nets are well known for being easily fooled [1] ... I wonder if you could create similar situations for self-driving cars. [1] http://www.evolvingai.org/fooling http://www.evolvingai.org/fooling
lambdaelite 11y ago
Toyota couldn't ignore these (voluntary) standards given that MISRA-C nor the earlier DO-178 didn't exist at the time the code was developed.

derefr 11y ago

Self-driving cars seem less scary to me. You can take unreliable components (like, say, a car with crappy firmware) and make a reliable system out of it if the software running on top is fault-tolerant. The "autopilot", unlike a human, will pull over the moment the car's brakes begin to fail, etc.