4 ms·

I really hate to be that guy, but I think my biggest worry is how each of these car manufacturers will handle such edge cases. Each car manufacturers autonomou

I really hate to be that guy, but I think my biggest worry is how each of these car manufacturers will handle such edge cases. Each car manufacturers autonomous driving is a black box, we don't know whether said edge case is handled or not, or what action the car will take. Maybe now's not the time but it would be nice to see a deterministic rule book that says, for this edge case, the system will react this way. We can't account for all edge cases but some default fall back would be nice.

swiley 9y ago
I still like the argument that critical firmware should be open sourced by law. You're required to be given the chance to read contracts before you sign them, I don't see why software should be different.
- sqeaky 9y ago
  Why is software like this allowed to be closed? We don't allow our plumbing, structure wiring, welding, or any other physical construction method to be closed away and hidden from view. Rather we demand these things be inspected, certified, documented and generally adhere to demonstrable codes and rules. Keeping you source hidden is saying "I have something to hide". Why should someone have something to hide in Pacemaker software, software driving vehicles or safety system on nuclear power plants or anything else where life could be lost?
  - hanbura 9y ago
    I don't think I have any right to see the blueprint of every building I enter. Ensured certification is very different from open source
    - sqeaky 9y ago
      I think in some municipalities you can walk down to the planning and get most the blueprints you would like. Where you can't it is partially because the analogy breaks down physical is slightly reduced by openness, its allows attackers of all kinds to plan better. More manpower can be focused to overcome and issue and a map is force multiplier. Security in information systems is strictly increased, because the flaws exist open or close, but when exposed in the open they are closed forever to all attacker regardless of manpower.
  - bluGill 9y ago
    We also have a concept of how to inspect plumbing. I could put all rules on a checklist and teach you how to check the plumbing in your house. (I'd have to look the rules up, but conceptually I could). Of course houses are all the same: a few toilets, sinks, tubs and showers. The differences are easy to handle in the rules. You could imagine something different: instead of water we have a special hand cleaner in this sink. My checklists cover the potable water system and the non-potable water drain system, this special cleaning fluid isn't covered and your cannot inspect it. We have a simple exception for those cases: a professional engineer can legally stamp it saying "I have thought of everything, this will work if installed according to these specs." Then the checklist is make sure that everything is up to those specs. I honestly do not know how to create a checklist for source code. Sure I can create some starts, but https://en.wikipedia.org/wiki/Underhanded_C_Contest https://en.wikipedia.org/wiki/Underhanded_C_Contest needs to be considered: can someone pass your checklist and still win?
- hanbura 9y ago
  How is software anything like a contract? The two seem completely unrelated to me
westoque 9y ago
Maybe we need an open source protocol that has all the standards and edge cases handled. I think competitiveness is a good thing and will drive innovation. But in this specific field, with regards to safety and security, it needs to be out in the open.
- AndrewKemendo 9y ago
  That's what Comma.ai was supposed to do, but there is no money in it apparently.
  - manarth 9y ago
    Comma.ai wasn't originally going to be open-source, Comma.ai intended was going to be a $999 proprietary retro-fit product. NHTSA (National Highway Traffic Safety Administration) asked Hotz to delay the launch on safety grounds. He then open-sourced the software specifically to avoid regulation and safety standards. "NHTSA only regulates physical products that are sold,” Hotz said. “They do not regulate open source software, which is a whole lot more like speech.” [1] https://qz.com/849669/to-dodge-government-regulation-self-driving-car-startup-comma-ai-released-all-their-code-for-free/ https://qz.com/849669/to-dodge-government-regulation-self-dr... [2] https://www.theverge.com/2016/11/30/13779336/comma-ai-autopilot-canceled-autonomous-car-software-free https://www.theverge.com/2016/11/30/13779336/comma-ai-autopi...
- at-fates-hands 9y ago
  >> But in this specific field, with regards to safety and security, it needs to be out in the open. This is one of the horrors of capitalism. If there's money to be made, you can bet that if someone creates a proprietary protocol, it won't be released and will force other companies to develop their own technology to compete. Yes, it will drive competition, but I fear that it will spawn a host of different, competing technologies and make the whole industry awash in proprietary technology. In most of those cases, the people who lose are the consumers.
  - manarth 9y ago
justin66 9y ago
IIHS and NHTSA will develop lists of oddities like this that they will test, I'm sure.

swiley 9y ago

I still like the argument that critical firmware should be open sourced by law. You're required to be given the chance to read contracts before you sign them, I don't see why software should be different.

sqeaky 9y ago
Why is software like this allowed to be closed? We don't allow our plumbing, structure wiring, welding, or any other physical construction method to be closed away and hidden from view. Rather we demand these things be inspected, certified, documented and generally adhere to demonstrable codes and rules. Keeping you source hidden is saying "I have something to hide". Why should someone have something to hide in Pacemaker software, software driving vehicles or safety system on nuclear power plants or anything else where life could be lost?
- hanbura 9y ago
  I don't think I have any right to see the blueprint of every building I enter. Ensured certification is very different from open source
  - sqeaky 9y ago
    I think in some municipalities you can walk down to the planning and get most the blueprints you would like. Where you can't it is partially because the analogy breaks down physical is slightly reduced by openness, its allows attackers of all kinds to plan better. More manpower can be focused to overcome and issue and a map is force multiplier. Security in information systems is strictly increased, because the flaws exist open or close, but when exposed in the open they are closed forever to all attacker regardless of manpower.
- bluGill 9y ago
  We also have a concept of how to inspect plumbing. I could put all rules on a checklist and teach you how to check the plumbing in your house. (I'd have to look the rules up, but conceptually I could). Of course houses are all the same: a few toilets, sinks, tubs and showers. The differences are easy to handle in the rules. You could imagine something different: instead of water we have a special hand cleaner in this sink. My checklists cover the potable water system and the non-potable water drain system, this special cleaning fluid isn't covered and your cannot inspect it. We have a simple exception for those cases: a professional engineer can legally stamp it saying "I have thought of everything, this will work if installed according to these specs." Then the checklist is make sure that everything is up to those specs. I honestly do not know how to create a checklist for source code. Sure I can create some starts, but https://en.wikipedia.org/wiki/Underhanded_C_Contest https://en.wikipedia.org/wiki/Underhanded_C_Contest needs to be considered: can someone pass your checklist and still win?
hanbura 9y ago
How is software anything like a contract? The two seem completely unrelated to me

westoque 9y ago

Maybe we need an open source protocol that has all the standards and edge cases handled. I think competitiveness is a good thing and will drive innovation. But in this specific field, with regards to safety and security, it needs to be out in the open.

AndrewKemendo 9y ago
That's what Comma.ai was supposed to do, but there is no money in it apparently.
- manarth 9y ago
  Comma.ai wasn't originally going to be open-source, Comma.ai intended was going to be a $999 proprietary retro-fit product. NHTSA (National Highway Traffic Safety Administration) asked Hotz to delay the launch on safety grounds. He then open-sourced the software specifically to avoid regulation and safety standards. "NHTSA only regulates physical products that are sold,” Hotz said. “They do not regulate open source software, which is a whole lot more like speech.” [1] https://qz.com/849669/to-dodge-government-regulation-self-driving-car-startup-comma-ai-released-all-their-code-for-free/ https://qz.com/849669/to-dodge-government-regulation-self-dr... [2] https://www.theverge.com/2016/11/30/13779336/comma-ai-autopilot-canceled-autonomous-car-software-free https://www.theverge.com/2016/11/30/13779336/comma-ai-autopi...
at-fates-hands 9y ago
>> But in this specific field, with regards to safety and security, it needs to be out in the open. This is one of the horrors of capitalism. If there's money to be made, you can bet that if someone creates a proprietary protocol, it won't be released and will force other companies to develop their own technology to compete. Yes, it will drive competition, but I fear that it will spawn a host of different, competing technologies and make the whole industry awash in proprietary technology. In most of those cases, the people who lose are the consumers.
- manarth 9y ago

justin66 9y ago

IIHS and NHTSA will develop lists of oddities like this that they will test, I'm sure.