2 ms·
I'd pessimistically imagine the real commit trees of actual SDC companies then are no different, given the "move fast and break things" ethos many of these comp
by Hydraulix989 6y ago
I'd pessimistically imagine the real commit trees of actual SDC companies then are no different, given the "move fast and break things" ethos many of these companies have.
Meanwhile, taking these extra rote precautions (e.g. cut down on global state, throw unit tests on everything, don't use go-to's) has no real effect when the elephant in the room is that even correctly-implemented state-of-the-art machine learning models only have classification accuracies in the mid 90%s at best.
That's (relatively) fine if you're deciding what ad to show to a teenager using TikTok. When we are talking about driving a car, that loosely means having an accident every 100 miles. Based on publicly available disengagement data, this corroborates (only Waymo has surpassed the 1000 mile mark, GM might have recently, too).
Meanwhile, verification in this domain only means replaying actual recorded sensor data in a "simulation" using one-off example scenarios, not doing any kind of formal analysis or exhaustive proofs -- which simply don't exist yet for ML. There's no real way currently to take an arbitrary mis-classified example and determine its root cause -- or even correct it within the existing model without reducing accuracy elsewhere!
It's literally impossible to avoid being unsafe, so code style rules and procedures amount to mere hoop-jumping
- bri3d 6y agoI agree with you overall - in my post, I was trying to accept the premise of the overall safety model: "the ML will break but as long as the driver is paying attention it's OK." This was out of an effort to be fair to Comma, as, to your point, this is the overall state of the industry. I disagree, however that code style rules and procedures amount to hoop-jumping, as not every failure mode is the same (especially if you do accept the driver-attention / driver-in-control premise). An unexpected failure at the control layer (especially the control override layer) is a different class of failure to an unexpected direction command from the ML model, for example.