3 ms·

In a recently reported issue with an Apple Car almost running over a pedestrian, someone on HN asked why can't we simply add the code: if (personInFrontOfC

by acoard 4y ago

In a recently reported issue with an Apple Car almost running over a pedestrian, someone on HN asked why can't we simply add the code: if (personInFrontOfCar) { halt(); } The reason, of course, is the self-driving algorithms are all done with machine learning. The model is trained with a corpus of information and the result is a "black box." The programmer could not insert the above `if` statement if they wanted to, they could not modify the source code of the model as if it was normal business logic. Instead, they would have to re-train the model on new data. This is what people mean when they say it is a "black box." There is no source code to edit, only training data that outputs an as-is model. It's similar to getting binary blobs, for eg Linux drivers. In contrast, I could fork regex if I wanted and add an `if` statement in the middle of it, and re-compile it. It is pure source code. Same with grep.

MathYouF 4y ago
From my admittedly limited knowledge of how self driving policy models are implemented, I believe the main black box AI is determining the part `if personInFrontOfCar:` (using object detection, masking, 3D pose estimation, et/or cetera) And actually defined policies on how to handle certain situations given that inference are possible, in the case specifically of self driving vehicles. My assumptions come from when I'd asked someone at NeurIPS in 2019 about what reinforcement learning methods they use as they said "I don't think any self driving companies are using reinforcement learning, it is too risky". I don't mean to imply this is hard to find information either, reading a few papers would be all it takes to clear up the degree to which most self driving policies are "controllable" in the way you describe, or at least to what degree. My main point is that I think it is the inferring of the environment (current state) rather than the chosen policy at each time step which is more of an error prone black box.

by acoard 4y ago

MathYouF 4y ago

From my admittedly limited knowledge of how self driving policy models are implemented, I believe the main black box AI is determining the part `if personInFrontOfCar:` (using object detection, masking, 3D pose estimation, et/or cetera) And actually defined policies on how to handle certain situations given that inference are possible, in the case specifically of self driving vehicles. My assumptions come from when I'd asked someone at NeurIPS in 2019 about what reinforcement learning methods they use as they said "I don't think any self driving companies are using reinforcement learning, it is too risky". I don't mean to imply this is hard to find information either, reading a few papers would be all it takes to clear up the degree to which most self driving policies are "controllable" in the way you describe, or at least to what degree. My main point is that I think it is the inferring of the environment (current state) rather than the chosen policy at each time step which is more of an error prone black box.