One way to avoid these types of crashes IMO is anomaly detection. It's quite simple to do anomaly detection in pixels using modern deep pixel prediction nets li

One way to avoid these types of crashes IMO is anomaly detection. It's quite simple to do anomaly detection in pixels using modern deep pixel prediction nets like PredNet. In my experiments you get a few seconds lead time on something like a car cutting you off (the car starts to head out of the lane before actually crossing it for example). This allows alerting the driver, and with a full windshield HUD you could even highlight the anomalous pixels on the windshield. The nice thing about this is that it can be trained in an unsupervised manner on all the available data. Some important details are to find anomalies in object bounding boxes, using something like Tensorflow's object detection pretrained net. Otherwise buildings with lots of striations would light up the anomaly detector. Also, you should detect anomalies in a human colorspace like CIELAB so that white cars (#fff) are not artificially weighted as more anomalous. Finally, you could use this as input to a planner like Model Predictive Control where a higher cost is incurred for approaching anomalous objects.