Kind of relevant to this is the NTSB analysis of a self-driving crash in 2017: https://www.ntsb.gov/investigations/accidentreports/reports/hab1906.pdf https://

Kind of relevant to this is the NTSB analysis of a self-driving crash in 2017: https://www.ntsb.gov/investigations/accidentreports/reports/hab1906.pdf https://www.ntsb.gov/investigations/accidentreports/reports/... Basically a truck was backing up into an alley - it was at an angle when the self-driving vehicle approached, but a little kid would have been able to figure out that it needed to straighten before it finished backing in. The self-driving vehicle didn't understand this, and stopped at a "safe distance" which happened to be within the arc that the truck cab had to sweep in order to finish its maneuver. It's quite possible that LLM-like models could learn things like this, but we don't have vast amounts of easily accessible training data, because everyone just knows this sort of shit, and we don't have good vocabulary for it - we just say "look at that" or the equivalent. (I'll add that I'm sure a lot of knowledge like this is encoded in the physics engines of various games, but I doubt we have a good way to link that sort of procedural code knowledge to the symbolic knowledge in LLMs)