5 ms·

>This looks like Cruise's fault Imagine having to write the code that "detects" if a pedestrain is caught underneath the car

mumblemumble 3y ago
It's not clear to me that this is purely a software problem; it might be a "the car doesn't even have the sensors necessary to detect this kind of situation" problem. It could also be a "the software can't anticipate that this would be happening the way a human can because it doesn't have object permanence" problem. That's probably at least part of how a human would do it. But I have a suspicion that we're very, very far away from being able to get computers to do that in any sort of reliable way. So I'm guessing we're probably looking at specialized sensors if we want to handle this kind of situation appropriately. Basically some sort of self-driving car equivalent of a human driver getting out of the car to check and make sure everything's OK.
- marcosdumay 3y ago
  I would be surprised if it turns out that those self-driving cars don't have object permanence. Making things move around without a physical model looks like a really nasty infinite bug-breeder.
  - FireBeyond 3y ago
    For the longest time, at least the visual representation of trains at a level crossing strongly implied challenges around object permanence for Tesla. I will say that I recognize that the visual representation may not be a 1:1 (in fact quite certainly isn't) mapping of the computer's "world view". But it was quite ridiculous and definitely disconcerting seeing a level crossing with a train going through it and seeing the Tesla I was in acting like there was a traffic light that was erratically going red, while dozens of semi trucks flickered in and out of existence across the screen in front of me.
    - marcosdumay 3y ago
      Well, ok, in hindsight I have to make an exception for Tesla here. They are famously flickery, and I do expect them to have a bug after the other. But I really don't expect that from the cars that go around without a driver.
      - phatskat 3y ago
        I’m guessing because Tesla uses cameras and everyone else uses LiDAR, right?
        marcosdumay 3y ago
        Because Tesla is not self-driving, and thus their designers have no need to solve those bugs.
        phatskat 3y ago
        They sure aren’t, Musk just keeps saying “this year”, and has been saying it for like a decade?
    - mike_d 3y ago
      > strongly implied challenges around object permanence for Tesla Anyone who has ever ridden in a Tesla knows they have no real perception of the world around them. Even just driving down the road they show icons of the things around them, which will randomly appear and disappear or change from busses to pedestrians to cars.
  - mumblemumble 3y ago
    And yet a common thread in post-mortems on so many self-driving car collisions boils down to, "the car doesn't have object permanence." It may have some ability to track specific things in specific ways in a manner that's hard-coded by engineers. But if that's the same thing as an actual mental model of the world then that implies that we've had a significant piece of AGI in the form of many video games for about half a century now and just not known it.
- PH95VuimJjqBqy 3y ago
  yeah, another poster mentioned they would have heard the screaming. audio is difficult because a human can differentiate a woman screaming from the sounds of construction, etc. And also can tell that it's someone in shock vs someone in pain. I'm unsure if audio processing is good enough for that yet, but imagine if someone were to try and you could get these vehicles to stop by screaming near them.
  - kaibee 3y ago
    An ML model to differentiate human-in-pain screaming from other noise sounds like scifi-satire.
spaced-out 3y ago
That's not hard for me to imagine. Now that I think about it, it should probably be standard for autonomous cars -- if there's an accident or something unexpected it should pull over, unless it can't reach the side of the road without hitting something or someone is trapped under the car.
- phatskat 3y ago
  > it should pull over, unless it can't reach the side of the road without hitting something or someone is trapped under the car. Or…insert how many other edge cases? There are plenty of scenarios where it shouldn’t pull over, aren’t there? Training the myriad unique situations and proper responses just feels untenable, at least with current technology.
b112 3y ago
Imagine being told not to, because other code has a higher priority.
riscy 3y ago
or rather, collect training data on how to best handle running a pedestrian over.
- PH95VuimJjqBqy 3y ago
  I'm not gonna lie, I got a good laugh out of your comment.
KennyBlanken 3y ago
Or you can not code your autonomous vehicle to try to move after getting into a collision, because if someone is caught under the car, you shouldn't try to move the car anyway. It's hilarious that the fucking things park themselves if you put a traffic cone on the hood, but if they get into a crash, especially one with a pedestrian? BEEP BOOP MUST MOVE!
TulliusCicero 3y ago
I guarantee you every serious self driving car company (at least in the US) was looking at exactly this scenario after this happened with Cruise.
- bluGill 3y ago
  That is the beauty of technology: once we identify a problem we can do something about it and apply the fix to everything. While this is the type of case you can argue nobody would have expected before, once it happens you know and can setup lab situations where similar things happen and then figure out how best to respond over months of effort.
  - aeturnum 3y ago
    This is absolutely not a case nobody would have expected before? People end up under cars in accidents regularly. Imo any company that "found out" about this scenario through the cruise incident is guilty of gross negligence.
    - TulliusCicero 3y ago
      I'm sure all the bigger ones had looked at the possibility before, but it's only natural to revisit it when something horrible like this happens to a competitor.
      - aeturnum 3y ago
        I am sure that you don't generally start with this scenario - but I have to say again that this is a common thing. People end up under cars all the time (in accidents). There is just no way anyone should excuse any company for "not considering" this scenario if their cars are on roads. I'm sure it's hard! Other industries recognize corner cases where they need to design to avoid really ugly long tail outcomes and do so with ethical rigor. Most planes do not crash into the ocean and yet you are educated on how to escape in a water landing at the start of every flight.
planb 3y ago
Code that identifies if anything is stuck under the car would be useful for a lot of cases but especially if an accident already has happended. Besides that, hard edge cases will always exist and can only be solved by a human (or an AGI?) in the loop. But they are't too common, so just detecting that the situation is not solvable by the algorithm, doing nothing and waiting for a remote operator should scale well enough.

mumblemumble 3y ago

It's not clear to me that this is purely a software problem; it might be a "the car doesn't even have the sensors necessary to detect this kind of situation" problem. It could also be a "the software can't anticipate that this would be happening the way a human can because it doesn't have object permanence" problem. That's probably at least part of how a human would do it. But I have a suspicion that we're very, very far away from being able to get computers to do that in any sort of reliable way. So I'm guessing we're probably looking at specialized sensors if we want to handle this kind of situation appropriately. Basically some sort of self-driving car equivalent of a human driver getting out of the car to check and make sure everything's OK.

marcosdumay 3y ago
I would be surprised if it turns out that those self-driving cars don't have object permanence. Making things move around without a physical model looks like a really nasty infinite bug-breeder.
- FireBeyond 3y ago
  For the longest time, at least the visual representation of trains at a level crossing strongly implied challenges around object permanence for Tesla. I will say that I recognize that the visual representation may not be a 1:1 (in fact quite certainly isn't) mapping of the computer's "world view". But it was quite ridiculous and definitely disconcerting seeing a level crossing with a train going through it and seeing the Tesla I was in acting like there was a traffic light that was erratically going red, while dozens of semi trucks flickered in and out of existence across the screen in front of me.
  - marcosdumay 3y ago
    Well, ok, in hindsight I have to make an exception for Tesla here. They are famously flickery, and I do expect them to have a bug after the other. But I really don't expect that from the cars that go around without a driver.
    - phatskat 3y ago
      I’m guessing because Tesla uses cameras and everyone else uses LiDAR, right?
      - marcosdumay 3y ago
        Because Tesla is not self-driving, and thus their designers have no need to solve those bugs.
        phatskat 3y ago
        They sure aren’t, Musk just keeps saying “this year”, and has been saying it for like a decade?
  - mike_d 3y ago
    > strongly implied challenges around object permanence for Tesla Anyone who has ever ridden in a Tesla knows they have no real perception of the world around them. Even just driving down the road they show icons of the things around them, which will randomly appear and disappear or change from busses to pedestrians to cars.
- mumblemumble 3y ago
  And yet a common thread in post-mortems on so many self-driving car collisions boils down to, "the car doesn't have object permanence." It may have some ability to track specific things in specific ways in a manner that's hard-coded by engineers. But if that's the same thing as an actual mental model of the world then that implies that we've had a significant piece of AGI in the form of many video games for about half a century now and just not known it.
PH95VuimJjqBqy 3y ago
yeah, another poster mentioned they would have heard the screaming. audio is difficult because a human can differentiate a woman screaming from the sounds of construction, etc. And also can tell that it's someone in shock vs someone in pain. I'm unsure if audio processing is good enough for that yet, but imagine if someone were to try and you could get these vehicles to stop by screaming near them.
- kaibee 3y ago
  An ML model to differentiate human-in-pain screaming from other noise sounds like scifi-satire.

spaced-out 3y ago

That's not hard for me to imagine. Now that I think about it, it should probably be standard for autonomous cars -- if there's an accident or something unexpected it should pull over, unless it can't reach the side of the road without hitting something or someone is trapped under the car.

phatskat 3y ago
> it should pull over, unless it can't reach the side of the road without hitting something or someone is trapped under the car. Or…insert how many other edge cases? There are plenty of scenarios where it shouldn’t pull over, aren’t there? Training the myriad unique situations and proper responses just feels untenable, at least with current technology.

b112 3y ago

Imagine being told not to, because other code has a higher priority.

riscy 3y ago

or rather, collect training data on how to best handle running a pedestrian over.

PH95VuimJjqBqy 3y ago
I'm not gonna lie, I got a good laugh out of your comment.

KennyBlanken 3y ago

Or you can not code your autonomous vehicle to try to move after getting into a collision, because if someone is caught under the car, you shouldn't try to move the car anyway. It's hilarious that the fucking things park themselves if you put a traffic cone on the hood, but if they get into a crash, especially one with a pedestrian? BEEP BOOP MUST MOVE!

TulliusCicero 3y ago

I guarantee you every serious self driving car company (at least in the US) was looking at exactly this scenario after this happened with Cruise.

bluGill 3y ago
That is the beauty of technology: once we identify a problem we can do something about it and apply the fix to everything. While this is the type of case you can argue nobody would have expected before, once it happens you know and can setup lab situations where similar things happen and then figure out how best to respond over months of effort.
- aeturnum 3y ago
  This is absolutely not a case nobody would have expected before? People end up under cars in accidents regularly. Imo any company that "found out" about this scenario through the cruise incident is guilty of gross negligence.
  - TulliusCicero 3y ago
    I'm sure all the bigger ones had looked at the possibility before, but it's only natural to revisit it when something horrible like this happens to a competitor.
    - aeturnum 3y ago
      I am sure that you don't generally start with this scenario - but I have to say again that this is a common thing. People end up under cars all the time (in accidents). There is just no way anyone should excuse any company for "not considering" this scenario if their cars are on roads. I'm sure it's hard! Other industries recognize corner cases where they need to design to avoid really ugly long tail outcomes and do so with ethical rigor. Most planes do not crash into the ocean and yet you are educated on how to escape in a water landing at the start of every flight.

planb 3y ago

Code that identifies if anything is stuck under the car would be useful for a lot of cases but especially if an accident already has happended. Besides that, hard edge cases will always exist and can only be solved by a human (or an AGI?) in the loop. But they are't too common, so just detecting that the situation is not solvable by the algorithm, doing nothing and waiting for a remote operator should scale well enough.