4 ms·

Does it matter? Once you’re on public roads, you need to ALWAYS fail-safe. And that means not blocking the road/intersections when something unexpected happens

by MBCook 10mo ago

Does it matter? Once you’re on public roads, you need to ALWAYS fail-safe. And that means not blocking the road/intersections when something unexpected happens. If you can physically get out of the way, you need to. Period.

JumpCrisscross 10mo ago
> Does it matter Yes. OP is inferring Waymo's internal processes from this meltdown. ("Makes me think there are likely other obvious use cases they haven’t thought about proactively either.") If Waymo literally didn't foresee a blackout, that's a systemic problem. If, on the other hand, there was some weird power and cellular meltdown that coïncided with something else, that's a fixable edge case.
- MBCook 10mo ago
  >If Waymo literally didn't foresee a blackout, that's a systemic problem. I agree with this bit > If, on the other hand, there was some weird power and cellular meltdown that coïncided with something else, that's a fixable edge case. This is what I have a problem with. That’s not an edge case. There will always be a weird thing no one programmed for. Remember a few years ago when a semi truck overturned somewhere and poured slimy eels all over the highway? No one‘s ever gonna program for that. It doesn’t matter. There has to be an absolute minimum fail safe that can always work if the car is capable of moving safely. The fact that a human driver couldn’t be reached to press a button to say to execute that is not acceptable. Not having the human available is a totally foreseeable problem. It’s Google. They know networks fail.
  - cgriswald 10mo ago
    This isn't to disagree with your overall point about proper emergency mitigation and having humans available. > Remember a few years ago when a semi truck overturned somewhere and poured slimy eels all over the highway? No one‘s ever gonna program for that. While the cause is unusual, this is really just three things that everyone absolutely should be programming into their autonomous vehicles: accidents, road debris, and slick conditions.
    - MBCook 10mo ago
      Certainly. That one was interesting both because of the odd specifics of it and because it made the road more slippery than any normal accident where just a bunch of boxes of random dry goods fell over. It just happens to make a fantastic example of “thing no one is ever going to foresee“. If there wasn’t footage how many people would even believe it happened?
- andsoitis 10mo ago
  > > Does it matter > Yes. OP is inferring Waymo's internal processes from this meltdown. ("Makes me think there are likely other obvious use cases they haven’t thought about proactively either.") No, I'm not inferring internal processes. I'm guessing level of critical thinking. When you are creating autonomous vehicles, one of the things that you want to risk assess and have mitigation for is what you want the vehicles to do in case the systems they depend on fail (e.g. electricity, comms). Now, it could be that the team has anticipated those things but some other failure in their systems have caused vehicles to stop in the middle of intersections, blocking traffic (as per article). I'm super curious to learn more about what Waymo encountered and how they plan to up their game.
  - JumpCrisscross 10mo ago
    > I'm not inferring internal processes…I'm guessing level of critical thinking Genuine question: how do these differ? Isn’t the level of critical thinking of Waymo’s employees internal to it? (What’s the mens rea analogue for a company?)
    - andsoitis 10mo ago
      Are you asking what's the difference between critical thinking and an organizational process?
      - JumpCrisscross 10mo ago
        No, though now I’m reëvaluating the validity of the rest of your comment.
- lubujackson 10mo ago
  The "coinciding problems" should be an assumption, not a edge case we reason away. Because black swan events are always going to have cascading issues - a big earthquake means lights out AND cell towers overloaded or out, not to mention debris in streets, etc. What they need is a "shit is fucked fallback" that cedes control. Maybe there is a special bluetooth command any police or ambulance can send if nearby, like clear the intersection/road. Or maybe the doors just unlock and any human can physically enter and drive the car up to X distance. To techies and lawyers it may sound impossible, but for normal humans, that certainly sounds better. Like that Mitch Hedberg joke, when an escalator is out of order it becomes stairs. When a Waymo breaks it should become a car.
  - MBCook 10mo ago
    > Or maybe the doors just unlock and any human can physically enter and drive the car up to X distance. Do the even have physical controls to do that at this point? I’ve never been in one so I don’t know how different they are from normal cars today.
    - jerlam 10mo ago
      The Waymos still have all their normal driver controls. There is a process where law enforcement can enter the vehicle, call Waymo and verify their law enforcement status, and then switch the vehicle into manual mode and drive it as normal. Here is their instructions for law enforcement in the Waymo Emergency Response Guide: https://storage.googleapis.com/waymo-uploads/files/first%20responder/Jaguar%20I-PACE%20California%20Emergency%20Response%20Guide%20and%20Law%20Enforcement%20Interaction%20Protocol%20March%202025.pdf#%5B%7B%22num%22%3A55%2C%22gen%22%3A0%7D%2C%7B%22name%22%3A%22XYZ%22%7D%2Cnull%2Cnull%2C0%5D https://storage.googleapis.com/waymo-uploads/files/first%20r...
      - MBCook 10mo ago
        Ok. Thanks. I must have been thinking of something else. Didn’t Google have little self-driving vehicles without controls that were limited to pre-planned routes on non-public roads on their campus? Obviously a hugely different problem domain.
scoofy 10mo ago
A fail-safe is EXACTLY blocking roads at intersections without power, not proceeding through intersections without power. It's much safer to be stopped than to keep going. I honestly wish the humans driving through blacked out intersections without slowing down in my neighborhood last night actually understood this.
- empressplay 10mo ago
  An intersection without power is just a 4-way stop.
  - scoofy 10mo ago
    An intersection without power is supposed to be treated as a 4-way stop. An unfortunately high, nontrivial number of drivers last night were not following that rule.
    - MBCook 10mo ago
      And yet the humans managed. Even at a normal four-way stop with stop signs people sometimes blow through it. The Waymo has to handle it. That’s part of driving. It can creep through at 3 miles an hour if it thinks that’s what’s safe. All it has to do is get out of the intersection.
      - scoofy 10mo ago
        The outrage people would rightly have at Waymo allowing a number of its vehicles to blow the lights would be huge. People running blacked out lights is unacceptable.
        MBCook 10mo ago
        Who said “blow through”? Waymos know how to handle 4 way stops.
        10mo ago
addaon 10mo ago
> Once you’re on public roads, you need to ALWAYS fail-safe. Yes. > And that means not blocking the road/intersections when something unexpected happens. No. Fail-operational is not the only allowable fail-safe condition for automobiles. For example, it is acceptable for loss of propulsion to cause stop-in-lane — the alternative would be to require high-availability propulsion systems, or to require drivers to always have enough kinetic energy to coast to side. This just isn’t the case. One can argue that when operating a fleet with correlated failure modes the rules should change a bit, but that’s a separate topic.

by MBCook 10mo ago

JumpCrisscross 10mo ago

> Does it matter Yes. OP is inferring Waymo's internal processes from this meltdown. ("Makes me think there are likely other obvious use cases they haven’t thought about proactively either.") If Waymo literally didn't foresee a blackout, that's a systemic problem. If, on the other hand, there was some weird power and cellular meltdown that coïncided with something else, that's a fixable edge case.

MBCook 10mo ago
>If Waymo literally didn't foresee a blackout, that's a systemic problem. I agree with this bit > If, on the other hand, there was some weird power and cellular meltdown that coïncided with something else, that's a fixable edge case. This is what I have a problem with. That’s not an edge case. There will always be a weird thing no one programmed for. Remember a few years ago when a semi truck overturned somewhere and poured slimy eels all over the highway? No one‘s ever gonna program for that. It doesn’t matter. There has to be an absolute minimum fail safe that can always work if the car is capable of moving safely. The fact that a human driver couldn’t be reached to press a button to say to execute that is not acceptable. Not having the human available is a totally foreseeable problem. It’s Google. They know networks fail.
- cgriswald 10mo ago
  This isn't to disagree with your overall point about proper emergency mitigation and having humans available. > Remember a few years ago when a semi truck overturned somewhere and poured slimy eels all over the highway? No one‘s ever gonna program for that. While the cause is unusual, this is really just three things that everyone absolutely should be programming into their autonomous vehicles: accidents, road debris, and slick conditions.
  - MBCook 10mo ago
    Certainly. That one was interesting both because of the odd specifics of it and because it made the road more slippery than any normal accident where just a bunch of boxes of random dry goods fell over. It just happens to make a fantastic example of “thing no one is ever going to foresee“. If there wasn’t footage how many people would even believe it happened?
andsoitis 10mo ago
> > Does it matter > Yes. OP is inferring Waymo's internal processes from this meltdown. ("Makes me think there are likely other obvious use cases they haven’t thought about proactively either.") No, I'm not inferring internal processes. I'm guessing level of critical thinking. When you are creating autonomous vehicles, one of the things that you want to risk assess and have mitigation for is what you want the vehicles to do in case the systems they depend on fail (e.g. electricity, comms). Now, it could be that the team has anticipated those things but some other failure in their systems have caused vehicles to stop in the middle of intersections, blocking traffic (as per article). I'm super curious to learn more about what Waymo encountered and how they plan to up their game.
- JumpCrisscross 10mo ago
  > I'm not inferring internal processes…I'm guessing level of critical thinking Genuine question: how do these differ? Isn’t the level of critical thinking of Waymo’s employees internal to it? (What’s the mens rea analogue for a company?)
  - andsoitis 10mo ago
    Are you asking what's the difference between critical thinking and an organizational process?
    - JumpCrisscross 10mo ago
      No, though now I’m reëvaluating the validity of the rest of your comment.
lubujackson 10mo ago
The "coinciding problems" should be an assumption, not a edge case we reason away. Because black swan events are always going to have cascading issues - a big earthquake means lights out AND cell towers overloaded or out, not to mention debris in streets, etc. What they need is a "shit is fucked fallback" that cedes control. Maybe there is a special bluetooth command any police or ambulance can send if nearby, like clear the intersection/road. Or maybe the doors just unlock and any human can physically enter and drive the car up to X distance. To techies and lawyers it may sound impossible, but for normal humans, that certainly sounds better. Like that Mitch Hedberg joke, when an escalator is out of order it becomes stairs. When a Waymo breaks it should become a car.
- MBCook 10mo ago
  > Or maybe the doors just unlock and any human can physically enter and drive the car up to X distance. Do the even have physical controls to do that at this point? I’ve never been in one so I don’t know how different they are from normal cars today.
  - jerlam 10mo ago
    The Waymos still have all their normal driver controls. There is a process where law enforcement can enter the vehicle, call Waymo and verify their law enforcement status, and then switch the vehicle into manual mode and drive it as normal. Here is their instructions for law enforcement in the Waymo Emergency Response Guide: https://storage.googleapis.com/waymo-uploads/files/first%20responder/Jaguar%20I-PACE%20California%20Emergency%20Response%20Guide%20and%20Law%20Enforcement%20Interaction%20Protocol%20March%202025.pdf#%5B%7B%22num%22%3A55%2C%22gen%22%3A0%7D%2C%7B%22name%22%3A%22XYZ%22%7D%2Cnull%2Cnull%2C0%5D https://storage.googleapis.com/waymo-uploads/files/first%20r...
    - MBCook 10mo ago
      Ok. Thanks. I must have been thinking of something else. Didn’t Google have little self-driving vehicles without controls that were limited to pre-planned routes on non-public roads on their campus? Obviously a hugely different problem domain.

scoofy 10mo ago

A fail-safe is EXACTLY blocking roads at intersections without power, not proceeding through intersections without power. It's much safer to be stopped than to keep going. I honestly wish the humans driving through blacked out intersections without slowing down in my neighborhood last night actually understood this.

empressplay 10mo ago
An intersection without power is just a 4-way stop.
- scoofy 10mo ago
  An intersection without power is supposed to be treated as a 4-way stop. An unfortunately high, nontrivial number of drivers last night were not following that rule.
  - MBCook 10mo ago
    And yet the humans managed. Even at a normal four-way stop with stop signs people sometimes blow through it. The Waymo has to handle it. That’s part of driving. It can creep through at 3 miles an hour if it thinks that’s what’s safe. All it has to do is get out of the intersection.
    - scoofy 10mo ago
      The outrage people would rightly have at Waymo allowing a number of its vehicles to blow the lights would be huge. People running blacked out lights is unacceptable.
      - MBCook 10mo ago
        Who said “blow through”? Waymos know how to handle 4 way stops.
        10mo ago

addaon 10mo ago

> Once you’re on public roads, you need to ALWAYS fail-safe. Yes. > And that means not blocking the road/intersections when something unexpected happens. No. Fail-operational is not the only allowable fail-safe condition for automobiles. For example, it is acceptable for loss of propulsion to cause stop-in-lane — the alternative would be to require high-availability propulsion systems, or to require drivers to always have enough kinetic energy to coast to side. This just isn’t the case. One can argue that when operating a fleet with correlated failure modes the rules should change a bit, but that’s a separate topic.