10 ms·

SF cops, firefighters vent prior to big vote on driverless car expansion

33 points by wpietri 3y ago

wpietri 3y ago
I submitted this because there are a lot of good examples of how driving is not just a process of spotting objects and following lanes, but a social process of interacting with people. I came across another example a couple of days ago. A guy was telling a story about he was out walking his two bigger dogs in the evening. Across the street, a tiny dog started barking angrily. The tiny dog got away from its owner started to cross the street, heading into traffic. So the guy with the big dogs stepped a bit into the street, waving to attract the attention of an oncoming driver who he thought might not see the small dog ready to run under the wheels. I have no idea what a robot car might make of that, but I certainly wouldn't want to be the one who has to write the sort of social processing that determines the guy is trying to signal something, so we should stop until we figure out what the signal means.
- x86x87 3y ago
  You get a robodog that is replaceable. If your dog gets totalled you just download it into a new body at home. You're also going to probably need robodog insurance since restoring the robodog ain't cheap /s
- idopmstuff 3y ago
  > so we should stop until we figure out what the signal means. We should determine how to handle driverless cars based on their overall safety, not edge cases. If driverless cars have significantly fewer injuries/accidents/etc. per mile driven than human drivers, we should push to expand them. If they only drive better in certain areas, we should geofence them. If they're safer than human drivers but would hit the guy with the dogs in this case, we should still allow them. The opportunity cost of the other accidents that would've occurred if self-driving cars weren't allowed very much matters. Plus the guy with the dogs shouldn't be stepping out into the street (and the guy with the small dog should be able to maintain control of a small dog). Also, in a world of self-driving cars, the guy doesn't have to try to get somebody's attention out of fear they won't see the small dog, because self-driving cars don't have the limited visibility issues that a human does.
  - wpietri 3y ago
    Safety is all edge cases. That's especially true of cars; we all know that 99% of the time we can drive with our minds on other things. Our whole transportation infrastructure is engineered toward that end. We can't just handwave away the edge cases, because that's where all the harm happens. Your heuristic for allowing them where they're demonstrably safer is not a bad one, but too limited. Even if they become equivalent in deaths-per-mile terms (something not demonstrated, and something not currently demonstrable given how opaque and secretive these companies are), other factors matter. And your fantasy of a world of self-driving cars is a little too fantastical to me. Are current self-driving cars better able to avoid hitting a small dog that runs out into the street? It's possible, but I would doubt it. Will they get and remain there? Again, it's possible. But it's also plausible that some manager's spreadsheet will decide that the lawsuits from owners of dead small dogs will be cheaper than better sensors for a billion self-driving cars.
    - idopmstuff 3y ago
      > Your heuristic for allowing them where they're demonstrably safer is not a bad one, but too limited. Even if they become equivalent in deaths-per-mile terms (something not demonstrated, and something not currently demonstrable given how opaque and secretive these companies are), other factors matter. What other factors? I think you're wrong about this - at the high level, what matters is do these cars save lives/prevent injuries overall. If you're saying that's not the only thing that matters, what is more important than that? > Are current self-driving cars better able to avoid hitting a small dog that runs out into the street? It's possible, but I would doubt it. The way cars are engineered, humans generally can't see a small dog directly in front of the car. Tough to be worse than that. Remember that most of the complaints about self-driving cars are that they just stop when they don't know what to do. If them stopping too much is the problematic behavior, it's weird to me that you and other folks here are using examples where they might run someone over. It just seems like an attempt to find theoretical fault, which is easy to do.
      - wpietri 3y ago
        > What other factors? You named one, injuries. Can you really not think of more? > humans generally can't see a small dog directly in front of the car Sure, but they have extensive hardware and data for social cognition. So they could well see a person with a leash running after an out-of-sight dog and infer the dog. They could see a person waving at them to stop because a of a dog. They could hear the dog barking. They could have seen the dog 30 seconds ago and said, "Hey, where did that dog get to?" They could see the dogs in the park and slow down to keep a closer eye out. They could do all manner of things that are well beyond the reach of modern technology. And I'll note you conveniently skipped over the case where car companies just skipped over worrying about the dog at all because it allowed them to "increase shareholder value".
mathewpregasen 3y ago
"Cruise, Waymo pitch: First responders should train to disable self-driving cars" this is a crazy sentence
- wpietri 3y ago
  Right? Like a firefighter trying to put out a burning car should have to get into some "your call is important to us" phone queue just to get the robot car off the firehose.
- x86x87 3y ago
  "He’s still disgruntled by an incident last weekend in the Richmond District, during which an autonomous vehicle entered the scene and parked between a fire engine and a vehicle on fire."
- setr 3y ago
  They have emergency override controls and trained procedures around it for elevators; I don’t see much of a leap for autonomous cars, assuming the override process isn’t itself ridiculous. Give firefighters a way to completely take control for 10 miles or something with a relatively low barrier to entry seems like a perfect way to eliminate vast numbers of weird scenarios
  - abwizz 3y ago
    the method will leak eventually subjecting all selfdriving cars to dos attacks
    - olyjohn 3y ago
      The potential for abuse of a car driving around with nobody in it is massive. The thing is you can already DoS them with a knife. Just poke a little hole in the tires and that thing isn't going anywhere. You don't even need to book a ride or even get in it.
    - setr 3y ago
      Sure, but you don’t need to handover full capability for it to be sufficient. That’s why I noted an arbitrary limit on mileage once the procedure is engaged — you can imagine all sorts of arbitrary other limitations; eg once in 24 hrs, limited featureset, auto-enabled tracking, flags to server that it’s been engaged, etc.
electrondood 3y ago
Tesla doesn't have this problem. I think the approach of progressively autonomous non-geofenced driver assist software is better than geofenced driverless. Tesla has a fleet of millions of cars, collecting 1m+ miles per day of real-world data. When Cruise encounters a novel situation, it halts traffic. When a Tesla running FSD does, the human takes over, it gets reported as an edge case, edge cases are labeled and bucketed by frequency and tackled in that order, failing unit tests are written, the entire fleet is queried for data on similar situations, the models are trained, the edge case is fixed, and an update is pushed to the entire fleet over the air.
- warning26 3y ago
  Or, the Tesla simply just plows into a barrier at highway speeds, which also solves the problem! Progress!
  - mwint 3y ago
    You’re not engaging in good faith.
    - simfree 3y ago
      Tesla isn't engaging with the express lane barriers on I-5 in good faith. This is a multi-year, life threatening issue that Tesla doesn't appear to care to resolve.
      - warning26 3y ago
        Yup; the fact that humans can't be expected to be ready to intervene at a moment's notice is a well-known problem in human factors engineering. GP's assertion that Tesla's strategy is good is consequently laughable.
- blamazon 3y ago
  If I take it as axiomatic that Tesla's approach carries a higher level of mortal risk, and I boil this down in a reductive way, I interpret you as saying: it's better to increase the loss of life and limb than slow auto traffic. How do you square that ethically?
JumpCrisscross 3y ago
Color me skeptical of a San Francisco bureaucracy pleading for more time to process approvals against the backdrop of a plebiscite. It's fair to say we need to speak to San Francisco's police and firefighters to get them the data they need to craft cooperation agreements. (For example, an emergency navigation override.) But absent even a single policy proposal or timeline, and considering the Taxi Workers Alliance board member's involvement, it's difficult to craft a compelling case for why safety is actually--versus theoretically--impeded.
- wpietri 3y ago
  Sorry, what's your model for how safety isn't impacted by firefighters and police having to deal with the sort of incidents described but occurring much more frequently?
  - JumpCrisscross 3y ago
    > what's your model for how safety isn't impacted by firefighters and police having to deal with the sort of incidents described but occurring much more frequently? I never said as much. We're extrapolating potential safety risks from anecdotes of interactions. I'm just saying maybe that extrapolation is off.
    - wpietri 3y ago
      You said "it's difficult to craft a compelling case for why safety is actually--versus theoretically--impeded". That apparently means you believe there is a world where these incidents happening 10 more frequently [1] will not impede safety. I would like to know what that world is. But if you can't make a case for a world where there's zero safety impact, then it seems like it would be pretty easy to make a case for a world where there's some safety impact. Personally, I'd say that a bunch of people whose whole job is public safety saying that public safety is impeded is a decent case that public safety is impeded. But since you apparently believe otherwise, I'm looking for your explanation of how there's zero safety impact. [1] "‘Blanket the city’: CEO says SF can handle 10 times more Cruise driverless vehicles": https://missionlocal.org/2023/08/cruise-origin-waymo-robotaxi-driverless-car-autonomous-vehicle-california-public-utilities-commission/ https://missionlocal.org/2023/08/cruise-origin-waymo-robotax...
Mageek 3y ago
Isn’t Waymo already operating a full service in Phoenix? Haven’t heard of many issues from there.
olyjohn 3y ago
I think they should just put those expensive-ass push bars to work that they have on their vehicles. It would take all of 2-3 minutes so shove the misbehaving car out of the way. And if it gets damaged, so what? Maybe Cruze having to come out and repair totalled cars every time they block an emergency scene, might get them to do something about it. I mean 55 wrecked cars would put a nice dent in their fleet!
- bombcar 3y ago
  It’s probably already in the law that the police/fire can move an abandoned vehicle, and a driverless car with no driver is obviously abandoned, soooooo…
  - olyjohn 3y ago
    Yeah I mean I thought it was pretty common for fire fighters to smash windows and tow cars that are blocking hydrants. I don't see this as any different. Like is a $20,000 car that's part of a giant fleet worth saving when someone's life or health is at risk?
pgodzin 3y ago
If we introduced regular cars for the first time in this political environment, we'd never allow them on the roads. Our existing cars on the road severely impede emergency vehicles, through regular stand-still traffic, double-parking without anyone in the car, parking in front of hydrants, etc. We have learned to ignore and work around those issues, but it's hard to do the same with novel issues that we give people the power to block and delay.

33 points by wpietri 3y ago

I submitted this because there are a lot of good examples of how driving is not just a process of spotting objects and following lanes, but a social process of interacting with people. I came across another example a couple of days ago. A guy was telling a story about he was out walking his two bigger dogs in the evening. Across the street, a tiny dog started barking angrily. The tiny dog got away from its owner started to cross the street, heading into traffic. So the guy with the big dogs stepped a bit into the street, waving to attract the attention of an oncoming driver who he thought might not see the small dog ready to run under the wheels. I have no idea what a robot car might make of that, but I certainly wouldn't want to be the one who has to write the sort of social processing that determines the guy is trying to signal something, so we should stop until we figure out what the signal means.

x86x87 3y ago
You get a robodog that is replaceable. If your dog gets totalled you just download it into a new body at home. You're also going to probably need robodog insurance since restoring the robodog ain't cheap /s
idopmstuff 3y ago
> so we should stop until we figure out what the signal means. We should determine how to handle driverless cars based on their overall safety, not edge cases. If driverless cars have significantly fewer injuries/accidents/etc. per mile driven than human drivers, we should push to expand them. If they only drive better in certain areas, we should geofence them. If they're safer than human drivers but would hit the guy with the dogs in this case, we should still allow them. The opportunity cost of the other accidents that would've occurred if self-driving cars weren't allowed very much matters. Plus the guy with the dogs shouldn't be stepping out into the street (and the guy with the small dog should be able to maintain control of a small dog). Also, in a world of self-driving cars, the guy doesn't have to try to get somebody's attention out of fear they won't see the small dog, because self-driving cars don't have the limited visibility issues that a human does.
- wpietri 3y ago
  Safety is all edge cases. That's especially true of cars; we all know that 99% of the time we can drive with our minds on other things. Our whole transportation infrastructure is engineered toward that end. We can't just handwave away the edge cases, because that's where all the harm happens. Your heuristic for allowing them where they're demonstrably safer is not a bad one, but too limited. Even if they become equivalent in deaths-per-mile terms (something not demonstrated, and something not currently demonstrable given how opaque and secretive these companies are), other factors matter. And your fantasy of a world of self-driving cars is a little too fantastical to me. Are current self-driving cars better able to avoid hitting a small dog that runs out into the street? It's possible, but I would doubt it. Will they get and remain there? Again, it's possible. But it's also plausible that some manager's spreadsheet will decide that the lawsuits from owners of dead small dogs will be cheaper than better sensors for a billion self-driving cars.
  - idopmstuff 3y ago
    > Your heuristic for allowing them where they're demonstrably safer is not a bad one, but too limited. Even if they become equivalent in deaths-per-mile terms (something not demonstrated, and something not currently demonstrable given how opaque and secretive these companies are), other factors matter. What other factors? I think you're wrong about this - at the high level, what matters is do these cars save lives/prevent injuries overall. If you're saying that's not the only thing that matters, what is more important than that? > Are current self-driving cars better able to avoid hitting a small dog that runs out into the street? It's possible, but I would doubt it. The way cars are engineered, humans generally can't see a small dog directly in front of the car. Tough to be worse than that. Remember that most of the complaints about self-driving cars are that they just stop when they don't know what to do. If them stopping too much is the problematic behavior, it's weird to me that you and other folks here are using examples where they might run someone over. It just seems like an attempt to find theoretical fault, which is easy to do.
    - wpietri 3y ago
      > What other factors? You named one, injuries. Can you really not think of more? > humans generally can't see a small dog directly in front of the car Sure, but they have extensive hardware and data for social cognition. So they could well see a person with a leash running after an out-of-sight dog and infer the dog. They could see a person waving at them to stop because a of a dog. They could hear the dog barking. They could have seen the dog 30 seconds ago and said, "Hey, where did that dog get to?" They could see the dogs in the park and slow down to keep a closer eye out. They could do all manner of things that are well beyond the reach of modern technology. And I'll note you conveniently skipped over the case where car companies just skipped over worrying about the dog at all because it allowed them to "increase shareholder value".

mathewpregasen 3y ago

"Cruise, Waymo pitch: First responders should train to disable self-driving cars" this is a crazy sentence

wpietri 3y ago
Right? Like a firefighter trying to put out a burning car should have to get into some "your call is important to us" phone queue just to get the robot car off the firehose.
x86x87 3y ago
"He’s still disgruntled by an incident last weekend in the Richmond District, during which an autonomous vehicle entered the scene and parked between a fire engine and a vehicle on fire."
setr 3y ago
They have emergency override controls and trained procedures around it for elevators; I don’t see much of a leap for autonomous cars, assuming the override process isn’t itself ridiculous. Give firefighters a way to completely take control for 10 miles or something with a relatively low barrier to entry seems like a perfect way to eliminate vast numbers of weird scenarios
- abwizz 3y ago
  the method will leak eventually subjecting all selfdriving cars to dos attacks
  - olyjohn 3y ago
    The potential for abuse of a car driving around with nobody in it is massive. The thing is you can already DoS them with a knife. Just poke a little hole in the tires and that thing isn't going anywhere. You don't even need to book a ride or even get in it.
  - setr 3y ago
    Sure, but you don’t need to handover full capability for it to be sufficient. That’s why I noted an arbitrary limit on mileage once the procedure is engaged — you can imagine all sorts of arbitrary other limitations; eg once in 24 hrs, limited featureset, auto-enabled tracking, flags to server that it’s been engaged, etc.

electrondood 3y ago

Tesla doesn't have this problem. I think the approach of progressively autonomous non-geofenced driver assist software is better than geofenced driverless. Tesla has a fleet of millions of cars, collecting 1m+ miles per day of real-world data. When Cruise encounters a novel situation, it halts traffic. When a Tesla running FSD does, the human takes over, it gets reported as an edge case, edge cases are labeled and bucketed by frequency and tackled in that order, failing unit tests are written, the entire fleet is queried for data on similar situations, the models are trained, the edge case is fixed, and an update is pushed to the entire fleet over the air.

warning26 3y ago
Or, the Tesla simply just plows into a barrier at highway speeds, which also solves the problem! Progress!
- mwint 3y ago
  You’re not engaging in good faith.
  - simfree 3y ago
    Tesla isn't engaging with the express lane barriers on I-5 in good faith. This is a multi-year, life threatening issue that Tesla doesn't appear to care to resolve.
    - warning26 3y ago
      Yup; the fact that humans can't be expected to be ready to intervene at a moment's notice is a well-known problem in human factors engineering. GP's assertion that Tesla's strategy is good is consequently laughable.
blamazon 3y ago
If I take it as axiomatic that Tesla's approach carries a higher level of mortal risk, and I boil this down in a reductive way, I interpret you as saying: it's better to increase the loss of life and limb than slow auto traffic. How do you square that ethically?

JumpCrisscross 3y ago

Color me skeptical of a San Francisco bureaucracy pleading for more time to process approvals against the backdrop of a plebiscite. It's fair to say we need to speak to San Francisco's police and firefighters to get them the data they need to craft cooperation agreements. (For example, an emergency navigation override.) But absent even a single policy proposal or timeline, and considering the Taxi Workers Alliance board member's involvement, it's difficult to craft a compelling case for why safety is actually--versus theoretically--impeded.

wpietri 3y ago
Sorry, what's your model for how safety isn't impacted by firefighters and police having to deal with the sort of incidents described but occurring much more frequently?
- JumpCrisscross 3y ago
  > what's your model for how safety isn't impacted by firefighters and police having to deal with the sort of incidents described but occurring much more frequently? I never said as much. We're extrapolating potential safety risks from anecdotes of interactions. I'm just saying maybe that extrapolation is off.
  - wpietri 3y ago
    You said "it's difficult to craft a compelling case for why safety is actually--versus theoretically--impeded". That apparently means you believe there is a world where these incidents happening 10 more frequently [1] will not impede safety. I would like to know what that world is. But if you can't make a case for a world where there's zero safety impact, then it seems like it would be pretty easy to make a case for a world where there's some safety impact. Personally, I'd say that a bunch of people whose whole job is public safety saying that public safety is impeded is a decent case that public safety is impeded. But since you apparently believe otherwise, I'm looking for your explanation of how there's zero safety impact. [1] "‘Blanket the city’: CEO says SF can handle 10 times more Cruise driverless vehicles": https://missionlocal.org/2023/08/cruise-origin-waymo-robotaxi-driverless-car-autonomous-vehicle-california-public-utilities-commission/ https://missionlocal.org/2023/08/cruise-origin-waymo-robotax...

Mageek 3y ago

Isn’t Waymo already operating a full service in Phoenix? Haven’t heard of many issues from there.

olyjohn 3y ago

I think they should just put those expensive-ass push bars to work that they have on their vehicles. It would take all of 2-3 minutes so shove the misbehaving car out of the way. And if it gets damaged, so what? Maybe Cruze having to come out and repair totalled cars every time they block an emergency scene, might get them to do something about it. I mean 55 wrecked cars would put a nice dent in their fleet!

bombcar 3y ago
It’s probably already in the law that the police/fire can move an abandoned vehicle, and a driverless car with no driver is obviously abandoned, soooooo…
- olyjohn 3y ago
  Yeah I mean I thought it was pretty common for fire fighters to smash windows and tow cars that are blocking hydrants. I don't see this as any different. Like is a $20,000 car that's part of a giant fleet worth saving when someone's life or health is at risk?

pgodzin 3y ago

If we introduced regular cars for the first time in this political environment, we'd never allow them on the roads. Our existing cars on the road severely impede emergency vehicles, through regular stand-still traffic, double-parking without anyone in the car, parking in front of hydrants, etc. We have learned to ignore and work around those issues, but it's hard to do the same with novel issues that we give people the power to block and delay.