19 ms·

San Francisco cops pull over a Cruise driverless car for no lights on

311 points by bmitc 5y ago

bmitc 5y ago
I remain confused on how these cars are allowed on the road. Did they pass a driver's license check? Who's responsible? Who gets the ticket? In this instance, you can see that the car performed a dangerous maneuver by running from the cops, with its lights still off, and crossing an intersection, which is then spun by the company as finding a "safer" spot to pull over at. Even though, it found a spot essentially identical to where it was. It also took off pretty fast.
- howunpleasant 5y ago
  > Did they pass a driver's license check? No > Who's responsible? Cruise and their insurance
  - sandworm101 5y ago
    Insurance doesnt pay for crimes. You cannot buy "speeding insurance" to pay your traffic tickets anymore than you can hire someone else to go to jail for you.
    - dghughes 5y ago
      >"...anymore than you can hire someone else to go to jail for you." * * unless you're rich
    - kevingadd 5y ago
      The company probably has various kinds of liability insurance that would cover things like lawyer's fees
- eru 5y ago
  There's laws and regulations for this kind of stuff, and you need to get a permit. Details depend on location / jurisdiction. Some Googling brought up eg https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/ https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...
- makeitdouble 5y ago
  In case no one is specifically identified for the infraction, doesn't it come back to the car owner ? Not in the US, but that's how it would work here for illegal parking or if the driver fled from the cops and the car is never found for instance. Even if the owner doesn't own a driving license, that's just where the buck stops. The owner if of course free to try to shove it somewhere else, but that's on them to make it work.
  - brigade 5y ago
    Parking citations attach to the car (as does civil collision liability generally), but moving violations can only be levied against the driver. Which is how some red light camera fines are avoided, if the driver isn’t clearly identifiable by the camera.
    - gambiting 5y ago
      Here in UK it's always the owner of the car who is responsible, unless they can say who was driving the car at the time. If they can't then the fine defaults to them - the assumption being that they are responsible for knowing who is using their vehicle. In other countries with similar laws(Poland) it has been argued in front of court that it cannot be the owner's responsibility to know who is driving their car at any given point especially if there are multiple people who do that usually - it should be the police's job to establish who they are fining.
      - rvnx 5y ago
        It's quite logic and an easy rule to take responsible the driver no matter whether they have a driver assist feature on or not. Then the driver could sue the manufacturer for any defect or if the manufacturer misled the user into thinking the system was safe and didn't need human attention. This way, if a system is unreliable, users will start blaming the system and stop using the feature or buying this car. Also, it allows manufacturers to roll-out features faster and learn faster from the real world. If the manufacturer is directly responsible when driver assist features are engaged, but not the driver, then the manufacturer will get scared to roll anything, and progress will be much slower, and there will be more victims because the drivers think "I don't care, I'm not responsible, it's even better if the auto mode is on, even if less safe, cause I won't be responsible"
- logifail 5y ago
  > Did they pass a driver's license check? Thought experiment: imagine a particular self-driving algo were to do something that would warrant a human driver getting their driving licence suspended, would that mean all the cars with that same algo would be simultanously suspended, or just that one car?
  - alisonkisk 5y ago
  - qayxc 5y ago
    That is a very intriguing scenario indeed. Due to the non-deterministic nature of many self-driving systems, I'd reckon a broader investigation would have to follow. Should that investigation reveal a reproducible behaviour, a recall or general suspension of vehicles running that particular software version would seem like a reasonable response. Otherwise suspending just that one car could be an option. It's uncharted territory both in terms of the legal system and w.r.t accountability so I guess no one has an answer to that yet.
    - pxmpxm 5y ago
      How is it non-deterministic? If you run the same trained model on the same input data, you should arrive at the exactly identical output.
      - qayxc 5y ago
        > How is it non-deterministic? It's non-deterministic in a practical sense; you cannot reproduce the exact circumstances that caused a certain behaviour. Maybe a bug flew by one of the cameras at a certain moment, or a pedestrian with certain type of clothing passed by. You get the idea.
        eru 5y ago
        Depends. In principle you can log exactly what data the camera sends to the decision making unit. And you could replay that data. The camera itself might not be deterministic enough to reproduce the same data with the same beetle, but replaying the log should work. (Keep the log in a 'black box' on the car, and perhaps only keep the last x hours of data, if necessary.) I think Waymo already does something like these kinds of replays.
- sjmulder 5y ago
  What worries me a little bit is how this can shift blame further on other (more vulnerable) road users once such vehicles become more common, solidifying even stronger the notion that roads are for motor vehicles and for motor vehicles alone.
  - eru 5y ago
    I'd say actually less so. As a pedestrian I'd be much more confident stepping in front of a self-driving car, than stepping in front of a human-driven car. So much so, that I've read about people's fears that self-driving cars will be subject to bullying by pedestrians (and other drivers).
    - bmitc 5y ago
      > As a pedestrian I'd be much more confident stepping in front of a self-driving car, than stepping in front of a human-driven car. Why?
      - eru 5y ago
        Because self-driving cars don't get distracted or tired, and they don't get angry either.
- ShakataGaNai 5y ago
  > Did they pass a driver's license check? Yes. That would be the AV permit that the cars operate under. Also as they are programmed to obey all road laws, it's exceedingly unlikely they will cause any moving infractions. > Who's responsible? Who gets the ticket? Cruise. Their name is on the AV permit, it's their insurance. > spot essentially identical to where it was. Before it was in a lane of traffic next to a parklet, after it was pulled over partially into what looks to be a parking spot. The SFPD officer also was no longer next to the vehicle. > how these cars are allowed on the road At the end of the day the argument is that these cars are safer than the average driver. Having spent a lot of time in SF, I've seen a lot of these cruise vehicles moving about. I remember watching them in the early days in the Dogpatch, when an errant cone would end up with them being stuck until manual action was taken. Slowly they got better and smarter. Yes, it's a hard transitional time. Mistakes may happen, but the same is true for humans. And not all AV technology is the same. Cruise has limited itself a great deal so it doesn't try to take on the entire world at the same time. Just the streets of SF, which are hard enough. How many Cruise traffic accidents have you read about? How many people injured or killed?
  - dogman144 5y ago
    Isn't OP about a moving infraction it caused, despite its programming? Arguing the probabilities of an outcome is an engineering-centric approach to selling these cars to the public, and it doesn't land well IMO/in this thread from others. It makes a false equivalence b/t how we value human agency's role in outcomes, and something other-than-human. Most people seem to prefer the risk outcomes of something I decided, vs. a faceless control system.
  - bmitc 5y ago
    Where do I apply for a permit to reassign culpability for my driving to a corporation? > At the end of the day the argument is that these cars are safer than the average driver. The last data I saw said otherwise, and that data was primarily collected in ideal conditions. That has been a few years, but these companies are not forthcoming with their data.
    - 5y ago
shafyy 5y ago
How do you pull over an autonomous car? Like, how does it know that it needs to pull over?
- thematrixturtle 5y ago
  "We work closely with the SFPD on how to interact with our vehicles and have a dedicated phone number for them to call in situations like this.” And yes, that seems thoroughly impractical at scale.
  - shafyy 5y ago
    Ok. So I guess the SFPD has been tailing the Cruise car, called the number, and then Cruise personnel made it pull over remotely?
    - ChuckNorris89 5y ago
      If that's the case then it's pretty impractical and double standard vs a human driver who needs to pull over immediately when instructed by the police or face potentially grave consequences if he doesn't immediately comply. When I'm at the wheel, I can't just tell the cop tailing me "yeah officer, I didn't pull over when you told me to behause you should have called me on my cell to tell me that" lol.
      - hericium 5y ago
        > double standard vs a human driver who needs to pull over immediately when instructed by the police or face potentially grave consequences Especially where Police is so hasty to use their cruisers as weapons[1] [1] https://eu.usatoday.com/story/news/nation/2021/06/10/arkansas-police-rammed-pregnant-womans-suv-traffic-stop-lawsuit/7645474002/ https://eu.usatoday.com/story/news/nation/2021/06/10/arkansa...
        soco 5y ago
        On an unrelated note: why are the US police cars called "cruisers"? Are they a different category so to say?
gtirloni 5y ago
Imagine if every failure in a traditional car became news.
- qayxc 5y ago
  This is relevant, though. There are traffic laws and regulations in place for human drivers and this incident poses a conundrum: how are minor transgressions handled with autonomous vehicles? Who gets fined? The owner? The manufacturer? Do the rules not apply to machines or to a lesser degree and if so, why? Lots of interesting legal questions that are much more relevant in practise than hypothetical (and IMHO completely misguided) trolly-problem related ethical discussions.
  - Ekaros 5y ago
    I think it is reasonable that the owner or operating entity gets the fine. Then they can sue the manufacturer or service provider for any issues.
    - zip1234 5y ago
      I would venture that manufacturers should prevent drivers from breaking the law when possible. For example, autonomous cars rolling stops and speeding should not be possible.
- martopix 5y ago
  The news here isn't the headlights failure.
- bo1024 5y ago
  It’s the first real autonomous testbed, and the same software presumably drives all their cars (up to A/B tests). Unlike in traditional hardware, the cause of these errors is very opaque and difficult to understand (at least for us and possibly the company as well). That makes it plausible that equally dangerous and simple errors still exist. Imagine if a traditional company had just released a new car model and someone was driving it and the front right wheel just fell off the car. It would be news because it indicates that something very basic was misunderstood by the designers and engineers. You probably wouldn’t want to get in such a car. Now imagine the general reaction is “cars are complex systems, the other three wheels actually did quite well, we fully expect to update the design to keep the front right wheel on in the future.”
surgi 5y ago
I find the explanation offered by Cruise amusing: the car supposedly "figured out" that it is in a dangerous spot to be pulled over and "tried to move to a better one" (extremely complex behavior), while the actual reason for the car being pulled over was not having its headlights on (seemingly a relatively simple bug).
- cdot2 5y ago
  If a person did this they'd probably be in some serious trouble
  - dgellow 5y ago
    Is it an euphemism for dead? /s
    - cdot2 5y ago
      no but you'd probably be arrested
  - DennisP 5y ago
    Not necessarily, if you make your intentions clear, e.g. by slowing down and putting your hazards on.
    - alisonkisk 5y ago
    - sseagull 5y ago
      Well, most of the time. If the cop feels like it, they can make you have a very bad time. https://amp.usatoday.com/amp/7645474002 https://amp.usatoday.com/amp/7645474002 “An Arkansas woman says she had no safe place to pull over in July 2020 when a state trooper tried to stop her for speeding, so she turned on her hazard lights and slowed down. Moments later, the officer rammed her vehicle, causing it to flip over and injuring the woman”
cdot2 5y ago
"Cruise vehicles are only authorized to drive from 10PM to 6AM, which obviously makes headlights pretty important." So why rely on the automatic headlights at all? Why not just set the headlights on manually?
- mobilio 5y ago
  In EU there is a law that you should drive with headlight all time. Old cars drive with low beam, new cars comes with daytime running lights (DRLs). https://en.wikipedia.org/wiki/Daytime_running_lamp#European_Union https://en.wikipedia.org/wiki/Daytime_running_lamp#European_...
  - cdot2 5y ago
    The US has this law for motorcycles. Maybe we should apply it to driverless cars as well
    - mobilio 5y ago
      That's an good idea. Actually few LED lights didn't require too much power and maintenance.
    - unnah 5y ago
      Yes, every self-driving car should have a row of LED lights at the front. Also, they should indicate that the AI is actively scanning its environment by switching the LEDs on and off in a regular left-to-right pattern.
      - ajsnigrutin 5y ago
        And those lights should be in a vertical row of 8 red lights, going left-to right. And when the car speaks to the driver and the surroundings, there should be three vertical bar graphs made of red leds to indicate the speech. There is a good controller for those kinds of lights, named "knight industry two thousand", but i think there is and upgrade to 'three tousand' available too.
      - 5y ago
grapescheesee 5y ago
Sir, step out of the car please. I am sorry, I can't do that Dave.
- grapescheesee 5y ago
  'Afraid Mode' coming soon!
- 4dahalibut 5y ago
  https://slatestarcodex.com/2015/04/21/universal-love-said-the-cactus-person/ https://slatestarcodex.com/2015/04/21/universal-love-said-th...
jwr 5y ago
I am really worried by the fact that I am the unwilling tester in the Great Driverless Car Experiment. Tradition has it that when you load-test a new bridge, you put the architect underneath. I feel like this, except I didn't design those driverless cars, somebody else did. Being an experienced software engineer, my trust in the software in these cars is pretty low. And yet they are testing them on me, because I can be the one getting killed. I think we should set a much higher bar for allowing those cars on the streets, rather than "it kinda works, so let's roll with it".
- deleted 5y ago
  [deleted]
- fallingknife 5y ago
  You could say the same for regular cars when they first arrived on the road. Saying there should be a higher bar is useless unless you are able to say exactly what that bar should be.
  - MereInterest 5y ago
    People did say exactly the same for regular cars. The result was a massive advertising campaign to establish the concept of jaywalking, and to make the street the sole domain of the car. I can easily imagine a similar campaign to label pedestrians as irresponsible for walking outside without an ultrasonic emitter, or a lidar reflector, or whatever integrates with the self-driving system. Whatever shifts the responsibility away from the car and onto the deceased.
    - MathCodeLove 5y ago
      You say "advertising campaign" as if it were some conspiracy by the world's big auto manufacturers 1. Do you have any sources? 2. You don't think that roads should be a car only domain? What do you think roads are for?
      - jeromegv 5y ago
IgorPartola 5y ago
Being used to riding motorcycles in the US, why are headlights not always on for all vehicles at this point? LED bulbs last a very long time and there is little reason to not have them on during the day. I know, it’s much more fun to anthropomorphize the car in this case, but the simple solution is to just hard wire them.
- SnowHill9902 5y ago
  You may be approaching a check point and be asked to turn off the outside lights.
- deleted 5y ago
  [deleted]
  - deleted 5y ago
    [deleted]
- bin_bash 5y ago
  Mechanics need to be able to turn them off when they work on them or they’ll get blinded. Every car I’ve driven in the last 10 years seems to have an auto mode though. I’d be surprised if the Chevy’s these are based on don’t have an auto mode. It could be that a fuse blew here.
  - bee_rider 5y ago
    I bet it is something boring, like a fuse blew, or maybe they still have the auto/manual lights selector even though the car is autonomous, and somebody misconfigured it. You'd think 'are the lights on' would be a self-check it would perform, though. GM has to know that headlights go out occasionally, right?
    - bin_bash 5y ago
      > You'd think 'are the lights on' would be a self-check it would perform, though. If it wasn't before I would bet it will be soon.
qwertox 5y ago
How about putting a piece of cloth over a sensor.
coldcode 5y ago
So what would happen if the passenger is arrested by the police for being in a car that refused to stop? I wonder what legal principle could be used to justify this. Involuntary failure to stop? Are there any laws about driverless vehicles and who is responsible for problems?
- MisterTea 5y ago
  Why would they arrest the passenger?
- throwawaylinux 5y ago
  > So what would happen if the passenger is arrested by the police for being in a car that refused to stop? Just for being in the car? Presumably they would be questioned and released. > I wonder what legal principle could be used to justify this. "Probable cause" of having committed a crime is typically the principle I believe. > Involuntary failure to stop? Are there any laws about driverless vehicles and who is responsible for problems? I don't know and can't imagine what involuntary failure to stop might be. I'm sure they could fit an existing law to the situation if they suspected the passenger was instructing the car to not stop.
lubujackson 5y ago
The real surprise here is that SF police pulled over a car at all. Have been here for more than a decade and I can count the times I have seen a car pulled over in SF on one finger - meanwhile, I have seen numerous cars blow through red lights, speeding, U-turns in the middle of the street etc. in front of police with no response.
- pie42000 5y ago
  Yeah, this was my impression as well. In Chicago you have police actively pulling people over all the time. In SF i have literally not seen a single person pulled over in the year I've been here. For context, in Chicago i would see about 1 per day, and double that in Wisconsin.
  - colinmhayes 5y ago
    > In Chicago you have police actively pulling people over all the time Lived in Chicago for 25 years, this is pretty much the opposite of my experience. Seems like 90% of the time someone is pulled over here they're being arrested so I don't think it's because of a traffic violation.
  - pbrb 5y ago
    In Chicago? Like, in the city proper and not the suburbs? I've lived most of my life in Chicago, and it's basically unheard of to get pulled over for a traffic violation. Lately, blatant disregard for traffic laws in Chicago has gotten so bad that it's a little terrifying. It's become a common thread on the r/chicago subreddit with regular posts pointing out how bad it's gotten and how police never stop anybody. Here are two posts from yesterday alone: https://www.reddit.com/r/chicago/comments/tzyrmn/red_lights_are_discretionary_now/ https://www.reddit.com/r/chicago/comments/tzyrmn/red_lights_... https://www.reddit.com/r/chicago/comments/u0x3sd/im_getting_reeeaaaaaallll_sick_of_cars_not/ https://www.reddit.com/r/chicago/comments/u0x3sd/im_getting_... EDIT: Wanted to point this out b/c I think it's a larger problem in general of people not following rules of the road, not isolated to specific cities.
- _hhkc 5y ago
  Yeah this is the real surprise for me as well. How many cars drive around SF with no license plates and the SFPD doesn't do anything about it.
noasaservice 5y ago
A human gets pulled over and gets a ticket. A bot gets pulled over and the cop acquiesces to the company. Of course, monied interests dont get tickets. Only us plebes do.
nkingsy 5y ago
Just my anecdote, saw a cruise car trying to turn left from the middle lane on Franklin. Came to a complete stop, bringing traffic to a halt, and waited for the cars on its left to clear out, including anyone who went around, then cautiously moved into the left lane and turned. Some kind of rude, selfish programming I’d be yelling at a person for.
- axg11 5y ago
  I wonder if Cruise or other AVs have a programmed threshold to "give up"? A human driver in the middle lane might decide to keep driving on and take a different route rather than hold those people up.
- MisterTea 5y ago
  > Some kind of rude, selfish programming I’d be yelling at a person for. Sounds like an inexperienced or incompetent driver who lacks confidence. Basically, the kind of driver that shouldn't be on the road.
  - zip1234 5y ago
    Perhaps all human drivers are incompetent? In the USA there are 5 million crashes that involve insurance every year. Insurance companies say that someone will get in a crash (however minor) once every ~18 years of driving on average.
  - nkingsy 5y ago
    Rereading my comment yeah that’s harsh. Not trying to promote road rage though it is kind of fun to generally be loud from the safety of your car when no one can hear you.
- AlotOfReading 5y ago
  The intention is for the car to be a very boring, unremarkable driver. This sort of behavior has been noted before and most incidents were thought to have been fixed by a merge ~6 months ago, but there can always be missed scenarios/interactions. Was this more recent?
timcavel 5y ago
gothungry 5y ago
There are a lot of things concerning about these cars driving themselves around the city -- most of which are typically covered in these comment threads -- but here are a few questions that seem relevant to this particular incident: How are police officers trained to deal with these machines? Hopefully its more than "call this number"; are Cruise support lines required by law to be open when their cars are on the streets? What mechanism to police or city officials have to control these machines? Do they have some interface to put the vehicle in neutral and roll it away? Assuming no, why not? What's the right protocol if a car gets a flat tire or is driving around with a door open? Would a driverless car by a different manufacture follow all the same protocols? I'm sure there are clear answers to similar questions -- for example -- for the BART or Muni. It is preposterous that something this significant and potentially dangerous can be deployed on streets in a city with tens of thousands of pedestrians, and no messaging to the public about these details.
- woah 5y ago
  I think they all still have test drivers in them right?
- ra7 5y ago
  They do outreach programs for local police and fire departments on first responder training and have published relevant materials. Waymo: https://waymo.com/firstresponders/ https://waymo.com/firstresponders/ Cruise: https://www.getcruise.com/firstresponders https://www.getcruise.com/firstresponders
xyzzy21 5y ago
The police should have shot the tires out and boxed in the vehicle to stop it from escaping.
- free-malloc 5y ago
  Don't forget pistol whipping the application just because it was written in php.
Animats 5y ago
Cruise Automation has a terrible video for first responders.[1] The dialog sounds like it was written by lawyers. Summary: - Vehicles recognize lights and sirens and will pull over. - Call Cruise's control center at 888-662-7103 in case of trouble with a vehicle. The control center needs the name of the vehicle, which is painted on the car in 3 places. Or at least the location. - Chocking and blocking the wheels, as firefighters routinely do, is safe. - Other emergency procedures are the same as for a regular Chevy Bolt. [1] https://www.youtube.com/watch?v=ZM3kfauMgZY https://www.youtube.com/watch?v=ZM3kfauMgZY

311 points by bmitc 5y ago

I remain confused on how these cars are allowed on the road. Did they pass a driver's license check? Who's responsible? Who gets the ticket? In this instance, you can see that the car performed a dangerous maneuver by running from the cops, with its lights still off, and crossing an intersection, which is then spun by the company as finding a "safer" spot to pull over at. Even though, it found a spot essentially identical to where it was. It also took off pretty fast.

howunpleasant 5y ago
> Did they pass a driver's license check? No > Who's responsible? Cruise and their insurance
- sandworm101 5y ago
  Insurance doesnt pay for crimes. You cannot buy "speeding insurance" to pay your traffic tickets anymore than you can hire someone else to go to jail for you.
  - dghughes 5y ago
    >"...anymore than you can hire someone else to go to jail for you." * * unless you're rich
  - kevingadd 5y ago
    The company probably has various kinds of liability insurance that would cover things like lawyer's fees
eru 5y ago
There's laws and regulations for this kind of stuff, and you need to get a permit. Details depend on location / jurisdiction. Some Googling brought up eg https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/ https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...
makeitdouble 5y ago
In case no one is specifically identified for the infraction, doesn't it come back to the car owner ? Not in the US, but that's how it would work here for illegal parking or if the driver fled from the cops and the car is never found for instance. Even if the owner doesn't own a driving license, that's just where the buck stops. The owner if of course free to try to shove it somewhere else, but that's on them to make it work.
- brigade 5y ago
  Parking citations attach to the car (as does civil collision liability generally), but moving violations can only be levied against the driver. Which is how some red light camera fines are avoided, if the driver isn’t clearly identifiable by the camera.
  - gambiting 5y ago
    Here in UK it's always the owner of the car who is responsible, unless they can say who was driving the car at the time. If they can't then the fine defaults to them - the assumption being that they are responsible for knowing who is using their vehicle. In other countries with similar laws(Poland) it has been argued in front of court that it cannot be the owner's responsibility to know who is driving their car at any given point especially if there are multiple people who do that usually - it should be the police's job to establish who they are fining.
    - rvnx 5y ago
      It's quite logic and an easy rule to take responsible the driver no matter whether they have a driver assist feature on or not. Then the driver could sue the manufacturer for any defect or if the manufacturer misled the user into thinking the system was safe and didn't need human attention. This way, if a system is unreliable, users will start blaming the system and stop using the feature or buying this car. Also, it allows manufacturers to roll-out features faster and learn faster from the real world. If the manufacturer is directly responsible when driver assist features are engaged, but not the driver, then the manufacturer will get scared to roll anything, and progress will be much slower, and there will be more victims because the drivers think "I don't care, I'm not responsible, it's even better if the auto mode is on, even if less safe, cause I won't be responsible"
logifail 5y ago
> Did they pass a driver's license check? Thought experiment: imagine a particular self-driving algo were to do something that would warrant a human driver getting their driving licence suspended, would that mean all the cars with that same algo would be simultanously suspended, or just that one car?
- alisonkisk 5y ago
- qayxc 5y ago
  That is a very intriguing scenario indeed. Due to the non-deterministic nature of many self-driving systems, I'd reckon a broader investigation would have to follow. Should that investigation reveal a reproducible behaviour, a recall or general suspension of vehicles running that particular software version would seem like a reasonable response. Otherwise suspending just that one car could be an option. It's uncharted territory both in terms of the legal system and w.r.t accountability so I guess no one has an answer to that yet.
  - pxmpxm 5y ago
    How is it non-deterministic? If you run the same trained model on the same input data, you should arrive at the exactly identical output.
    - qayxc 5y ago
      > How is it non-deterministic? It's non-deterministic in a practical sense; you cannot reproduce the exact circumstances that caused a certain behaviour. Maybe a bug flew by one of the cameras at a certain moment, or a pedestrian with certain type of clothing passed by. You get the idea.
      - eru 5y ago
        Depends. In principle you can log exactly what data the camera sends to the decision making unit. And you could replay that data. The camera itself might not be deterministic enough to reproduce the same data with the same beetle, but replaying the log should work. (Keep the log in a 'black box' on the car, and perhaps only keep the last x hours of data, if necessary.) I think Waymo already does something like these kinds of replays.
sjmulder 5y ago
What worries me a little bit is how this can shift blame further on other (more vulnerable) road users once such vehicles become more common, solidifying even stronger the notion that roads are for motor vehicles and for motor vehicles alone.
- eru 5y ago
  I'd say actually less so. As a pedestrian I'd be much more confident stepping in front of a self-driving car, than stepping in front of a human-driven car. So much so, that I've read about people's fears that self-driving cars will be subject to bullying by pedestrians (and other drivers).
  - bmitc 5y ago
    > As a pedestrian I'd be much more confident stepping in front of a self-driving car, than stepping in front of a human-driven car. Why?
    - eru 5y ago
      Because self-driving cars don't get distracted or tired, and they don't get angry either.
ShakataGaNai 5y ago
> Did they pass a driver's license check? Yes. That would be the AV permit that the cars operate under. Also as they are programmed to obey all road laws, it's exceedingly unlikely they will cause any moving infractions. > Who's responsible? Who gets the ticket? Cruise. Their name is on the AV permit, it's their insurance. > spot essentially identical to where it was. Before it was in a lane of traffic next to a parklet, after it was pulled over partially into what looks to be a parking spot. The SFPD officer also was no longer next to the vehicle. > how these cars are allowed on the road At the end of the day the argument is that these cars are safer than the average driver. Having spent a lot of time in SF, I've seen a lot of these cruise vehicles moving about. I remember watching them in the early days in the Dogpatch, when an errant cone would end up with them being stuck until manual action was taken. Slowly they got better and smarter. Yes, it's a hard transitional time. Mistakes may happen, but the same is true for humans. And not all AV technology is the same. Cruise has limited itself a great deal so it doesn't try to take on the entire world at the same time. Just the streets of SF, which are hard enough. How many Cruise traffic accidents have you read about? How many people injured or killed?
- dogman144 5y ago
  Isn't OP about a moving infraction it caused, despite its programming? Arguing the probabilities of an outcome is an engineering-centric approach to selling these cars to the public, and it doesn't land well IMO/in this thread from others. It makes a false equivalence b/t how we value human agency's role in outcomes, and something other-than-human. Most people seem to prefer the risk outcomes of something I decided, vs. a faceless control system.
- bmitc 5y ago
  Where do I apply for a permit to reassign culpability for my driving to a corporation? > At the end of the day the argument is that these cars are safer than the average driver. The last data I saw said otherwise, and that data was primarily collected in ideal conditions. That has been a few years, but these companies are not forthcoming with their data.
  - 5y ago

shafyy 5y ago

How do you pull over an autonomous car? Like, how does it know that it needs to pull over?

thematrixturtle 5y ago
"We work closely with the SFPD on how to interact with our vehicles and have a dedicated phone number for them to call in situations like this.” And yes, that seems thoroughly impractical at scale.
- shafyy 5y ago
  Ok. So I guess the SFPD has been tailing the Cruise car, called the number, and then Cruise personnel made it pull over remotely?
  - ChuckNorris89 5y ago
    If that's the case then it's pretty impractical and double standard vs a human driver who needs to pull over immediately when instructed by the police or face potentially grave consequences if he doesn't immediately comply. When I'm at the wheel, I can't just tell the cop tailing me "yeah officer, I didn't pull over when you told me to behause you should have called me on my cell to tell me that" lol.
    - hericium 5y ago
      > double standard vs a human driver who needs to pull over immediately when instructed by the police or face potentially grave consequences Especially where Police is so hasty to use their cruisers as weapons[1] [1] https://eu.usatoday.com/story/news/nation/2021/06/10/arkansas-police-rammed-pregnant-womans-suv-traffic-stop-lawsuit/7645474002/ https://eu.usatoday.com/story/news/nation/2021/06/10/arkansa...
      - soco 5y ago
        On an unrelated note: why are the US police cars called "cruisers"? Are they a different category so to say?

gtirloni 5y ago

Imagine if every failure in a traditional car became news.

qayxc 5y ago
This is relevant, though. There are traffic laws and regulations in place for human drivers and this incident poses a conundrum: how are minor transgressions handled with autonomous vehicles? Who gets fined? The owner? The manufacturer? Do the rules not apply to machines or to a lesser degree and if so, why? Lots of interesting legal questions that are much more relevant in practise than hypothetical (and IMHO completely misguided) trolly-problem related ethical discussions.
- Ekaros 5y ago
  I think it is reasonable that the owner or operating entity gets the fine. Then they can sue the manufacturer or service provider for any issues.
  - zip1234 5y ago
    I would venture that manufacturers should prevent drivers from breaking the law when possible. For example, autonomous cars rolling stops and speeding should not be possible.
martopix 5y ago
The news here isn't the headlights failure.
bo1024 5y ago
It’s the first real autonomous testbed, and the same software presumably drives all their cars (up to A/B tests). Unlike in traditional hardware, the cause of these errors is very opaque and difficult to understand (at least for us and possibly the company as well). That makes it plausible that equally dangerous and simple errors still exist. Imagine if a traditional company had just released a new car model and someone was driving it and the front right wheel just fell off the car. It would be news because it indicates that something very basic was misunderstood by the designers and engineers. You probably wouldn’t want to get in such a car. Now imagine the general reaction is “cars are complex systems, the other three wheels actually did quite well, we fully expect to update the design to keep the front right wheel on in the future.”

surgi 5y ago

I find the explanation offered by Cruise amusing: the car supposedly "figured out" that it is in a dangerous spot to be pulled over and "tried to move to a better one" (extremely complex behavior), while the actual reason for the car being pulled over was not having its headlights on (seemingly a relatively simple bug).

cdot2 5y ago
If a person did this they'd probably be in some serious trouble
- dgellow 5y ago
  Is it an euphemism for dead? /s
  - cdot2 5y ago
    no but you'd probably be arrested
- DennisP 5y ago
  Not necessarily, if you make your intentions clear, e.g. by slowing down and putting your hazards on.
  - alisonkisk 5y ago
  - sseagull 5y ago
    Well, most of the time. If the cop feels like it, they can make you have a very bad time. https://amp.usatoday.com/amp/7645474002 https://amp.usatoday.com/amp/7645474002 “An Arkansas woman says she had no safe place to pull over in July 2020 when a state trooper tried to stop her for speeding, so she turned on her hazard lights and slowed down. Moments later, the officer rammed her vehicle, causing it to flip over and injuring the woman”

cdot2 5y ago

"Cruise vehicles are only authorized to drive from 10PM to 6AM, which obviously makes headlights pretty important." So why rely on the automatic headlights at all? Why not just set the headlights on manually?

mobilio 5y ago
In EU there is a law that you should drive with headlight all time. Old cars drive with low beam, new cars comes with daytime running lights (DRLs). https://en.wikipedia.org/wiki/Daytime_running_lamp#European_Union https://en.wikipedia.org/wiki/Daytime_running_lamp#European_...
- cdot2 5y ago
  The US has this law for motorcycles. Maybe we should apply it to driverless cars as well
  - mobilio 5y ago
    That's an good idea. Actually few LED lights didn't require too much power and maintenance.
  - unnah 5y ago
    Yes, every self-driving car should have a row of LED lights at the front. Also, they should indicate that the AI is actively scanning its environment by switching the LEDs on and off in a regular left-to-right pattern.
    - ajsnigrutin 5y ago
      And those lights should be in a vertical row of 8 red lights, going left-to right. And when the car speaks to the driver and the surroundings, there should be three vertical bar graphs made of red leds to indicate the speech. There is a good controller for those kinds of lights, named "knight industry two thousand", but i think there is and upgrade to 'three tousand' available too.
    - 5y ago

grapescheesee 5y ago

Sir, step out of the car please. I am sorry, I can't do that Dave.

grapescheesee 5y ago
'Afraid Mode' coming soon!
4dahalibut 5y ago
https://slatestarcodex.com/2015/04/21/universal-love-said-the-cactus-person/ https://slatestarcodex.com/2015/04/21/universal-love-said-th...

jwr 5y ago

I am really worried by the fact that I am the unwilling tester in the Great Driverless Car Experiment. Tradition has it that when you load-test a new bridge, you put the architect underneath. I feel like this, except I didn't design those driverless cars, somebody else did. Being an experienced software engineer, my trust in the software in these cars is pretty low. And yet they are testing them on me, because I can be the one getting killed. I think we should set a much higher bar for allowing those cars on the streets, rather than "it kinda works, so let's roll with it".

deleted 5y ago
[deleted]
fallingknife 5y ago
You could say the same for regular cars when they first arrived on the road. Saying there should be a higher bar is useless unless you are able to say exactly what that bar should be.
- MereInterest 5y ago
  People did say exactly the same for regular cars. The result was a massive advertising campaign to establish the concept of jaywalking, and to make the street the sole domain of the car. I can easily imagine a similar campaign to label pedestrians as irresponsible for walking outside without an ultrasonic emitter, or a lidar reflector, or whatever integrates with the self-driving system. Whatever shifts the responsibility away from the car and onto the deceased.
  - MathCodeLove 5y ago
    You say "advertising campaign" as if it were some conspiracy by the world's big auto manufacturers 1. Do you have any sources? 2. You don't think that roads should be a car only domain? What do you think roads are for?
    - jeromegv 5y ago

IgorPartola 5y ago

Being used to riding motorcycles in the US, why are headlights not always on for all vehicles at this point? LED bulbs last a very long time and there is little reason to not have them on during the day. I know, it’s much more fun to anthropomorphize the car in this case, but the simple solution is to just hard wire them.

SnowHill9902 5y ago
You may be approaching a check point and be asked to turn off the outside lights.
deleted 5y ago
[deleted]
- deleted 5y ago
  [deleted]
bin_bash 5y ago
Mechanics need to be able to turn them off when they work on them or they’ll get blinded. Every car I’ve driven in the last 10 years seems to have an auto mode though. I’d be surprised if the Chevy’s these are based on don’t have an auto mode. It could be that a fuse blew here.
- bee_rider 5y ago
  I bet it is something boring, like a fuse blew, or maybe they still have the auto/manual lights selector even though the car is autonomous, and somebody misconfigured it. You'd think 'are the lights on' would be a self-check it would perform, though. GM has to know that headlights go out occasionally, right?
  - bin_bash 5y ago
    > You'd think 'are the lights on' would be a self-check it would perform, though. If it wasn't before I would bet it will be soon.

qwertox 5y ago

How about putting a piece of cloth over a sensor.

coldcode 5y ago

So what would happen if the passenger is arrested by the police for being in a car that refused to stop? I wonder what legal principle could be used to justify this. Involuntary failure to stop? Are there any laws about driverless vehicles and who is responsible for problems?

MisterTea 5y ago
Why would they arrest the passenger?
throwawaylinux 5y ago
> So what would happen if the passenger is arrested by the police for being in a car that refused to stop? Just for being in the car? Presumably they would be questioned and released. > I wonder what legal principle could be used to justify this. "Probable cause" of having committed a crime is typically the principle I believe. > Involuntary failure to stop? Are there any laws about driverless vehicles and who is responsible for problems? I don't know and can't imagine what involuntary failure to stop might be. I'm sure they could fit an existing law to the situation if they suspected the passenger was instructing the car to not stop.

lubujackson 5y ago

The real surprise here is that SF police pulled over a car at all. Have been here for more than a decade and I can count the times I have seen a car pulled over in SF on one finger - meanwhile, I have seen numerous cars blow through red lights, speeding, U-turns in the middle of the street etc. in front of police with no response.

pie42000 5y ago
Yeah, this was my impression as well. In Chicago you have police actively pulling people over all the time. In SF i have literally not seen a single person pulled over in the year I've been here. For context, in Chicago i would see about 1 per day, and double that in Wisconsin.
- colinmhayes 5y ago
  > In Chicago you have police actively pulling people over all the time Lived in Chicago for 25 years, this is pretty much the opposite of my experience. Seems like 90% of the time someone is pulled over here they're being arrested so I don't think it's because of a traffic violation.
- pbrb 5y ago
  In Chicago? Like, in the city proper and not the suburbs? I've lived most of my life in Chicago, and it's basically unheard of to get pulled over for a traffic violation. Lately, blatant disregard for traffic laws in Chicago has gotten so bad that it's a little terrifying. It's become a common thread on the r/chicago subreddit with regular posts pointing out how bad it's gotten and how police never stop anybody. Here are two posts from yesterday alone: https://www.reddit.com/r/chicago/comments/tzyrmn/red_lights_are_discretionary_now/ https://www.reddit.com/r/chicago/comments/tzyrmn/red_lights_... https://www.reddit.com/r/chicago/comments/u0x3sd/im_getting_reeeaaaaaallll_sick_of_cars_not/ https://www.reddit.com/r/chicago/comments/u0x3sd/im_getting_... EDIT: Wanted to point this out b/c I think it's a larger problem in general of people not following rules of the road, not isolated to specific cities.
_hhkc 5y ago
Yeah this is the real surprise for me as well. How many cars drive around SF with no license plates and the SFPD doesn't do anything about it.

noasaservice 5y ago

A human gets pulled over and gets a ticket. A bot gets pulled over and the cop acquiesces to the company. Of course, monied interests dont get tickets. Only us plebes do.

nkingsy 5y ago

Just my anecdote, saw a cruise car trying to turn left from the middle lane on Franklin. Came to a complete stop, bringing traffic to a halt, and waited for the cars on its left to clear out, including anyone who went around, then cautiously moved into the left lane and turned. Some kind of rude, selfish programming I’d be yelling at a person for.

axg11 5y ago
I wonder if Cruise or other AVs have a programmed threshold to "give up"? A human driver in the middle lane might decide to keep driving on and take a different route rather than hold those people up.
MisterTea 5y ago
> Some kind of rude, selfish programming I’d be yelling at a person for. Sounds like an inexperienced or incompetent driver who lacks confidence. Basically, the kind of driver that shouldn't be on the road.
- zip1234 5y ago
  Perhaps all human drivers are incompetent? In the USA there are 5 million crashes that involve insurance every year. Insurance companies say that someone will get in a crash (however minor) once every ~18 years of driving on average.
- nkingsy 5y ago
  Rereading my comment yeah that’s harsh. Not trying to promote road rage though it is kind of fun to generally be loud from the safety of your car when no one can hear you.
AlotOfReading 5y ago
The intention is for the car to be a very boring, unremarkable driver. This sort of behavior has been noted before and most incidents were thought to have been fixed by a merge ~6 months ago, but there can always be missed scenarios/interactions. Was this more recent?

timcavel 5y ago

gothungry 5y ago

There are a lot of things concerning about these cars driving themselves around the city -- most of which are typically covered in these comment threads -- but here are a few questions that seem relevant to this particular incident: How are police officers trained to deal with these machines? Hopefully its more than "call this number"; are Cruise support lines required by law to be open when their cars are on the streets? What mechanism to police or city officials have to control these machines? Do they have some interface to put the vehicle in neutral and roll it away? Assuming no, why not? What's the right protocol if a car gets a flat tire or is driving around with a door open? Would a driverless car by a different manufacture follow all the same protocols? I'm sure there are clear answers to similar questions -- for example -- for the BART or Muni. It is preposterous that something this significant and potentially dangerous can be deployed on streets in a city with tens of thousands of pedestrians, and no messaging to the public about these details.

woah 5y ago
I think they all still have test drivers in them right?
ra7 5y ago
They do outreach programs for local police and fire departments on first responder training and have published relevant materials. Waymo: https://waymo.com/firstresponders/ https://waymo.com/firstresponders/ Cruise: https://www.getcruise.com/firstresponders https://www.getcruise.com/firstresponders

xyzzy21 5y ago

The police should have shot the tires out and boxed in the vehicle to stop it from escaping.

free-malloc 5y ago
Don't forget pistol whipping the application just because it was written in php.

Animats 5y ago

Cruise Automation has a terrible video for first responders.[1] The dialog sounds like it was written by lawyers. Summary: - Vehicles recognize lights and sirens and will pull over. - Call Cruise's control center at 888-662-7103 in case of trouble with a vehicle. The control center needs the name of the vehicle, which is painted on the car in 3 places. Or at least the location. - Chocking and blocking the wheels, as firefighters routinely do, is safe. - Other emergency procedures are the same as for a regular Chevy Bolt. [1] https://www.youtube.com/watch?v=ZM3kfauMgZY https://www.youtube.com/watch?v=ZM3kfauMgZY