6 ms·

> But how do we distinguish real emergency scenario from fake one? A car shouldn't make that call. It should err on the side of safety and assuming the emergen

by jxf 4y ago

> But how do we distinguish real emergency scenario from fake one? A car shouldn't make that call. It should err on the side of safety and assuming the emergency is real. When you see flashing lights on the highway coming up behind you, do you think "I bet that ambulance is faking it so they can get home faster", or do you just move over?

Swenrekcah 4y ago
But then you have essentially enabled everyone to wave your car to a stop, which could be very convenient for certain types of people. I think the simple truth is that we are still a long long long time away from fully autonomous cars to be safely deployed anywhere except on very specific paths that are mostly closed off to everyone else. But they could be tremendously useful there, for instance driving buses or airport shuttles and that kind of thing.
- ocdtrekkie 4y ago
  So this is currently possible today: I can walk out on any street today and redirect traffic. With or without my reflective vest, drivers will generally listen to me. And self-driving vehicles must as well. Note that when I am doing traffic direction with appropriate authority, most drivers will never have heard of the organization I'm a part of (a local CERT team), nor have any real way to vet the authority I have. The authority they can determine I have is mostly that I'm doing it and no police officer has come by to tell me to stop doing it. (Note that CERT members do not and should not self-deploy, mind you, and if you choose not to heed our directions, we almost certainly can find a cop car to come let you know you screwed up.) Also bear in mind, many people directing traffic aren't public safety personnel at all. Construction workers are private employees, events have their own staff which direct traffic in various contexts, including public roads. There's no definitive way to determine that someone is authorized to direct traffic, cars just need to obey them.
  - Swenrekcah 4y ago
    >There's no definitive way to determine that someone is authorized to direct traffic, cars just need to obey them. Exactly! But there still is a way. You don't obey anyone in any circumstances because that would be dangerous. People have a feeling for it, computers don't yet.
    - LarryMullins 4y ago
      I'd obey such traffic commands from just about anybody in plain street clothes, unless they looked obviously crazy or if I was in part of the world where car jacking is common. At the very least I'd stop and ask them what was up. Maybe there is some dangerous condition ahead and they're trying to warn me. Does the computerized car understand the meaning of the words "Turn back because the bridge is out"?
      - Swenrekcah 4y ago
        I miswrote, my meaning wasn’t that I would stop for nobody, but that I wouldn’t stop for just anybody in just any situation. Essentially the same that you said. This is something that people are very good at judging but software isn’t. I’m trying to say that until we have figured out AGI, autonomous cars driving among people are very dangerous.
      - bee_rider 4y ago
        I bet you’d eyeball it though, right? Like if someone was telling cars to stop, looked nervous, and had a hand in a large enough pocket to hold a gun, I might head in the direction they were sending people (depending on visibility I guess) but I’d probably not slow down.
  - kube-system 4y ago
    Human drivers are smart, and will generally judge your appearance, confidence, and context to determine whether or not your actions are legitimate. And you likely always pass these tests. Imagine a teenager, dressed in street wear, who jumps into the middle of a street to stop traffic in a way that has no explanation given the scene, while their friends laugh on the sidewalk as they observe. Or a group of men in masks stopping a car in an alley as their accomplices surround the vehicle. Any human driver will recognize these scenarios as illegitimate. It isn't just the act of directing traffic that is judged in isolation.
    - ecnahc515 4y ago
      In basically every example you gave, you still have to stop until people move though. The result is the same, despite everything else. The only one that's a bit iffy is if you're being carjacked, is a bit of an exception. Legitimacy isn't really the problem, it's the circumstance of a human being in front of your car that prevents you from moving forward, and beyond getting out of your car and talking to them, or just waiting, you don't have much of a choice in what options you have. I agree with the intent of your point, but what's the alternative? You can't just keep driving if someone is standing in the middle of the road.
      - kube-system 4y ago
        Those were two examples, but they're not exhaustive. Traffic direction can (and often does) happen without someone blocking the vehicle. In the case of an illegitimate request, a human driver might question the request and take a different action instead. Are you being directed to run a red light? If you judge that someone is maliciously asking you to do so, you probably will choose to remain still instead. Are you being directed to stop? If you judge that someone is maliciously asking you to do so, you may choose to turn around, reverse, turn into a parking lot or otherwise remove your vehicle from the situation. You don't have to run people over to refuse a command to stop. And if you were completely blocked from moving, you'd probably call the police. My point is that "always follow hand signals" is trivially problematic. I would expect that if self-driving cars followed any hand signal they saw, you'd see kids doing it for the lulz on the sidewalk in front of their school. ..."traffic jam challenge". After all, someone is less likely to act maliciously towards a human than they are a machine.
  - mr_toad 4y ago
    > Note that when I am doing traffic direction with appropriate authority, most drivers will never have heard of the organization I'm a part of Reading between the lines it doesn’t seem like there’s a lot of communication to emergency services etc on how to deal with autonomous vehicles. Even the police seem to be operating under misconceptions about what the vehicles can and can’t do.
    - maxerickson 4y ago
      Isn't it the vehicle operators operating the vehicles while ignoring that there are things that the vehicles must do in certain scenarios to be suitable for operation?
    - ocdtrekkie 4y ago
      So there's no monolithic entity that handles public safety in any given area. Overlapping jurisdictions of police, fire, public works, utility companies, construction companies, volunteer orgs, etc. all will end up on the streets for different reasons. If self-driving cars cannot cope with this, they should not be allowed on public roads. Very little has convinced me they should currently be street legal.
  - alasdair_ 4y ago
    >most drivers will never have heard of the organization I'm a part of (a local CERT team) Will you have to file a CVE after the incident?
    - ocdtrekkie 4y ago
      As a sysadmin and a volunteer responder, I do find the fact CISA and FEMA each have a CERT program that are completely different things pretty funny. Community Emergency Response Team: https://community.fema.gov/PreparednessCommunity/s/welcome-to-cert?language=en_US https://community.fema.gov/PreparednessCommunity/s/welcome-t...
upwardbound 4y ago
Agreed - faking an emergency is highly frowned upon and I'm betting it's a crime in most places. There's even a similarity with the following aspect of the Geneva Convention, which is that pretending to surrender is so harmful to norms that it is literally a war crime. https://en.wikipedia.org/wiki/Perfidy https://en.wikipedia.org/wiki/Perfidy "In the context of war, perfidy is a form of deception in which one side promises to act in good faith (such as by raising a flag of truce) with the intention of breaking that promise ... Perfidy constitutes a breach of the laws of war and so is a war crime, as it degrades the protections and mutual restraints developed in the interest of all parties, combatants and civilians. " Pretending to surrender is something that's shown a lot in Star Trek and similar shows, and Pirates of the Carribean, but in our real-life world of 2023 it's a UN war crime to pretend to surrender, and it's really not a nice thing to do because if some people sometimes pretend to surrender, then pretty soon no one will trust surrender attempts and people who wanted to stop fighting will needlessly be lost. The idea of criminals faking a medical emergency is similar imho. If I were in a position of determining how much time robbers should serve in prison, I would personally recommend a much much longer sentence if they faked a medical emergency, for exactly this reason.
- LarryMullins 4y ago
  I saw a video showing perfidy committed by a Russian soldier in Ukraine. Several Russians were coming out of a building surrendering to some Ukrainian soldiers, when one of them came out with a gun and started shooting. All the Russians were killed in the subsequent firefight, because one of them ruined it for the rest. It may seem like a clever ruse in movies, but in real life it gets people killed.
  - sofixa 4y ago
    It'd be funny if it wasn't so terrible. At this point it would be easier to list the war crimes not committed by the Russian army, they seem to be well intent on speed running the Geneva convention. One can only hope one day there's trials in e.g. The Hague to bring all war criminals to justice.
kelnos 4y ago
I think the potential issue is that the driver of a non-emergency vehicle could start gesturing in particular ways that could trigger the autonomous car to think it was an emergency vehicle. So then you have to teach the car what all the various kinds of emergency vehicles look like, in every jurisdiction. That's hard enough, but then what happens if it's an unmarked police car, for example?
kube-system 4y ago
I think the point is that humans do make this judgement call. We know what a real emergency vehicle and what real emergency personnel looks like, and we know what a crazy person waving their arms looks like. When some idiot in the opposing lanes is waving for me to take a left across multiple lanes of moving traffic, I know that I should ignore their idiotic traffic direction. I know that when there's an emergency situation with a fire fighter telling me to continue through a red light, I know that I should follow the directions.
- nerdbert 4y ago
  I hope you're not suggesting like a hater that a Tesla with FSD (available next week, I promise) isn't capable of performing a snap psych eval of any random person standing at the side of the road.
  - kube-system 4y ago
    I am suggesting that a car which follows all human gestures is not better than a car that follows no human gestures.
BurningFrog 4y ago
To spell it out, if an SDC Taxi stops for anyone waving frantically, how do you prevent robbers and rapists from routinely harvesting their passengers?
- polygamous_bat 4y ago
  Exactly how you stop people from flagging a human driven taxi with plights for help and then robbing or raping the passenger. This doesn't seem like a problem specific to self driving. Relevant xkcd: https://xkcd.com/1958/ https://xkcd.com/1958/
  - BurningFrog 4y ago
    A Taxi has a sober human driver with professional experience to handle such situations. A piece of software with some cameras and actuators is really not the same.
mikedg 4y ago
self driving car AI challenge, run into the cross walk and start waving every car through
_carbyau_ 4y ago
I agree with treat it as real. Then, if the car has an adult passenger you can ask them for their judgement (and responsibility for the outcome!) If the car has no passenger. Then stop and move slowly as required/indicated. Sure, it may be a car thief, but without a passenger, it is simply another car theft.
dragonwriter 4y ago
> When you see flashing lights on the highway coming up behind you, do you think “I bet that ambulance is faking it so they can get home faster”, or do you just move over? “You” in the specific or general sense? Because, my observations of highway behavior suggest that actual humans are…far from ideal at this.

by jxf 4y ago

Swenrekcah 4y ago

But then you have essentially enabled everyone to wave your car to a stop, which could be very convenient for certain types of people. I think the simple truth is that we are still a long long long time away from fully autonomous cars to be safely deployed anywhere except on very specific paths that are mostly closed off to everyone else. But they could be tremendously useful there, for instance driving buses or airport shuttles and that kind of thing.

ocdtrekkie 4y ago
So this is currently possible today: I can walk out on any street today and redirect traffic. With or without my reflective vest, drivers will generally listen to me. And self-driving vehicles must as well. Note that when I am doing traffic direction with appropriate authority, most drivers will never have heard of the organization I'm a part of (a local CERT team), nor have any real way to vet the authority I have. The authority they can determine I have is mostly that I'm doing it and no police officer has come by to tell me to stop doing it. (Note that CERT members do not and should not self-deploy, mind you, and if you choose not to heed our directions, we almost certainly can find a cop car to come let you know you screwed up.) Also bear in mind, many people directing traffic aren't public safety personnel at all. Construction workers are private employees, events have their own staff which direct traffic in various contexts, including public roads. There's no definitive way to determine that someone is authorized to direct traffic, cars just need to obey them.
- Swenrekcah 4y ago
  >There's no definitive way to determine that someone is authorized to direct traffic, cars just need to obey them. Exactly! But there still is a way. You don't obey anyone in any circumstances because that would be dangerous. People have a feeling for it, computers don't yet.
  - LarryMullins 4y ago
    I'd obey such traffic commands from just about anybody in plain street clothes, unless they looked obviously crazy or if I was in part of the world where car jacking is common. At the very least I'd stop and ask them what was up. Maybe there is some dangerous condition ahead and they're trying to warn me. Does the computerized car understand the meaning of the words "Turn back because the bridge is out"?
    - Swenrekcah 4y ago
      I miswrote, my meaning wasn’t that I would stop for nobody, but that I wouldn’t stop for just anybody in just any situation. Essentially the same that you said. This is something that people are very good at judging but software isn’t. I’m trying to say that until we have figured out AGI, autonomous cars driving among people are very dangerous.
    - bee_rider 4y ago
      I bet you’d eyeball it though, right? Like if someone was telling cars to stop, looked nervous, and had a hand in a large enough pocket to hold a gun, I might head in the direction they were sending people (depending on visibility I guess) but I’d probably not slow down.
- kube-system 4y ago
  Human drivers are smart, and will generally judge your appearance, confidence, and context to determine whether or not your actions are legitimate. And you likely always pass these tests. Imagine a teenager, dressed in street wear, who jumps into the middle of a street to stop traffic in a way that has no explanation given the scene, while their friends laugh on the sidewalk as they observe. Or a group of men in masks stopping a car in an alley as their accomplices surround the vehicle. Any human driver will recognize these scenarios as illegitimate. It isn't just the act of directing traffic that is judged in isolation.
  - ecnahc515 4y ago
    In basically every example you gave, you still have to stop until people move though. The result is the same, despite everything else. The only one that's a bit iffy is if you're being carjacked, is a bit of an exception. Legitimacy isn't really the problem, it's the circumstance of a human being in front of your car that prevents you from moving forward, and beyond getting out of your car and talking to them, or just waiting, you don't have much of a choice in what options you have. I agree with the intent of your point, but what's the alternative? You can't just keep driving if someone is standing in the middle of the road.
    - kube-system 4y ago
      Those were two examples, but they're not exhaustive. Traffic direction can (and often does) happen without someone blocking the vehicle. In the case of an illegitimate request, a human driver might question the request and take a different action instead. Are you being directed to run a red light? If you judge that someone is maliciously asking you to do so, you probably will choose to remain still instead. Are you being directed to stop? If you judge that someone is maliciously asking you to do so, you may choose to turn around, reverse, turn into a parking lot or otherwise remove your vehicle from the situation. You don't have to run people over to refuse a command to stop. And if you were completely blocked from moving, you'd probably call the police. My point is that "always follow hand signals" is trivially problematic. I would expect that if self-driving cars followed any hand signal they saw, you'd see kids doing it for the lulz on the sidewalk in front of their school. ..."traffic jam challenge". After all, someone is less likely to act maliciously towards a human than they are a machine.
- mr_toad 4y ago
  > Note that when I am doing traffic direction with appropriate authority, most drivers will never have heard of the organization I'm a part of Reading between the lines it doesn’t seem like there’s a lot of communication to emergency services etc on how to deal with autonomous vehicles. Even the police seem to be operating under misconceptions about what the vehicles can and can’t do.
  - maxerickson 4y ago
    Isn't it the vehicle operators operating the vehicles while ignoring that there are things that the vehicles must do in certain scenarios to be suitable for operation?
  - ocdtrekkie 4y ago
    So there's no monolithic entity that handles public safety in any given area. Overlapping jurisdictions of police, fire, public works, utility companies, construction companies, volunteer orgs, etc. all will end up on the streets for different reasons. If self-driving cars cannot cope with this, they should not be allowed on public roads. Very little has convinced me they should currently be street legal.
- alasdair_ 4y ago
  >most drivers will never have heard of the organization I'm a part of (a local CERT team) Will you have to file a CVE after the incident?
  - ocdtrekkie 4y ago
    As a sysadmin and a volunteer responder, I do find the fact CISA and FEMA each have a CERT program that are completely different things pretty funny. Community Emergency Response Team: https://community.fema.gov/PreparednessCommunity/s/welcome-to-cert?language=en_US https://community.fema.gov/PreparednessCommunity/s/welcome-t...

upwardbound 4y ago

Agreed - faking an emergency is highly frowned upon and I'm betting it's a crime in most places. There's even a similarity with the following aspect of the Geneva Convention, which is that pretending to surrender is so harmful to norms that it is literally a war crime. https://en.wikipedia.org/wiki/Perfidy https://en.wikipedia.org/wiki/Perfidy "In the context of war, perfidy is a form of deception in which one side promises to act in good faith (such as by raising a flag of truce) with the intention of breaking that promise ... Perfidy constitutes a breach of the laws of war and so is a war crime, as it degrades the protections and mutual restraints developed in the interest of all parties, combatants and civilians. " Pretending to surrender is something that's shown a lot in Star Trek and similar shows, and Pirates of the Carribean, but in our real-life world of 2023 it's a UN war crime to pretend to surrender, and it's really not a nice thing to do because if some people sometimes pretend to surrender, then pretty soon no one will trust surrender attempts and people who wanted to stop fighting will needlessly be lost. The idea of criminals faking a medical emergency is similar imho. If I were in a position of determining how much time robbers should serve in prison, I would personally recommend a much much longer sentence if they faked a medical emergency, for exactly this reason.

LarryMullins 4y ago
I saw a video showing perfidy committed by a Russian soldier in Ukraine. Several Russians were coming out of a building surrendering to some Ukrainian soldiers, when one of them came out with a gun and started shooting. All the Russians were killed in the subsequent firefight, because one of them ruined it for the rest. It may seem like a clever ruse in movies, but in real life it gets people killed.
- sofixa 4y ago
  It'd be funny if it wasn't so terrible. At this point it would be easier to list the war crimes not committed by the Russian army, they seem to be well intent on speed running the Geneva convention. One can only hope one day there's trials in e.g. The Hague to bring all war criminals to justice.

kelnos 4y ago

I think the potential issue is that the driver of a non-emergency vehicle could start gesturing in particular ways that could trigger the autonomous car to think it was an emergency vehicle. So then you have to teach the car what all the various kinds of emergency vehicles look like, in every jurisdiction. That's hard enough, but then what happens if it's an unmarked police car, for example?

kube-system 4y ago

I think the point is that humans do make this judgement call. We know what a real emergency vehicle and what real emergency personnel looks like, and we know what a crazy person waving their arms looks like. When some idiot in the opposing lanes is waving for me to take a left across multiple lanes of moving traffic, I know that I should ignore their idiotic traffic direction. I know that when there's an emergency situation with a fire fighter telling me to continue through a red light, I know that I should follow the directions.

nerdbert 4y ago
I hope you're not suggesting like a hater that a Tesla with FSD (available next week, I promise) isn't capable of performing a snap psych eval of any random person standing at the side of the road.
- kube-system 4y ago
  I am suggesting that a car which follows all human gestures is not better than a car that follows no human gestures.

BurningFrog 4y ago

To spell it out, if an SDC Taxi stops for anyone waving frantically, how do you prevent robbers and rapists from routinely harvesting their passengers?

polygamous_bat 4y ago
Exactly how you stop people from flagging a human driven taxi with plights for help and then robbing or raping the passenger. This doesn't seem like a problem specific to self driving. Relevant xkcd: https://xkcd.com/1958/ https://xkcd.com/1958/
- BurningFrog 4y ago
  A Taxi has a sober human driver with professional experience to handle such situations. A piece of software with some cameras and actuators is really not the same.

mikedg 4y ago

self driving car AI challenge, run into the cross walk and start waving every car through

_carbyau_ 4y ago

I agree with treat it as real. Then, if the car has an adult passenger you can ask them for their judgement (and responsibility for the outcome!) If the car has no passenger. Then stop and move slowly as required/indicated. Sure, it may be a car thief, but without a passenger, it is simply another car theft.

dragonwriter 4y ago

> When you see flashing lights on the highway coming up behind you, do you think “I bet that ambulance is faking it so they can get home faster”, or do you just move over? “You” in the specific or general sense? Because, my observations of highway behavior suggest that actual humans are…far from ideal at this.