8 ms·

Sounds like having support for interpreting human gestures will solve some of the problems. But how do we distinguish real emergency scenario from fake one? S

Sounds like having support for interpreting human gestures will solve some of the problems. But how do we distinguish real emergency scenario from fake one? Such kinds of problems will only scale up as we apply L5 cars take on more public roads, but I wonder how long or what changes will have to be done in current road signs, critical city infra to support them.

outworlder 4y ago
> Sounds like having support for interpreting human gestures will solve some of the problems. Add that to the _LONG_ list of issues that will have to be solved until we can even think about releasing autonomous cars in the streets.
jxf 4y ago
> But how do we distinguish real emergency scenario from fake one? A car shouldn't make that call. It should err on the side of safety and assuming the emergency is real. When you see flashing lights on the highway coming up behind you, do you think "I bet that ambulance is faking it so they can get home faster", or do you just move over?
- Swenrekcah 4y ago
  But then you have essentially enabled everyone to wave your car to a stop, which could be very convenient for certain types of people. I think the simple truth is that we are still a long long long time away from fully autonomous cars to be safely deployed anywhere except on very specific paths that are mostly closed off to everyone else. But they could be tremendously useful there, for instance driving buses or airport shuttles and that kind of thing.
  - ocdtrekkie 4y ago
    So this is currently possible today: I can walk out on any street today and redirect traffic. With or without my reflective vest, drivers will generally listen to me. And self-driving vehicles must as well. Note that when I am doing traffic direction with appropriate authority, most drivers will never have heard of the organization I'm a part of (a local CERT team), nor have any real way to vet the authority I have. The authority they can determine I have is mostly that I'm doing it and no police officer has come by to tell me to stop doing it. (Note that CERT members do not and should not self-deploy, mind you, and if you choose not to heed our directions, we almost certainly can find a cop car to come let you know you screwed up.) Also bear in mind, many people directing traffic aren't public safety personnel at all. Construction workers are private employees, events have their own staff which direct traffic in various contexts, including public roads. There's no definitive way to determine that someone is authorized to direct traffic, cars just need to obey them.
    - Swenrekcah 4y ago
      >There's no definitive way to determine that someone is authorized to direct traffic, cars just need to obey them. Exactly! But there still is a way. You don't obey anyone in any circumstances because that would be dangerous. People have a feeling for it, computers don't yet.
      - LarryMullins 4y ago
        I'd obey such traffic commands from just about anybody in plain street clothes, unless they looked obviously crazy or if I was in part of the world where car jacking is common. At the very least I'd stop and ask them what was up. Maybe there is some dangerous condition ahead and they're trying to warn me. Does the computerized car understand the meaning of the words "Turn back because the bridge is out"?
        Swenrekcah 4y ago
        I miswrote, my meaning wasn’t that I would stop for nobody, but that I wouldn’t stop for just anybody in just any situation. Essentially the same that you said. This is something that people are very good at judging but software isn’t. I’m trying to say that until we have figured out AGI, autonomous cars driving among people are very dangerous.
        bee_rider 4y ago
        I bet you’d eyeball it though, right? Like if someone was telling cars to stop, looked nervous, and had a hand in a large enough pocket to hold a gun, I might head in the direction they were sending people (depending on visibility I guess) but I’d probably not slow down.
    - kube-system 4y ago
      Human drivers are smart, and will generally judge your appearance, confidence, and context to determine whether or not your actions are legitimate. And you likely always pass these tests. Imagine a teenager, dressed in street wear, who jumps into the middle of a street to stop traffic in a way that has no explanation given the scene, while their friends laugh on the sidewalk as they observe. Or a group of men in masks stopping a car in an alley as their accomplices surround the vehicle. Any human driver will recognize these scenarios as illegitimate. It isn't just the act of directing traffic that is judged in isolation.
- upwardbound 4y ago
  Agreed - faking an emergency is highly frowned upon and I'm betting it's a crime in most places. There's even a similarity with the following aspect of the Geneva Convention, which is that pretending to surrender is so harmful to norms that it is literally a war crime. https://en.wikipedia.org/wiki/Perfidy https://en.wikipedia.org/wiki/Perfidy "In the context of war, perfidy is a form of deception in which one side promises to act in good faith (such as by raising a flag of truce) with the intention of breaking that promise ... Perfidy constitutes a breach of the laws of war and so is a war crime, as it degrades the protections and mutual restraints developed in the interest of all parties, combatants and civilians. " Pretending to surrender is something that's shown a lot in Star Trek and similar shows, and Pirates of the Carribean, but in our real-life world of 2023 it's a UN war crime to pretend to surrender, and it's really not a nice thing to do because if some people sometimes pretend to surrender, then pretty soon no one will trust surrender attempts and people who wanted to stop fighting will needlessly be lost. The idea of criminals faking a medical emergency is similar imho. If I were in a position of determining how much time robbers should serve in prison, I would personally recommend a much much longer sentence if they faked a medical emergency, for exactly this reason.
  - LarryMullins 4y ago
    I saw a video showing perfidy committed by a Russian soldier in Ukraine. Several Russians were coming out of a building surrendering to some Ukrainian soldiers, when one of them came out with a gun and started shooting. All the Russians were killed in the subsequent firefight, because one of them ruined it for the rest. It may seem like a clever ruse in movies, but in real life it gets people killed.
    - sofixa 4y ago
      It'd be funny if it wasn't so terrible. At this point it would be easier to list the war crimes not committed by the Russian army, they seem to be well intent on speed running the Geneva convention. One can only hope one day there's trials in e.g. The Hague to bring all war criminals to justice.
- kelnos 4y ago
  I think the potential issue is that the driver of a non-emergency vehicle could start gesturing in particular ways that could trigger the autonomous car to think it was an emergency vehicle. So then you have to teach the car what all the various kinds of emergency vehicles look like, in every jurisdiction. That's hard enough, but then what happens if it's an unmarked police car, for example?
- kube-system 4y ago
  I think the point is that humans do make this judgement call. We know what a real emergency vehicle and what real emergency personnel looks like, and we know what a crazy person waving their arms looks like. When some idiot in the opposing lanes is waving for me to take a left across multiple lanes of moving traffic, I know that I should ignore their idiotic traffic direction. I know that when there's an emergency situation with a fire fighter telling me to continue through a red light, I know that I should follow the directions.
  - nerdbert 4y ago
    I hope you're not suggesting like a hater that a Tesla with FSD (available next week, I promise) isn't capable of performing a snap psych eval of any random person standing at the side of the road.
    - kube-system 4y ago
      I am suggesting that a car which follows all human gestures is not better than a car that follows no human gestures.
- BurningFrog 4y ago
  To spell it out, if an SDC Taxi stops for anyone waving frantically, how do you prevent robbers and rapists from routinely harvesting their passengers?
  - polygamous_bat 4y ago
    Exactly how you stop people from flagging a human driven taxi with plights for help and then robbing or raping the passenger. This doesn't seem like a problem specific to self driving. Relevant xkcd: https://xkcd.com/1958/ https://xkcd.com/1958/
    - BurningFrog 4y ago
      A Taxi has a sober human driver with professional experience to handle such situations. A piece of software with some cameras and actuators is really not the same.
- mikedg 4y ago
  self driving car AI challenge, run into the cross walk and start waving every car through
- _carbyau_ 4y ago
  I agree with treat it as real. Then, if the car has an adult passenger you can ask them for their judgement (and responsibility for the outcome!) If the car has no passenger. Then stop and move slowly as required/indicated. Sure, it may be a car thief, but without a passenger, it is simply another car theft.
- dragonwriter 4y ago
  > When you see flashing lights on the highway coming up behind you, do you think “I bet that ambulance is faking it so they can get home faster”, or do you just move over? “You” in the specific or general sense? Because, my observations of highway behavior suggest that actual humans are…far from ideal at this.
Bilal_io 4y ago
No need for human gesture, just put a stop sign or a road closed sign. These were likely taught to the model as its ABCs
- ghaff 4y ago
  As a general principle, one can imagine special signals relevant to autonomous vehicles. But there's probably something of a chicken and egg sort of thing and I have problems with a lot of processes being required to deal with early adopters of this sort of technology.
- kelnos 4y ago
  It kinda sucks, though, to put the responsibility for this on the first responders, who already have enough to do dealing with the emergency.
- nerdbert 4y ago
  There are going to be times when someone is legitimately responding to an emergency and doesn't have a stop sign handy. And there are going to be a lot of pranksters who do, and will use them to mess with so-called self-driving cars.
neuralRiot 4y ago
One thing that is often overlooked is that software can only understand what it was trained to understand.
- deleted 4y ago
  [deleted]
__MatrixMan__ 4y ago
There's a connected vehicle protocol (DSRC) which broadcasts all kinds of metadata about your vehicle. There was stuff like "transaxle temperature" and "steering angle" in there, if I recall, so I'd be surprised if there wasn't also "emergency flashers engaged". Perhaps autonomous vehicles should be be sensitive to that field, signaling for manual control when they're too near a vehicle which has it enabled. That way any vehicle--not just the fire department--can create a "human control only" field by activating their emergency flashers. Perhaps emergency vehicles could be recognized such that they create a much larger field.
krisoft 4y ago
> But how do we distinguish real emergency scenario from fake one? You don’t. Bad actors can already pretend to be fireman doing fireman things and direct traffic. Human drivers will also be mislead by that. What is more, if you have a high viz vest, a hard hat, and more confidence than a garden slug you can also direct trafic. Not legally mind you, but more often than not people will follow your directions. If there is a whole gaggle of fake construction people and they have cones too they can probably re-route a highway. Whatever is stopping people from causing misschief (too often) by these “powers” doesn’t seem to be different in the case of self driving cars than with human driven cars. In fact the self-driving car is more likely to retain a perfect photographic evidence of the mischief maker’s face than a regular human driver.
8note 4y ago
How do people distinguish that? A person is likely to fall for a fake police officer and fake police car
- Rebelgecko 4y ago
  Have you seen any of the adversarial neural net stuff? I wouldn't be surprised if it was possible to trick a car into thinking someone is a police officer without formally impersonating a cop
phkahler 4y ago
>> but I wonder how long or what changes will have to be done in current road signs, critical city infra to support them. Unfortunately that attitude is common is the industry. The notion that the rest of the world needs to provide extra, up to date information for the shitty AI so it can function. I have news for ya. Emergency road repairs, fires, etc are not going to make it into your world database in a timely manner so a dumb vehicle can navigate the situation. It will all be over before anyone even thinks about fixing your system for you.
treeman79 4y ago
Was in backroads and ran into a cop directing travel. Apparently a small fair nearby. Dude was totally pissed and gesturing every which way wildly. When I couldn’t immediately decode. He just started swinging even more wildly. Turning redder and redder. Was fairly sure I was going to get shot arrested. So I just did a U turn. As I was leaving the next guy started yelling at cop that he was blocking all possible routes to his town.

outworlder 4y ago

> Sounds like having support for interpreting human gestures will solve some of the problems. Add that to the _LONG_ list of issues that will have to be solved until we can even think about releasing autonomous cars in the streets.

jxf 4y ago

> But how do we distinguish real emergency scenario from fake one? A car shouldn't make that call. It should err on the side of safety and assuming the emergency is real. When you see flashing lights on the highway coming up behind you, do you think "I bet that ambulance is faking it so they can get home faster", or do you just move over?

Swenrekcah 4y ago
But then you have essentially enabled everyone to wave your car to a stop, which could be very convenient for certain types of people. I think the simple truth is that we are still a long long long time away from fully autonomous cars to be safely deployed anywhere except on very specific paths that are mostly closed off to everyone else. But they could be tremendously useful there, for instance driving buses or airport shuttles and that kind of thing.
- ocdtrekkie 4y ago
  So this is currently possible today: I can walk out on any street today and redirect traffic. With or without my reflective vest, drivers will generally listen to me. And self-driving vehicles must as well. Note that when I am doing traffic direction with appropriate authority, most drivers will never have heard of the organization I'm a part of (a local CERT team), nor have any real way to vet the authority I have. The authority they can determine I have is mostly that I'm doing it and no police officer has come by to tell me to stop doing it. (Note that CERT members do not and should not self-deploy, mind you, and if you choose not to heed our directions, we almost certainly can find a cop car to come let you know you screwed up.) Also bear in mind, many people directing traffic aren't public safety personnel at all. Construction workers are private employees, events have their own staff which direct traffic in various contexts, including public roads. There's no definitive way to determine that someone is authorized to direct traffic, cars just need to obey them.
  - Swenrekcah 4y ago
    >There's no definitive way to determine that someone is authorized to direct traffic, cars just need to obey them. Exactly! But there still is a way. You don't obey anyone in any circumstances because that would be dangerous. People have a feeling for it, computers don't yet.
    - LarryMullins 4y ago
      I'd obey such traffic commands from just about anybody in plain street clothes, unless they looked obviously crazy or if I was in part of the world where car jacking is common. At the very least I'd stop and ask them what was up. Maybe there is some dangerous condition ahead and they're trying to warn me. Does the computerized car understand the meaning of the words "Turn back because the bridge is out"?
      - Swenrekcah 4y ago
        I miswrote, my meaning wasn’t that I would stop for nobody, but that I wouldn’t stop for just anybody in just any situation. Essentially the same that you said. This is something that people are very good at judging but software isn’t. I’m trying to say that until we have figured out AGI, autonomous cars driving among people are very dangerous.
      - bee_rider 4y ago
        I bet you’d eyeball it though, right? Like if someone was telling cars to stop, looked nervous, and had a hand in a large enough pocket to hold a gun, I might head in the direction they were sending people (depending on visibility I guess) but I’d probably not slow down.
  - kube-system 4y ago
    Human drivers are smart, and will generally judge your appearance, confidence, and context to determine whether or not your actions are legitimate. And you likely always pass these tests. Imagine a teenager, dressed in street wear, who jumps into the middle of a street to stop traffic in a way that has no explanation given the scene, while their friends laugh on the sidewalk as they observe. Or a group of men in masks stopping a car in an alley as their accomplices surround the vehicle. Any human driver will recognize these scenarios as illegitimate. It isn't just the act of directing traffic that is judged in isolation.
upwardbound 4y ago
Agreed - faking an emergency is highly frowned upon and I'm betting it's a crime in most places. There's even a similarity with the following aspect of the Geneva Convention, which is that pretending to surrender is so harmful to norms that it is literally a war crime. https://en.wikipedia.org/wiki/Perfidy https://en.wikipedia.org/wiki/Perfidy "In the context of war, perfidy is a form of deception in which one side promises to act in good faith (such as by raising a flag of truce) with the intention of breaking that promise ... Perfidy constitutes a breach of the laws of war and so is a war crime, as it degrades the protections and mutual restraints developed in the interest of all parties, combatants and civilians. " Pretending to surrender is something that's shown a lot in Star Trek and similar shows, and Pirates of the Carribean, but in our real-life world of 2023 it's a UN war crime to pretend to surrender, and it's really not a nice thing to do because if some people sometimes pretend to surrender, then pretty soon no one will trust surrender attempts and people who wanted to stop fighting will needlessly be lost. The idea of criminals faking a medical emergency is similar imho. If I were in a position of determining how much time robbers should serve in prison, I would personally recommend a much much longer sentence if they faked a medical emergency, for exactly this reason.
- LarryMullins 4y ago
  I saw a video showing perfidy committed by a Russian soldier in Ukraine. Several Russians were coming out of a building surrendering to some Ukrainian soldiers, when one of them came out with a gun and started shooting. All the Russians were killed in the subsequent firefight, because one of them ruined it for the rest. It may seem like a clever ruse in movies, but in real life it gets people killed.
  - sofixa 4y ago
    It'd be funny if it wasn't so terrible. At this point it would be easier to list the war crimes not committed by the Russian army, they seem to be well intent on speed running the Geneva convention. One can only hope one day there's trials in e.g. The Hague to bring all war criminals to justice.
kelnos 4y ago
I think the potential issue is that the driver of a non-emergency vehicle could start gesturing in particular ways that could trigger the autonomous car to think it was an emergency vehicle. So then you have to teach the car what all the various kinds of emergency vehicles look like, in every jurisdiction. That's hard enough, but then what happens if it's an unmarked police car, for example?
kube-system 4y ago
I think the point is that humans do make this judgement call. We know what a real emergency vehicle and what real emergency personnel looks like, and we know what a crazy person waving their arms looks like. When some idiot in the opposing lanes is waving for me to take a left across multiple lanes of moving traffic, I know that I should ignore their idiotic traffic direction. I know that when there's an emergency situation with a fire fighter telling me to continue through a red light, I know that I should follow the directions.
- nerdbert 4y ago
  I hope you're not suggesting like a hater that a Tesla with FSD (available next week, I promise) isn't capable of performing a snap psych eval of any random person standing at the side of the road.
  - kube-system 4y ago
    I am suggesting that a car which follows all human gestures is not better than a car that follows no human gestures.
BurningFrog 4y ago
To spell it out, if an SDC Taxi stops for anyone waving frantically, how do you prevent robbers and rapists from routinely harvesting their passengers?
- polygamous_bat 4y ago
  Exactly how you stop people from flagging a human driven taxi with plights for help and then robbing or raping the passenger. This doesn't seem like a problem specific to self driving. Relevant xkcd: https://xkcd.com/1958/ https://xkcd.com/1958/
  - BurningFrog 4y ago
    A Taxi has a sober human driver with professional experience to handle such situations. A piece of software with some cameras and actuators is really not the same.
mikedg 4y ago
self driving car AI challenge, run into the cross walk and start waving every car through
_carbyau_ 4y ago
I agree with treat it as real. Then, if the car has an adult passenger you can ask them for their judgement (and responsibility for the outcome!) If the car has no passenger. Then stop and move slowly as required/indicated. Sure, it may be a car thief, but without a passenger, it is simply another car theft.
dragonwriter 4y ago
> When you see flashing lights on the highway coming up behind you, do you think “I bet that ambulance is faking it so they can get home faster”, or do you just move over? “You” in the specific or general sense? Because, my observations of highway behavior suggest that actual humans are…far from ideal at this.

Bilal_io 4y ago

No need for human gesture, just put a stop sign or a road closed sign. These were likely taught to the model as its ABCs

ghaff 4y ago
As a general principle, one can imagine special signals relevant to autonomous vehicles. But there's probably something of a chicken and egg sort of thing and I have problems with a lot of processes being required to deal with early adopters of this sort of technology.
kelnos 4y ago
It kinda sucks, though, to put the responsibility for this on the first responders, who already have enough to do dealing with the emergency.
nerdbert 4y ago
There are going to be times when someone is legitimately responding to an emergency and doesn't have a stop sign handy. And there are going to be a lot of pranksters who do, and will use them to mess with so-called self-driving cars.

neuralRiot 4y ago

One thing that is often overlooked is that software can only understand what it was trained to understand.

deleted 4y ago
[deleted]

__MatrixMan__ 4y ago

There's a connected vehicle protocol (DSRC) which broadcasts all kinds of metadata about your vehicle. There was stuff like "transaxle temperature" and "steering angle" in there, if I recall, so I'd be surprised if there wasn't also "emergency flashers engaged". Perhaps autonomous vehicles should be be sensitive to that field, signaling for manual control when they're too near a vehicle which has it enabled. That way any vehicle--not just the fire department--can create a "human control only" field by activating their emergency flashers. Perhaps emergency vehicles could be recognized such that they create a much larger field.

krisoft 4y ago

> But how do we distinguish real emergency scenario from fake one? You don’t. Bad actors can already pretend to be fireman doing fireman things and direct traffic. Human drivers will also be mislead by that. What is more, if you have a high viz vest, a hard hat, and more confidence than a garden slug you can also direct trafic. Not legally mind you, but more often than not people will follow your directions. If there is a whole gaggle of fake construction people and they have cones too they can probably re-route a highway. Whatever is stopping people from causing misschief (too often) by these “powers” doesn’t seem to be different in the case of self driving cars than with human driven cars. In fact the self-driving car is more likely to retain a perfect photographic evidence of the mischief maker’s face than a regular human driver.

8note 4y ago

How do people distinguish that? A person is likely to fall for a fake police officer and fake police car

Rebelgecko 4y ago
Have you seen any of the adversarial neural net stuff? I wouldn't be surprised if it was possible to trick a car into thinking someone is a police officer without formally impersonating a cop

phkahler 4y ago

>> but I wonder how long or what changes will have to be done in current road signs, critical city infra to support them. Unfortunately that attitude is common is the industry. The notion that the rest of the world needs to provide extra, up to date information for the shitty AI so it can function. I have news for ya. Emergency road repairs, fires, etc are not going to make it into your world database in a timely manner so a dumb vehicle can navigate the situation. It will all be over before anyone even thinks about fixing your system for you.

treeman79 4y ago

Was in backroads and ran into a cop directing travel. Apparently a small fair nearby. Dude was totally pissed and gesturing every which way wildly. When I couldn’t immediately decode. He just started swinging even more wildly. Turning redder and redder. Was fairly sure I was going to get shot arrested. So I just did a U turn. As I was leaving the next guy started yelling at cop that he was blocking all possible routes to his town.