12 ms·
I'm slightly alarmed by the general scenario of a driverless vehicle approaching an emergency situation. A human-driven vehicle will respond to gestures from f
by retrac 4y ago
I'm slightly alarmed by the general scenario of a driverless vehicle approaching an emergency situation. A human-driven vehicle will respond to gestures from firefighters to go around, etc. And even most dull humans will have more situational awareness and flexibility than an AI is likely to, particularly as these sorts of scenarios are edge cases by definition. E.g. depending on your jurisdiction and the situation, it may be proper to run a red light to make way for an emergency vehicle. The insistence of the driver waving frantically at you is part of that judgement call. Software is not ready for that yet, I think.
- redtriumph 4y agoSounds like having support for interpreting human gestures will solve some of the problems. But how do we distinguish real emergency scenario from fake one? Such kinds of problems will only scale up as we apply L5 cars take on more public roads, but I wonder how long or what changes will have to be done in current road signs, critical city infra to support them.
- outworlder 4y ago> Sounds like having support for interpreting human gestures will solve some of the problems. Add that to the _LONG_ list of issues that will have to be solved until we can even think about releasing autonomous cars in the streets.
- jxf 4y ago> But how do we distinguish real emergency scenario from fake one? A car shouldn't make that call. It should err on the side of safety and assuming the emergency is real. When you see flashing lights on the highway coming up behind you, do you think "I bet that ambulance is faking it so they can get home faster", or do you just move over?
- Swenrekcah 4y agoBut then you have essentially enabled everyone to wave your car to a stop, which could be very convenient for certain types of people. I think the simple truth is that we are still a long long long time away from fully autonomous cars to be safely deployed anywhere except on very specific paths that are mostly closed off to everyone else. But they could be tremendously useful there, for instance driving buses or airport shuttles and that kind of thing.
- ocdtrekkie 4y agoSo this is currently possible today: I can walk out on any street today and redirect traffic. With or without my reflective vest, drivers will generally listen to me. And self-driving vehicles must as well. Note that when I am doing traffic direction with appropriate authority, most drivers will never have heard of the organization I'm a part of (a local CERT team), nor have any real way to vet the authority I have. The authority they can determine I have is mostly that I'm doing it and no police officer has come by to tell me to stop doing it. (Note that CERT members do not and should not self-deploy, mind you, and if you choose not to heed our directions, we almost certainly can find a cop car to come let you know you screwed up.) Also bear in mind, many people directing traffic aren't public safety personnel at all. Construction workers are private employees, events have their own staff which direct traffic in various contexts, including public roads. There's no definitive way to determine that someone is authorized to direct traffic, cars just need to obey them.
- Swenrekcah 4y ago>There's no definitive way to determine that someone is authorized to direct traffic, cars just need to obey them. Exactly! But there still is a way. You don't obey anyone in any circumstances because that would be dangerous. People have a feeling for it, computers don't yet.
- LarryMullins 4y agoI'd obey such traffic commands from just about anybody in plain street clothes, unless they looked obviously crazy or if I was in part of the world where car jacking is common. At the very least I'd stop and ask them what was up. Maybe there is some dangerous condition ahead and they're trying to warn me. Does the computerized car understand the meaning of the words "Turn back because the bridge is out"?
- upwardbound 4y agoAgreed - faking an emergency is highly frowned upon and I'm betting it's a crime in most places. There's even a similarity with the following aspect of the Geneva Convention, which is that pretending to surrender is so harmful to norms that it is literally a war crime. https://en.wikipedia.org/wiki/Perfidy https://en.wikipedia.org/wiki/Perfidy "In the context of war, perfidy is a form of deception in which one side promises to act in good faith (such as by raising a flag of truce) with the intention of breaking that promise ... Perfidy constitutes a breach of the laws of war and so is a war crime, as it degrades the protections and mutual restraints developed in the interest of all parties, combatants and civilians. " Pretending to surrender is something that's shown a lot in Star Trek and similar shows, and Pirates of the Carribean, but in our real-life world of 2023 it's a UN war crime to pretend to surrender, and it's really not a nice thing to do because if some people sometimes pretend to surrender, then pretty soon no one will trust surrender attempts and people who wanted to stop fighting will needlessly be lost. The idea of criminals faking a medical emergency is similar imho. If I were in a position of determining how much time robbers should serve in prison, I would personally recommend a much much longer sentence if they faked a medical emergency, for exactly this reason.
- LarryMullins 4y agoI saw a video showing perfidy committed by a Russian soldier in Ukraine. Several Russians were coming out of a building surrendering to some Ukrainian soldiers, when one of them came out with a gun and started shooting. All the Russians were killed in the subsequent firefight, because one of them ruined it for the rest. It may seem like a clever ruse in movies, but in real life it gets people killed.
- sofixa 4y agoIt'd be funny if it wasn't so terrible. At this point it would be easier to list the war crimes not committed by the Russian army, they seem to be well intent on speed running the Geneva convention. One can only hope one day there's trials in e.g. The Hague to bring all war criminals to justice.
- kelnos 4y agoI think the potential issue is that the driver of a non-emergency vehicle could start gesturing in particular ways that could trigger the autonomous car to think it was an emergency vehicle. So then you have to teach the car what all the various kinds of emergency vehicles look like, in every jurisdiction. That's hard enough, but then what happens if it's an unmarked police car, for example?
- kube-system 4y agoI think the point is that humans do make this judgement call. We know what a real emergency vehicle and what real emergency personnel looks like, and we know what a crazy person waving their arms looks like. When some idiot in the opposing lanes is waving for me to take a left across multiple lanes of moving traffic, I know that I should ignore their idiotic traffic direction. I know that when there's an emergency situation with a fire fighter telling me to continue through a red light, I know that I should follow the directions.
- nerdbert 4y agoI hope you're not suggesting like a hater that a Tesla with FSD (available next week, I promise) isn't capable of performing a snap psych eval of any random person standing at the side of the road.
- kube-system 4y agoI am suggesting that a car which follows all human gestures is not better than a car that follows no human gestures.
- BurningFrog 4y agoTo spell it out, if an SDC Taxi stops for anyone waving frantically, how do you prevent robbers and rapists from routinely harvesting their passengers?
- polygamous_bat 4y agoExactly how you stop people from flagging a human driven taxi with plights for help and then robbing or raping the passenger. This doesn't seem like a problem specific to self driving. Relevant xkcd: https://xkcd.com/1958/ https://xkcd.com/1958/
- BurningFrog 4y agoA Taxi has a sober human driver with professional experience to handle such situations. A piece of software with some cameras and actuators is really not the same.
- mikedg 4y agoself driving car AI challenge, run into the cross walk and start waving every car through
- _carbyau_ 4y agoI agree with treat it as real. Then, if the car has an adult passenger you can ask them for their judgement (and responsibility for the outcome!) If the car has no passenger. Then stop and move slowly as required/indicated. Sure, it may be a car thief, but without a passenger, it is simply another car theft.
- dragonwriter 4y ago> When you see flashing lights on the highway coming up behind you, do you think “I bet that ambulance is faking it so they can get home faster”, or do you just move over? “You” in the specific or general sense? Because, my observations of highway behavior suggest that actual humans are…far from ideal at this.
- Bilal_io 4y agoNo need for human gesture, just put a stop sign or a road closed sign. These were likely taught to the model as its ABCs
- ghaff 4y agoAs a general principle, one can imagine special signals relevant to autonomous vehicles. But there's probably something of a chicken and egg sort of thing and I have problems with a lot of processes being required to deal with early adopters of this sort of technology.
- kelnos 4y agoIt kinda sucks, though, to put the responsibility for this on the first responders, who already have enough to do dealing with the emergency.
- nerdbert 4y agoThere are going to be times when someone is legitimately responding to an emergency and doesn't have a stop sign handy. And there are going to be a lot of pranksters who do, and will use them to mess with so-called self-driving cars.
- neuralRiot 4y agoOne thing that is often overlooked is that software can only understand what it was trained to understand.
- deleted 4y ago[deleted]
- __MatrixMan__ 4y agoThere's a connected vehicle protocol (DSRC) which broadcasts all kinds of metadata about your vehicle. There was stuff like "transaxle temperature" and "steering angle" in there, if I recall, so I'd be surprised if there wasn't also "emergency flashers engaged". Perhaps autonomous vehicles should be be sensitive to that field, signaling for manual control when they're too near a vehicle which has it enabled. That way any vehicle--not just the fire department--can create a "human control only" field by activating their emergency flashers. Perhaps emergency vehicles could be recognized such that they create a much larger field.
- krisoft 4y ago> But how do we distinguish real emergency scenario from fake one? You don’t. Bad actors can already pretend to be fireman doing fireman things and direct traffic. Human drivers will also be mislead by that. What is more, if you have a high viz vest, a hard hat, and more confidence than a garden slug you can also direct trafic. Not legally mind you, but more often than not people will follow your directions. If there is a whole gaggle of fake construction people and they have cones too they can probably re-route a highway. Whatever is stopping people from causing misschief (too often) by these “powers” doesn’t seem to be different in the case of self driving cars than with human driven cars. In fact the self-driving car is more likely to retain a perfect photographic evidence of the mischief maker’s face than a regular human driver.
- 8note 4y agoHow do people distinguish that? A person is likely to fall for a fake police officer and fake police car
- Rebelgecko 4y agoHave you seen any of the adversarial neural net stuff? I wouldn't be surprised if it was possible to trick a car into thinking someone is a police officer without formally impersonating a cop
- phkahler 4y ago>> but I wonder how long or what changes will have to be done in current road signs, critical city infra to support them. Unfortunately that attitude is common is the industry. The notion that the rest of the world needs to provide extra, up to date information for the shitty AI so it can function. I have news for ya. Emergency road repairs, fires, etc are not going to make it into your world database in a timely manner so a dumb vehicle can navigate the situation. It will all be over before anyone even thinks about fixing your system for you.
- treeman79 4y agoWas in backroads and ran into a cop directing travel. Apparently a small fair nearby. Dude was totally pissed and gesturing every which way wildly. When I couldn’t immediately decode. He just started swinging even more wildly. Turning redder and redder. Was fairly sure I was going to get shot arrested. So I just did a U turn. As I was leaving the next guy started yelling at cop that he was blocking all possible routes to his town.
- colechristensen 4y ago> Software is not ready for that yet, I think. It will never happen. Robots will never have the intelligence to manage dangerous situations in the open world, only in controlled environments. After a series of accidents passes the unforgivable threshold, certain levels of autonomous driving will be banned completely.
- cowmoo728 4y agoMaybe I'm too optimistic about self driving but it seems that a fail-safe default of "upon encountering an active emergency situation, stop at a reasonable distance away" is good enough and better than what a significant minority of human drivers do.
- ulizzle 4y agoWho are the significant minority? This is the same claim always given by the A.I boosters. Who sees firefighters fighting a fire and decides to just plow through and make matters worse? Because by "significant minority" what you are saying is that 2 or 3 out of 10 human drivers do this.
- colechristensen 4y agoExactly. The fact that some humans are drunks or psychopaths does not mean that AI mistakes are going to just be accepted. We send people to prison when they make those kinds of mistakes, what do we do when machine learning models kill while functioning entirely normally?
- j16sdiz 4y agoSend a low level engineer to prison, of course.
- ulizzle 4y agoThat's actually not as wild as you think, because that's what happens when a building or bridge falls. The contractor that built the bridge, just like the company that designed it, the agency responsible for inspections and maintenance, etc, is also liable for damages, including prison if it's deemed so by the courts. Of course, software engineers get treated with kid's gloves, otherwise, everyone just freaks out and cries and cries, because their robotic utopia dreams are meaningless nonsense. Apparently, smart enough to code, but not smart enough to take responsibility for their actions.
- BoorishBears 4y agoTeleoperations is a thing.
- deleted 4y ago[deleted]
- jimjimjim 4y agodriverless cars will have endless problems. When I was on sales trip to KL in Malaysia there were times when the tropical rain could half flood a street in minutes. Our driver was able to creep around the shallow side of the road (alternating with cars in the other direction). How would a driverless car cope? Would it even detect that there was problem? and if it detected it and wouldn't drive on that might be the worst thing to do if the flooding was going to get worse.
- c22 4y agoThere's a one-lane bridge near where I live that would pose a similar challenge. Drivers coming from the east always have right-of-way, but the only way to know this is by reading the bespoke traffic sign posted next to the bridge. To make matters worse, during times of heavy traffic westbound drivers will often stop and let through a stream of eastbound traffic. How many cars are let through is negotiated with hand signals or flashing lights and smiles.
- thatguy0900 4y agoWe will get the worst of both worlds, the car will tell you to take over in dangerous or confusing conditions but the person taking over will have minimal driving experience, on account of the car taking it all.
- KRAKRISMOTT 4y agoSimple, just transfer control over to a remote operator. We already do this for delivery rovers (see e.g. LA's coco rovers). Not an impossible problem unless all communication infrastructure is down (solar storm, hurricane, that sort of thing).
- bdcravens 4y agoDriverless is a huge waste. Most of Tesla's bad press (the company/product, not Elon) is due to Autopilot and/or FSD.