17 ms·
Replaying real life: how the Waymo Driver avoids fatal human crashes
- stefan_ 6y agoWhat do you call that when you subconsciously suppress any tests or scenarios you know will break your code? Like you notice somethings kinda racey but it involves other components you don't control and so you don't do the parallel tests you probably should, that kinda thing. Because that is exactly the bad feeling I get with Waymo never actually testing anywhere that doesn't have mm-precise maps and the all important California weather. There seems to be this implicit understanding that when you drop the thing in light snow it will just careen off the side and nobody is particularly incentivized to find that yep, that is exactly what happens. (Even in these simulated scenarios here, you got what looks like a barren wasteland and exactly two cars)
- Retric 6y agoWaymo is opening testing in Ohio right now, so it’s going to deal with terrible weather soon enough. https://www.limaohio.com/news/438416/waymo-to-open-ohio-test-site-for-driverless-vehicles https://www.limaohio.com/news/438416/waymo-to-open-ohio-test...
- ketzo 6y agoWell, devil's advocate here, but teaching a car to drive in good weather is hard enough, and has lots of applications! It doesn't seem totally unreasonable to focus on that first?
- sippeangelo 6y agoYes, if basing your entire model on visible road markings that is broken by a light coating of crystallized water.
- ra7 6y agoNot if the roads are mapped. You have prior knowledge of exactly where the markings are even if it’s not entirely visible due to rain.
- chrisseaton 6y agoIf the road markings are visible, do you use them? Or do you ignore them and never use them since they may sometimes be covered? The former? So makes sense for an automatic system to do the same? > a light coating of crystallized water Do you mean snow?
- swsieber 6y agoOh. Personally I learned how to drive in good weather first.
- ra7 6y agoThey have been testing in snow in Michigan, rain in Washington and a bunch of other places. You just don’t see it because they’re not operating a commercial service there yet. Arizona is where bulk of their operations is currently and where they offer a commercial service, so it makes sense for them to focus on that area in their safety reports.
- kadoban 6y agoOf course they're incentivized to test and implement for other areas. Their system will be worth astronomically more if it works everywhere and if it requires less precise mapping. Why would you assume otherwise? Also, small thing but it's AZ weather, not CA.
- GhostVII 6y agoSo long as they are intentionally constraining themselves to well mapped places with good weather, what's the problem? Summer tires don't work in the snow either, but that's fine because they aren't expected to.
- reitzensteinm 6y agoEven if you require precise mapping, that's not at all a deal breaker. Google went and mapped rural Kenya to make Google Maps. Mapping wealthy cities with >1 million population for a product with vastly more revenue potential is a trivial hurdle.
- bsder 6y ago> There seems to be this implicit understanding that when you drop the thing in light snow it will just careen off the side and nobody is particularly incentivized to find that yep, that is exactly what happens. Most people from Southern states will careen off the side when dropped into light snow--I'm exaggerating, but not by a lot. If the Waymo car does nothing else short of "maintain appropriate following distance" it is already ahead of 90% of all drivers in all weather.
- paulryanrogers 6y agoFigure 3 has the responder slowing unusually soon. Makes me wonder if their model is overly fit to the sample. Still, if it's genuinely that good at responding and mitigating then the future looks bright.
- Swizec 6y agoFigure 3 almost looks like the responder is lagging. Feels like it slows down and speeds up in several places. Assuming that’s not a gif/video rendering bug, it’s a great way to get rear-ended.
- saalweachter 6y agoHere's the semi-annual reminder to everyone that you are probably following too closely, and this is exactly why you shouldn't. The rule of thumb is that you want 3-4 seconds between cars, which at highway speeds is 80-100 meters or yards. Yes, almost no one leaves that much room. That's part of why so many accidents happen. It's easy for you to do better.
- Swizec 6y agoI leave plenty of space for this exact reason. Doubly important on a motorcycle because we have less grip. My point was more that the gif looks choppy and I don’t think that’s how the car actually drives. Imagine being a passenger in a car that brakes hard and accelerates every 2 meters while approaching an intersection.
- jeffbee 6y agoWaymo does not drive like that. It's smooth, actually. Watch this guy's channel for daily videos of how the driver behaves in practice. https://www.youtube.com/watch?v=eq-QndXs-nI https://www.youtube.com/watch?v=eq-QndXs-nI
- saalweachter 6y ago
- neolog 6y agoWhy limit simulations to fatal crashes instead of all reported crashes?
- clapper 6y agoHow could they recreate every crash that's happened? It seems the police only keep detailed records of the fatal crashes that happened there.
- jocker12 6y ago"The simulations were not conducted independently of the company, nor were they reviewed by any third-party for verification prior to the company making them public." from https://www.theverge.com/2021/3/8/22315361/waymo-autonomous-vehicle-simulation-car-crash-deaths https://www.theverge.com/2021/3/8/22315361/waymo-autonomous-...
- clapper 6y agoWhy is that relevant to my point? How can they do all crashes without knowing the specifics of each crash? "This research was made possible because of the thorough crash reports the Chandler Police Department and Arizona Department of Transportation (ADOT) prepare following fatal crashes, and we’re grateful to ADOT, Chandler Police Department, and Arizona Department of Public Safety for making these reports available to the public upon request."
- mikepurvis 6y agoThis gap in data is a bugbear for cycling/walking advocates such as myself who feel that non-vehicles do an unseen and unfair portion of the work of preventing crashes which might otherwise involve them. That they basically paper over a myriad of bad driver behaviours and ghastly infrastructure designs by always being on high alert and generally getting really good at "jumping out of the way." You can partly tell how effective this system is because when there are pedestrians killed, commentators express regret that the victim didn't just jump out of the way like a normal pedestrian, or imply that it was their fault for other reasons such as wearing the wrong clothing, looking at their device, or choosing an inappropriate time of day to go for a walk. All of this leads to attitude among drivers and politicians that current solutions are fine because the only numbers they have are literally deaths. Anyway, one would think with SDVs that there'd be an explosion in data for this kind of thing, since the cars will witness and participate in many more close calls than could ever be reported or tracked by anyone, much less the police. I wonder how much Google is able to process their car data for these kinds of things? Potentially it could be a huge aid, not even just for law enforcement, but also for urban planners looking for the close call hotspots.
- robbmorganf 6y ago1) Would a random driver (random steering and throttle) also have avoided these crashes? It seems like they were fatal because several variables coincided very exactly, and really any other input would have avoided them. 2) What input is the Waymo driver using here? If these were not originally Waymo cars, then is it perceiving simulated video or getting raw access to ground truth vehicle positions?
- asdfasgasdgasdg 6y agoFor 2, Waymo has a simulation environment with a whole simulated physics and sensor suite. They use this environment to train their driving systems on far more virtual miles than the number of physical miles they drive. So presumably they induce the reconstructed initial conditions from the crash and then they run the simulation a lot of times to see if their driver also gets into the crash.
- deleted 6y ago[deleted]
- tantalor 6y agoThey should try human drivers in a simulator. You would have to be careful not to signal when the accident was about to happen, because accidents in real life are very uncommon. You could estimate how avoidable an accident was by the percentage of human drivers in the simulated "responder" vehicle that avoided it. Maybe some accidents are easily avoided by humans because the real responder driver was distracted. I'm not impressed if Waymo driver can also avoid these. But some accidents may be very difficult to avoid for humans. If Waymo driver can perform better in these cases then it's truly remarkable.
- mindvirus 6y agoThis is really cool. I suspect that when we have self driving cars, it's going to take off very quickly due to insurance. In particular, if you are in a non self driving car and in an accident with a self driving one, it'll be hard to not be shown as at fault. So insurance I would guess go up based on the percentage of self driving cars, and likely be really cheap for self driving cars. I wonder if there's a period though where people cut off self driving cars, knowing they won't get hit. I suspect pedestrians in cities will jaywalk a lot more (and maybe that's ok).
- johntb86 6y agoOn the other hand, if self-driving cars are as good at avoiding accidents as the article says that may cause the price of insurance for non-self-driving cars to fall.
- depingus 6y agoI recall reading a few years ago that insurance companies are looking to partner with car manufactures. The idea is, when you purchase a full self-driving vehicle, insurance will be included with the cost of the car.
- hardtke 6y agoI suspect the adoption process will be slow because consumers really don't want this product. Self driving cars will drive the speed limit and infuriate both other motorists and their owners who are getting passed by speeding human driven cars. There will also be well publicized stories of the few times that self driving cars perform worse than humans. All drivers think they are above average and therefore the statistical comparison to the average driver does not apply to them.
- dawnerd 6y ago> because consumers really don't want this product > Self driving cars will drive the speed limit Tesla is proving both to be false, today.
- Animats 6y agoYou can read the California DMV's set of autonomous vehicle accident reports.[1] Almost all the Waymo reports are "vehicle was entering intersection, detected cross traffic, stopped, was rear-ended by human driver". There's one Waymo report where someone ran a stop sign and hit them. The rear-ending problem will be solved as automatic emergency braking becomes standard on cars. Already, it's shipping on almost all high-end cars and 70-80% of midrange cars. There's a US auto industry goal of it being standard by 2022. That's probably the main feature needed for self-driving cars to coexist with human-driven cars. [1] https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/autonomous-vehicle-collision-reports/ https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...
- spike021 6y ago>The rear-ending problem will be solved as automatic emergency braking becomes standard on cars. Already, it's shipping on almost all high-end cars and 70-80% of midrange cars. There's a US auto industry goal of it being standard by 2022. That's probably the main feature needed for self-driving cars to coexist with human-driven cars. I really wish we could just enforce proper education of being observant. Most times when I hear about people getting rear-ended it's because the driver behind them was looking at their phone or otherwise distracted. We shouldn't invent features that allow that to continue. On the other hand, I think this is fine in some rare cases, like maybe an accident or some other incident happens up ahead, a driver stomps their brakes, and then the person behind them who doesn't know what's happened doesn't stop soon enough. But even for the latter case, you could argue that the DMV says to keep enough distance where in most cases you should be able to still come to a stop with barely a moment's notice as long as you're paying enough attention. Of course, most people don't leave that much space much, if ever.
- Waterluvian 6y agoI support tackling this problem with both your solutions. But I certainly wouldn't bet my money on yours.
- derekp7 6y agoUnfortunately, current cars have to be controlled by humans, and the human brain has certain characteristics (selected for by evolution) that makes it near impossible for it to be able to react to 100% of the driving situations. There are built-in efficiency circuits (similar to CPU branch prediction) that takes immediate action based on what it typically sees. So if 99.9999% of the time, when a car in front starts to accelerate, and doesn't slam on the breaks immediately after, the brain makes the assumption that the care in front will never slam on the breaks and devotes less brain-CPU time to "Oh No, that care in front just slammed on the breaks" processing. In general, human drivers don't do this unless they are very inexperienced. You don't mash on the gas and immediately mash on the breaks, instead you make sure there is nothing coming. If you can't see well enough that there is something coming, then you ease up slowly and only mash on the gas when you have enough information that you won't get side swiped going through the intersection. And yes, intersections that have a "stop line" far enough back that you can't see cross traffic are poorly designed. Or if cars are really expected to stop every time before a crosswalk, then advance 2 feet to stop again, this needs better enforcement so that activity becomes normalized to the point where the car behind would expect that (one way of doing this is to put two stop signs, one before the sidewalk, and one at the intersection).
- starpilot 6y agoBut what about choosing between hitting one doctor versus a group of school kids? /s
- visarga 6y agoOther humans put in place of the ones who caused the accidents would probably have a similar high avoidance rate. It just shows Waymo was better than a small sample.
- tonyedgecombe 6y agoWould they spot someone running a red light at speed? I imagine most drivers are just waiting for green and doing nothing more than that.
- bogwog 6y agoThat's ridiculous. All humans make mistakes.
- bumbada 6y agoThey can use it to start fixing the roads, not the cars. Just look at the cross intersection they use as an example, typical of the US. The ROAD is the death trap there. People are crossing each other at high speed, this is the BUG, and should be fixed. The proper solution is putting there a roundabout, and you will have instantly less fatal accidents. The US was the first to use roundabouts, but those early designs did not work well, and as a result we have those monstrosities like in the picture all around the US. With electric cars that record accidents, the first bugs that we should fix are those in the roads, like those concrete barriers that have no smooth transition but a front wall out of nowhere.
- tonyedgecombe 6y agohttps://www.youtube.com/watch?v=Ra_0DgnJ1uQ https://www.youtube.com/watch?v=Ra_0DgnJ1uQ shows an alternative approach to regular accident spots.
- Laarlf 6y agoI live in Germany. We have some roundabouts. And let me tell you: they rarely work better than red lights. Traffic flow requires more than roundabouts. They require proper planning, green waves, proper sizing and still different lanes for roundabouts. Don’t forget about semis either. Just throwing roundabouts somewhere won’t solve anything. I’ve seen that often enough. That will just create more traffic jams.
- igorkraw 6y agoThe literature says differently, and is also aware of your opinion. https://www.safetylit.org/citations/index.php?fuseaction=citations.viewdetails&citationIds[]=citjournalarticle_63289_19 https://www.safetylit.org/citations/index.php?fuseaction=cit...
- Laarlf 6y agoUndersized roundabouts with 1 lane are not fitting for certain places. As said, I have seen many traffic lights being replaced with roundabouts. If you go to the Netherlands you see how roundabouts are executed well. Also Denmark has well made roundabouts. Just putting a small roundabout somewhere without putting thought into it is making things considerably worse.
- larsrc 6y agoThis shows (sort of) that the computer does better in situations where humans are known to fail. It's entirely likely that the computers will have new and unexpected failure modes that humans would never run into.
- elric 6y agoAnother thing that would avoid fatal crashes is reducing the number of cars on the roads, making them smaller and making them slower. That's a lot cheaper than making them more complex.
- Laarlf 6y agoNot leaving your house would solve a lot of things. Why don’t people do that? Anyway, slowing down traffic is not the answer. Driving should either be more automated or more engaging so drivers stop using their phones. 2 hours of driving 20 mph in a straight line will be boring enough to create more accidents.
- zip1234 6y agoIf cars were limited to a certain speed it would for certain cut down on crashes. It would also cut down on fatalities. When the news article says "speed was not a factor", they are only referring to the fact that the driver was going at or under the posted speed. That does not mean speed was not a factor. You think it would not work because driving is boring? Perhaps if you can't handle driving responsibly, you should not do it?
- Laarlf 6y agoEurope has built roads forever that makes driving engaging. Europe has a lot less issues with phones while driving etc. Germany also solves that problem by having no speed limit when it gets boring. Making driving engaging is the best way to create mindful drivers.
- ddp26 6y agoA big caveat to this study is that, in the reconstructed simulations, once the self-driving car (SDC) deviates from what the human-driven car did, the behavior of all other agents becomes unrealistic. Often in these sims, instead of reacting to the new SDC trajectory, they replay their original behavior. Ideally, once the SDC deviates, all other agents are simulated as well. A tall order, but necessary if these counterfactuals are to hold weight.
- bogwog 6y ago> Road safety is a major, global public health crisis. More than 1.3 million people die on the world’s roads every year, according to WHO. That’s more people than die from HIV/AIDS, and is equivalent to a passenger plane’s worth of people crashing every single hour—or one death every 30 seconds. > I’ve spent over 20 years working in crash avoidance research, in the belief that improved driving technology is the key to reducing these needless deaths. This is a noble goal, but I find it really damn hard to ignore the possibility that this is all just another ploy by Google to steal our data and violate our privacy. Just imagine trying to push back against a technology that saves lives for something as "frivolous" as privacy. Why does it seem like the consumer can never win? The free market is supposed to be self balancing, yet I can't remember a time where it actually felt that way in the technology sector.
- darkwizard42 6y agoI think if we want to solve large scale intersectional problems like traffic (where lots of individuals with their own goals and own variables intersect) we will need to give up SOME of our privacy in terms of where we are headed (just heading), speed, car model/make or some kind of capabilities estimate... This could result in a smarter traffic flow, road design, or in Google's case powering safer self-driving cars. I'm up for any of those outcomes and am also concerned about giving up data unnecessarily. I think the crux of the article besides hyping up self-driving simulated driver's better decision making is that roads aren't designed as safely as they could be and some minor inconvenience to the user (time, some data given up) could save lots of lives.
- Strilanc 6y agoI really dislike the way the data is being summarized as "100% avoided or mitigated*". First, they're omitting the types of case they're failing to mitigate (rear endings in this case). Cue "60% of the time it works every time" anchorman scene. Second, "mitigated" is defined to mean a 25% relative reduction in chance of serious injury. Which is good, but again smacks of "60% of the time it works every time". Basically, it looks to me like someone wanted to be able to show a bold blue 100% and required the definitions they were using to be massaged to match. And apparently they got their way. This has damaged my trust in future summaries put out by Waymo. A more truthful summarization style would be e.g. "X fewer serious injuries over 72 incidents", referring to expected injuries and simulated incidents of course.
- kevincox 6y agoI agree the results are excellent but the way they try to over-present them is very scummy. I don't see why they didn't think the results were good enough to stand on their own.