23 ms·

Waymo pulls back the curtain on 6.1M miles of self-driving car data in Phoenix

265 points by ra7 6y ago

ghaff 6y ago
>Waymo says its vehicles were involved in 47 “contact events” with other road users, including other vehicles, pedestrians, and cyclists. Eighteen of these events occurred in real life, while 29 were in simulation. “Nearly all” of these collisions were the fault of a human driver or pedestrian, Waymo says, and none resulted in any “severe or life-threatening injuries.” While Waymo's statements may have been technically true, the pedestrian blaming at least strikes me as a bit tone deaf. Of course, pedestrians can do really stupid things. But, generally speaking, drivers have a lot of responsibility to still avoid hitting them under most circumstances. ADDED: The language that caught my eye is the Verge's and Waymo is just the facts. But pedestrians and cyclists are in general concerning because it doesn't take much. Even though a a pedestrian walking into the side of a suddenly(?) stopped vehicle is probably not generally a bad outcome.
- hyperdimension 6y ago
  What a lovely euphemism, "contact events." I suppose Chernobyl was just a simple 'thermal event,' and we're only in the middle of a moderate 'disease spread elevation.' EDIT: "Sorry boss, there was a ton of traffic on my commute. Seems there was a 27-vehicle 'spontaneous gathering' on the highway"
  - keymone 6y ago
    Would you call it a crash if pedestrian walks into side of the car that stopped to avoid another collision?
- pkulak 6y ago
  Yup. A pedestrian can technically be "at fault" for using the side of an intersection parallel to the painted crosswalk. Getting run over by a 5000-lb minivan doesn't seem like fair punishment.
  - seibelj 6y ago
    In Boston, jay walking is just what everyone does. No one will blame the jay walker if they get run over - it is the driver's fault.
    - blobbers 6y ago
      I'm reminded of https://www.vox.com/xpress/2014/11/18/7236471/cars-pedestrians-sidewalks-roads https://www.vox.com/xpress/2014/11/18/7236471/cars-pedestria...
    - Grazester 6y ago
      If the driver was not breaking any laws when they hit the pedestrian I bet nothing will come of it however. In NYC it seems like this is usually the case.
    - loughnane 6y ago
      This is correct and I think it’s the healthy attitude of a pedestrian friendly city. This is the sort of bar self-driving firms should be setting.
    - chrisseaton 6y ago
      ‘Jay walking’ is such a bullshit term. ‘People trying to get across a road’ is what it is. Pedestrians should always have right of way.
      - fastball 6y ago
        Even when crossing roads where the speed limit is 40MPH+?
        ghaff 6y ago
        Yeah. I see the clowns in Cambridge who dart out into a poorly lit road at night not at a crosswalk wearing dark clothing. I'm watching for them because I know people do it. But the idea that a pedestrian can do anything they want to get across a medium speed road is idiotic.
- IggleSniggle 6y ago
  I agree with you, and yet at the same time I’m not sure how to word it for the general public so as not to be “tone deaf.” You would almost need to accompany such statements with video, or somehow humanize the actions of the car. A human driver can feel remorse and/or psychological trauma when someone commits suicide by traffic even if there WAS something they could have done, but didn’t react quickly or appropriately enough. Because of that, we can sympathize with the driver even if, in retrospect, there was some better action they could have taken. Generally we wouldn’t talk to the driver about it, because we understand it was traumatic for the driver. A computer does not get the same benefit of the doubt.
- javagram 6y ago
  It depends on how the pedestrian is violating the rules. But I hope that AV data will show that AVs are significantly better than human drivers at following pedestrian related rules such as automatically stopping to let pedestrians cross at crosswalks without signal lights. (As a driver, I’ve been guilty of not stopping in many cases myself, sometimes because I don’t look far enough ahead and to the side and notice the person waiting for cars to stop)
- kube-system 6y ago
  If we are truly talking about "contact events", I have seen on multiple occasions, pedestrians walking into the side of stopped vehicles. This is definitely an area where data speaks louder than words, so good on them for releasing it. Edit: skimming through the paper, it does appear that at least one of the events was exactly this scenario.
  - 0_____0 6y ago
    it's truly remarkable. granted this is an exceptional case given the general mental state of people at burning man, but I have seen a cyclist ride straight into the back of a fully parked, Very Brightly Lit art car before.
- spankalee 6y ago
  You're quoting The Verge's words, not Waymo's there. I'm curious how Waymo actually stated this.
  - judge2020 6y ago
    https://storage.googleapis.com/sdc-prod/v1/safety-report/Waymo-Public-Road-Safety-Performance-Data.pdf https://storage.googleapis.com/sdc-prod/v1/safety-report/Way... See page 5
- lern_too_spel 6y ago
  According to the paper, the pedestrian walked into the side of a stationary Waymo vehicle. I don't know how the vehicle can be at fault in this case.
- d0mine 6y ago
  there are ~110 fatalities per million miles in US (6 "contact events" per million miles (> half in simulation) looks good in comparison) https://www.statista.com/statistics/191641/traffic-related-fatalities-in-the-united-states-since-1990/ https://www.statista.com/statistics/191641/traffic-related-f...
  - maxerickson 6y ago
    There's ~1 fatality per 100 million miles driven. It looks like you've calculated a rate, as there are about 110 annual deaths per million people.
    - d0mine 6y ago
      correct. There 10000 times error: there is actually ~0.011 fatalities per million miles (not ~110)
  - imtringued 6y ago
    By those numbers I should have died ten times already.
    - sib 6y ago
      By those numbers, the average US driver (or passenger, I suppose) would die more than once per year... ;)
- jefftk 6y ago
  From the paper (https://storage.googleapis.com/sdc-prod/v1/safety-report/Waymo-Public-Road-Safety-Performance-Data.pdf https://storage.googleapis.com/sdc-prod/v1/safety-report/Way...): "There were also no collisions (actual or simulated) in which the Waymo Driver struck a pedestrian or cyclist. There were three events (one actual, two simulated) in which the Waymo vehicle was struck by a pedestrian or cyclist. In each instance, the Waymo Driver decelerated and stopped, and a pedestrian or cyclist made contact with the right side of the stationary Waymo vehicle while the pedestrian or cyclist was traveling at low speeds." "in each of the 3 actual and simulated events in which a pedestrian or cyclist struck the Waymo vehicle at low speed, the Waymo vehicle had decelerated and stopped immediately prior to the contact or simulated contact in a way that may have differed from the cyclist’s or pedestrian’s expectations. This illustrates a key challenge faced by AVs operating in a predominantly human traffic system and underscores the importance of driving in a way that is interpretable and predictable by other road users." (Disclosure: I work for Google, a sibling company of Waymo, speaking only for myself)
  - 0xB31B1B 6y ago
    As a cyclist, this says nothing to me about the safety of these cars. I was biking and I "struck" a car that was heading opposite my direction and making a left hand turn across my path in the bike line and crossing my right of way. I struck the car on the side near the front passenger wheel at "slow speed" and was thrown over the hood of the car and then onto the ground. This incident was the cars fault for making a turn without consideration of my right of way even though I, the cyclist, was the one who struck the car. An SDC that freezes in front of cyclists is still dangerous to cyclists.
  - alextheparrot 6y ago
    They just don’t drive like humans. I almost walked into the side of a Cruise vehicle a few years back because the car did a double stop at a stop sign when I expected it to keep moving forward after the first stop. There was no other traffic and I wasn’t distracted - but it just stopped in the middle of the crosswalk after starting to go forward. Pedestrians assume certain things that are not necessarily true for self-driving vehicles, just as if I started acting erratically as a pedestrian it would really mess with cars. All in all, though, I’d much rather take that failure case (At least when walking) over the human failure case of hitting me while turning.
- ljhsiung 6y ago
  Firstly, this isn't direct at you, more at the title. I think the title of "47 events" is greatly misleading. While the simulations are definitely incredibly valuable, it's misleading to use the figures from simulations in any argument for how careless Waymo is. In fact, I think the heavy usage of simulation to model realistic events should be a plus, rather than how the title makes it seem like a bad thing. Secondly, If you read Table 1 [1] of the paper, you'll see that there was only 1 actual pedestrian event (and I guess 2 simulated events), and that was a person walking into a parked car (it's funny that they could measure the walking speed of the person at 2.7mph). Lastly, in Table 2 they enumerate all the events that "were the fault of the other human driver". Most of them are "failure to yield", for example-- "failure to yield to a vehicle approaching from the left while making a right turn at an unsignalized intersection." This touches on your bit on driver's responsibility. While it is true that Waymo isn't at fault, it makes me wonder how much "defensive driving" Waymo is including in their models. In the case above, without video/full context, you don't fully know whether it was an avoidable accent or not by, say, swerving to the other lane, or if that option wasn't possible due to another car. It'd be incredibly interesting to know the exact number corrective actions taken/attempted and number of probable accidents avoided. [1] https://storage.googleapis.com/sdc-prod/v1/safety-report/Waymo-Public-Road-Safety-Performance-Data.pdf https://storage.googleapis.com/sdc-prod/v1/safety-report/Way...
  - ghaff 6y ago
    Fair enough. I did not read the actual paper. My bad. >that was a person walking into a parked car (it's funny that they could measure the walking speed of the person at 2.7mph). And heh. I can't say the car's likely at fault in that case although it may still have behaved in a way the pedestrian didn't expect. And as someone else noted, some of the language I was quoting was from The Verge. The actual report is much more just the facts. To your broader point, a human driver could probably obey every traffic law and get into tons of at least minor accidents that would be technically the other driver's fault.
  - JackC 6y ago
    > I think the title of "47 events" is greatly misleading If I understand correctly, it's not the title that's misleading but the term "simulation." They're only talking about simulating what would have happened if the safety driver didn't intervene in real life, not simulations generally. A better word would be predictions -- they predict that there would have been 47 events, but safety drivers were able to prevent the majority of them.
- chrisseaton 6y ago
  > the pedestrian blaming at least strikes me as a bit tone deaf. Of course, pedestrians can do really stupid things. But, generally speaking, drivers have a lot of responsibility to still avoid hitting them under most circumstances Can you explain exactly what you think a driver is supposed to do to stop a pedestrian literally just walking into your stationary car? If you’re responsible for avoiding that, you tell me what you would have done?
  - sidibe 6y ago
    By making the car act more like people expect it to From the paper (which doesn't really pedestrian-blame as GP suggests) > In each instance, the Waymo vehicle had decelerated and stopped immediately prior to the contact or simulated contact in a way that may have differed from the cyclist’s or pedestrian’s expectations
    - chrisseaton 6y ago
      But > responsibility to still avoid hitting them They didn't hit them at all!
      - Dylan16807 6y ago
        Okay? But that wasn't the question you asked. Part of the answer to your question is that cars should neither hit pedestrians nor do erratic things that set up a collision.
    - michael1999 6y ago
      People also expect drivers to kill 20k per year. I don't think Waymo is likely to sign up to that goal whole-heartedly.
  - alextheparrot 6y ago
    If you’ve ever almost walked into one the failure case is really clear. They use stopping as a default mitigation strategy when something unpredictable happens - this isn’t always a good solution especially when suddenly blocking transit (Crosswalks, bike paths, railways, traffic) instead of just crossing it. This is a failure case humans can get cited for - I hit a car in the bike lane once when they did a left turn in front of me and it was legally the driver’s fault for blocking that path. It is just another characteristic they need to figure out in a complex problem.
- heavyset_go 6y ago
  I'm from an area where pedestrians have the right of way everywhere, so this isn't reassuring.
withinboredom 6y ago
I wonder if the AI is going to overfit traffic patterns and behavior. The rest of the world doesn’t drive like they’re in Phoenix.
- scep12 6y ago
  I think it's reasonable to expect that overfitting is on the radar for Waymo (no pun intended) given that 1) their parent/sibling company's expertise in the domain and 2) that they've been extensively training outside of Phoenix both in real-world and simulation. To continue down this path, I wonder how easy/hard it is for the Waymo "driver" to adjust to driving on the other side of the road.
  - stevesimmons 6y ago
    > driving on the other side of the road just flip the X axis of the camera input!
- IggleSniggle 6y ago
  I think the mid-game probably needs to be regional behavior fitting anyway.
- jeffbee 6y ago
  Is there any evidence that the Waymo approach involves "the AI" at all? I think you're projecting the approach of other, failed self-driving efforts onto Waymo.
  - computerphage 6y ago
    Are you suggesting that Waymo might not use machine learning? What do you imagine they do instead?
    - jeffbee 6y ago
      No, I understand they use NNs for object classification and so forth, but compare their approach to Tesla, where the overall architecture is camera → a miracle → steering actuators. It seems from the outside that Waymo relies less on the black box.
OnlineGladiator 6y ago
> Waymo says its vehicles were involved in 47 “contact events” with other road users, including other vehicles, pedestrians, and cyclists. Eighteen of these events occurred in real life, while 29 were in simulation. If they drove 6,100,000 real miles and ~5,000,000,000 simulation miles, and had 18 real contacts and ~29 simulated contacts - that strongly indicates their simulation is not representative of reality. Edit: My (old, now changed) numbers were wrong but my theory was right! So they drove 6,100,000 real miles and had 18 real contacts. Between June 2019 and April 2020 they drove ~5,000,000,000 simulated miles and had ~29 simulated contacts. So their simulator is 3 orders of magnitude safer than the real world. The only number that's really fuzzy here is 29 - the number of simulated contacts (because the dates don't match up perfectly, as this report from Waymo started counting in January 2019). But no matter how you slice it, there is clearly no way the simulated miles are representative of reality. https://techcrunch.com/2019/07/10/waymo-has-now-driven-10-billion-autonomous-miles-in-simulation/ https://techcrunch.com/2019/07/10/waymo-has-now-driven-10-bi... https://blog.waymo.com/2020/04/off-road-but-not-offline--simulation27.html https://blog.waymo.com/2020/04/off-road-but-not-offline--sim...
- spankalee 6y ago
  That's not what the article says. They drove 6.1 million miles autonomous with a safety driver.
- jartelt 6y ago
  The 6.1M miles was self-driving in real life, with a safety driver in the car. The simulation portion only comes into play if the safety driver intervenes in a real life situation. In this case, they go back and simulate what would have happened in the safety driver did nothing.
- mamon 6y ago
  Where did you get those numbers from? according to the article 6.1M miles was driven on the roads with safety drivers, and additional 65000 miles were driven without safety drivers. "Simulated contacts" are when safety drivers engages, and Waymo investigates what would happen if they did not, so this is as close to the real life as it gets.
317070 6y ago
Does someone know what a "simulated collision" is? How is it relevant? Or are those still collisions in the real world? EDIT: it is in the article, I read it too fast: > The company says it also counts events in which its trained safety drivers assume control of the vehicle to avoid a collision. Waymo’s engineers then simulate what would have happened had the driver not disengaged the vehicle’s self-driving system to generate a counterfactual, or “what if,” scenario. The company uses these events to examine how the vehicle would have reacted and then uses that data to improve its self-driving software. Ultimately, these counterfactual simulations can be “significantly more realistic” than simulated events that are generated “synthetically,” Waymo says.
- adrr 6y ago
  Simulated collision is an important stat. It shows how much improvement the AI needs to obtain to be on-par with a human driver.
  - uluyol 6y ago
    Almost, it should how much improvement the AI needs to avoid collisions a human driver did avoid. The AI might avoid collisions that the human didn't. In other words, you need 0 simulated collisions to be strictly better than a human driver (in the same setting). Not ~equal with sometimes better and sometimes worse.
- rconti 6y ago
  I still don't understand "in simulation", though, because of this line: > But the company highlighted eight incidents that it considered “most severe or potentially severe.” Three of these crashes occurred in real life and five only in simulation. Airbags were deployed in all eight incidents.
  - rightbyte 6y ago
    That is quite confusing, yes. Did they mean 5 simulated air bags?
jeffbee 6y ago
18 crashes in 6 million miles is pretty good. Middle-aged human drivers cause that rate of police-reportable crashes, and one injury per million miles.
- ilaksh 6y ago
  But Waymo did not cause the crashes.
mdorazio 6y ago
It's great to see actual data reported since so many AV companies hide it behind as many walls and fluffy press releases as possible. I think this speaks to the maturity of Waymo's system at this point. Two key paragraphs for me personally: "The most common type of crash involving Waymo’s vehicles was rear-end collisions. Waymo said it was involved in 14 actual and two simulated fender-benders, and in all but one, the other vehicle was the one doing the rear-ending." -> This is consistent with previous reports from Waymo. If one of their vehicles gets in an accident, it's extremely likely that the other (human) driver was at fault. Of course, this is somewhat compounded by the fact that Waymo vehicles actually try to follow road laws like attempting to stop for yellow lights and making full stops at stopsigns, which human drivers often don't do (and may not expect other vehicles to do). "The one incident where Waymo rear-ended another vehicle was in simulation: the company determined that the AV would have rear-ended another car that swerved in front of it and then braked hard despite a lack of obstruction ahead — which the company says was “consistent with antagonistic motive.” (There have been dozens of reports of Waymo’s autonomous vehicles being harassed by other drivers, including attempts to run them off the road.)" -> This doesn't surprise me really, but is still pretty sad. Many drivers are just assholes and AVs are going to be an easy target for such people.
- xxpor 6y ago
  >which human drivers often don't do (and may not expect other vehicles to do). This presents an interesting problem. It's obviously easier to program something to follow the law, given it's unambiguous. But the question is what are we optimizing for? The fewest crashes? That's probably the right thing to do given crashes are bad. In that case, isn't it better to do what people would expect other cars to do? But are Waymo constrained by the fact that if a self-driving car is programmed to get a ticket they could be held liable? Probably. I think the moral side of self-driving cars is just as hard or a harder problem than the technical side, and we haven't made the decisions as a society that we need to. If the government doesn't step up soon to lay out how this is going to work, the corporations will. And guess what: they'll choose whatever costs them the least amount of money. Not what's best for society.
  - 6y ago
mola 6y ago
When I cross the road (to get to the other side) I use eye contact as a very strong signal for my safety. I wonder what will replace this.
- JoshTko 6y ago
  Waymo can signal that it sees you by slowing down a good distance away.
harlanji 6y ago
Waymo does simulations? I have to update my surveillence program design. Pretty sure they’re able to track me, and do. I’m a vocal activist against their data collection, have approached them and taken photos to make the drivers uncomfortable and tweeted about marking them with red paintballs to indicate recording (goal: make them signal their collection activity). Literally every day I am greeted by patterns of Waymo vans, like 2 the moment I leave and at the majoroty of my destinations one+ buzzes me. They box me in at stoplights, park at my destinations. I know it sounds crazy, but a pretty simple program with a few compartmentalized functions would enable the legal team to feed my phone carrier loction stream into the dispatching system; could also be an element of prediction as well. None of the drivers or engineers would know what they’re building. The end result would be a surveillance system that can follow targets. The diagram is just pencilled on paper, but I’ve been deep enough in Silicon Valley to say it’s plausible. I also made a short story on SoundCloud, same handle. What’s the economic value of this data? An element of my message is also that they’re profiting off of homeless people who can’t opt out; eg. they could be targets for training a prediction system, since they will always be available and be in the same few locations. I want an opt-out function. I’ve been meaning to fugure out how to CCPA request their data, with my photo, license plate, photos of my car and bike. Lawyer in the house? Biz@harlanji.com
maliker 6y ago
That's around 7.71 crashes/million miles driven by Waymo. By comparison, US national statistics from 2018 [1] are around 2.08 crashes/million miles driven. Seems like they're far from human performance. There are ton of ways to slice this data, though. I used national stats for accidents where the police showed up; maybe Waymo's data includes no-police accidents. It would also be interesting to compare injury statistics. And of course this is Phoenix-specific data from Waymo, which is a place with good road conditions. [1] https://cdan.nhtsa.gov/tsftables/National%20Statistics.pdf https://cdan.nhtsa.gov/tsftables/National%20Statistics.pdf (edited to include simulated crashes, which were only averted through human safety driver intervention.)
- brixon 6y ago
  Did you count all incidents or only the ones that happened in real life and not the simulations?
  - maliker 6y ago
    I omitted the simulations in the Waymo count. Good idea to add them in?
    - IshKebab 6y ago
      Yeah you should include them because they were only prevented by a safety driver who won't exist in the long term.
      - maliker 6y ago
        Thanks! I missed that definition in the article. Updated the ratios. So it's actually 3.7x more accidents not 1.5x. Definitely changed my conclusion.
  - danielvf 6y ago
    Simulations counts were times when a safety driver took control of the vechile, and post-incident simulations with the same data show that the AI would have crashed the car without the safety driver. So fair to include them.
csours 6y ago
If you dig in and watch the vehicles, it's clear that the next barrier is communication with other road users pedestrians, cyclists, drivers (and passengers). Pedestrians don't know where the car is trying to go or when it will start or stop. Drivers get annoyed and brake check the vehicles. Drivers don't understand what the car is about to do next and drive erratically to get around it.
- jeffbee 6y ago
  People who "get annoyed and brake check the vehicles" should be apprehended and launched into the sun. There's no excuse for behavior like that.
hacker245678910 6y ago
wowwwww
deleted 6y ago
[deleted]
Hitton 6y ago
Looks nice, but I'm little sceptical about the rear end collisions. From previous videos of self-driving cars (I don't remember whether it was Waymo) I have seen, they tend to suddenly stop for seemingly no reason. And although the driver has duty to keep safe distance, from point of view of other drivers it might seem no different than "anatagonistic motive" mentioned in the article.
- ilaksh 6y ago
  Check out JJ Ricks's recent Waymo vehicles. There is no evidence of that being the case today.
  - zaroth 6y ago
    The evidence is the rear-end collisions cited by OP.
Animats 6y ago
That's comparable to the autonomous vehicle accident reports Google files with the California DMV. Minor rear-ending at low speed is the most common problem. The usual situation is where the Waymo vehicle has an obstructed line of sight at an intersection and enters the intersection very slowly. Then the system sees cross traffic and stops. The human-driven vehicle, following the Waymo vehicle, then fails to stop fast enough. There's one intersection in Mountain View which has a tree in the median high enough to block the LIDAR but clear at window height. That causes a very cautious intersection entrance. Waymo vehicles have been rear-ended twice there. As LIDAR units get cheaper and more are fitted, that problems should be fixed. Autonomous vehicles may need a "Back Off" signal, like flashing the brake lights at ultra-bright levels when someone gets too close. "The one actual, non-simulated angled collision occurred when a vehicle ran a red light at 36 mph, smashing into the side of a Waymo vehicle that was traveling through the intersection at 38 mph." Not Waymo's fault. I wonder if Waymo went to court, with full video of the driver running the red light. This is encouraging.
OliverGilan 6y ago
I'm not well versed in the different self driving techniques but isn't 6.1 miles a little low compared to say Tesla which is estimated to have autopilot miles in the billions [0]? [0]: https://lexfridman.com/tesla-autopilot-miles-and-vehicles/#:~:text=The%20following%20is%20a%20plot,Tesla%20vehicles%3A%2022.5%20billion%20miles https://lexfridman.com/tesla-autopilot-miles-and-vehicles/#:...
- forgot-my-pw 6y ago
  Waymo did 6.1M miles in Phoenix, Arizona alone. Waymo did a lot of simulated driving, they claim about 200M miles a day: https://blog.waymo.com/2020/04/off-road-but-not-offline--simulation27.html https://blog.waymo.com/2020/04/off-road-but-not-offline--sim...
loosescrews 6y ago
The severity scale used in the paper is backwards. Their S0 means "no injury expected" and their S3 means "possible critical injuries expected". Google's Issue Tracker [1] uses the opposite scale [2]. [1] https://issuetracker.google.com/ https://issuetracker.google.com/ [2] https://developers.google.com/issue-tracker/concepts/issues https://developers.google.com/issue-tracker/concepts/issues
- abirkill 6y ago
  They're using the ISO 26262 scale for severity, not their own: https://en.wikipedia.org/wiki/ISO_26262#Part_9:_Automotive_Safety_Integrity_Level_(ASIL)-oriented_and_safety-oriented_analysis https://en.wikipedia.org/wiki/ISO_26262#Part_9:_Automotive_S...
just_steve_h 6y ago
I believe that the data-collection should be publicly escrowed for safety-critical research such as for autonomous vehicles, therapeutics, and so on. It's too easy for employees to subtly edit or shape data to please their managers / directors / shareholders. It can be as subtle as "cleaning" the data by removing records with "noisy" or "unreliable" data, or "correcting" outliers. In the Waymo case, I consider self-disclosure to be self-serving.
ummonk 6y ago
Waymo cars could be following the law and still be extremely difficult for other drivers to deal with. E.g. I don't know how much they've improved since but a few years back I was in a short merge for my daily commute and there was a Waymo car in the lane I was trying to merge into which just drove parallel to me without speeding up or slowing down as most human drivers do to make way. This was the most difficult merge I had experienced on that spot in years of driving that same commute.
Jabbles 6y ago
"Only" 74,000 miles in driverless mode. That is less than I expected. Executive Summary https://storage.googleapis.com/sdc-prod/v1/safety-report/Waymo-Safety-Methodologies-and-Readiness-Determinations.pdf https://storage.googleapis.com/sdc-prod/v1/safety-report/Way...
- bgee 6y ago
  Google engineer not working on Waymo, opinions are my own. I think "driverless" here is different from "self-driving", i.e. 'Waymo uses “driverless” here to refer to operations in which the ADS controls the vehicle for the entire trip without a human driver (whether in the vehicle or at a remote location) expected to assume any part of the driving task.' at page 5 of that report. I think this driverless mode is only available to public on Oct 8th [0]. [0]: https://blog.waymo.com/2020/10/waymo-is-opening-its-fully-driverless.html https://blog.waymo.com/2020/10/waymo-is-opening-its-fully-dr...
  - Jabbles 6y ago
    That's how I interpreted it too, but I'm still surprised it's not 10x. I would have thought that paying the drivers is a major expense, and collecting "real" data 24/7 would be a priority.
warxmike 6y ago
Geohot is gonna love this one
twilio 6y ago
> Many of Waymo’s competitors, including Argo, Aurora, Cruise, Zoox, Nuro Since these are the only competitors that are directly mentioned, is it safe to say that they're closest in performance to Waymo?

265 points by ra7 6y ago

>Waymo says its vehicles were involved in 47 “contact events” with other road users, including other vehicles, pedestrians, and cyclists. Eighteen of these events occurred in real life, while 29 were in simulation. “Nearly all” of these collisions were the fault of a human driver or pedestrian, Waymo says, and none resulted in any “severe or life-threatening injuries.” While Waymo's statements may have been technically true, the pedestrian blaming at least strikes me as a bit tone deaf. Of course, pedestrians can do really stupid things. But, generally speaking, drivers have a lot of responsibility to still avoid hitting them under most circumstances. ADDED: The language that caught my eye is the Verge's and Waymo is just the facts. But pedestrians and cyclists are in general concerning because it doesn't take much. Even though a a pedestrian walking into the side of a suddenly(?) stopped vehicle is probably not generally a bad outcome.

hyperdimension 6y ago
What a lovely euphemism, "contact events." I suppose Chernobyl was just a simple 'thermal event,' and we're only in the middle of a moderate 'disease spread elevation.' EDIT: "Sorry boss, there was a ton of traffic on my commute. Seems there was a 27-vehicle 'spontaneous gathering' on the highway"
- keymone 6y ago
  Would you call it a crash if pedestrian walks into side of the car that stopped to avoid another collision?
pkulak 6y ago
Yup. A pedestrian can technically be "at fault" for using the side of an intersection parallel to the painted crosswalk. Getting run over by a 5000-lb minivan doesn't seem like fair punishment.
- seibelj 6y ago
  In Boston, jay walking is just what everyone does. No one will blame the jay walker if they get run over - it is the driver's fault.
  - blobbers 6y ago
    I'm reminded of https://www.vox.com/xpress/2014/11/18/7236471/cars-pedestrians-sidewalks-roads https://www.vox.com/xpress/2014/11/18/7236471/cars-pedestria...
  - Grazester 6y ago
    If the driver was not breaking any laws when they hit the pedestrian I bet nothing will come of it however. In NYC it seems like this is usually the case.
  - loughnane 6y ago
    This is correct and I think it’s the healthy attitude of a pedestrian friendly city. This is the sort of bar self-driving firms should be setting.
  - chrisseaton 6y ago
    ‘Jay walking’ is such a bullshit term. ‘People trying to get across a road’ is what it is. Pedestrians should always have right of way.
    - fastball 6y ago
      Even when crossing roads where the speed limit is 40MPH+?
      - ghaff 6y ago
        Yeah. I see the clowns in Cambridge who dart out into a poorly lit road at night not at a crosswalk wearing dark clothing. I'm watching for them because I know people do it. But the idea that a pedestrian can do anything they want to get across a medium speed road is idiotic.
IggleSniggle 6y ago
I agree with you, and yet at the same time I’m not sure how to word it for the general public so as not to be “tone deaf.” You would almost need to accompany such statements with video, or somehow humanize the actions of the car. A human driver can feel remorse and/or psychological trauma when someone commits suicide by traffic even if there WAS something they could have done, but didn’t react quickly or appropriately enough. Because of that, we can sympathize with the driver even if, in retrospect, there was some better action they could have taken. Generally we wouldn’t talk to the driver about it, because we understand it was traumatic for the driver. A computer does not get the same benefit of the doubt.
javagram 6y ago
It depends on how the pedestrian is violating the rules. But I hope that AV data will show that AVs are significantly better than human drivers at following pedestrian related rules such as automatically stopping to let pedestrians cross at crosswalks without signal lights. (As a driver, I’ve been guilty of not stopping in many cases myself, sometimes because I don’t look far enough ahead and to the side and notice the person waiting for cars to stop)
kube-system 6y ago
If we are truly talking about "contact events", I have seen on multiple occasions, pedestrians walking into the side of stopped vehicles. This is definitely an area where data speaks louder than words, so good on them for releasing it. Edit: skimming through the paper, it does appear that at least one of the events was exactly this scenario.
- 0_____0 6y ago
  it's truly remarkable. granted this is an exceptional case given the general mental state of people at burning man, but I have seen a cyclist ride straight into the back of a fully parked, Very Brightly Lit art car before.
spankalee 6y ago
You're quoting The Verge's words, not Waymo's there. I'm curious how Waymo actually stated this.
- judge2020 6y ago
  https://storage.googleapis.com/sdc-prod/v1/safety-report/Waymo-Public-Road-Safety-Performance-Data.pdf https://storage.googleapis.com/sdc-prod/v1/safety-report/Way... See page 5
lern_too_spel 6y ago
According to the paper, the pedestrian walked into the side of a stationary Waymo vehicle. I don't know how the vehicle can be at fault in this case.
d0mine 6y ago
there are ~110 fatalities per million miles in US (6 "contact events" per million miles (> half in simulation) looks good in comparison) https://www.statista.com/statistics/191641/traffic-related-fatalities-in-the-united-states-since-1990/ https://www.statista.com/statistics/191641/traffic-related-f...
- maxerickson 6y ago
  There's ~1 fatality per 100 million miles driven. It looks like you've calculated a rate, as there are about 110 annual deaths per million people.
  - d0mine 6y ago
    correct. There 10000 times error: there is actually ~0.011 fatalities per million miles (not ~110)
- imtringued 6y ago
  By those numbers I should have died ten times already.
  - sib 6y ago
    By those numbers, the average US driver (or passenger, I suppose) would die more than once per year... ;)
jefftk 6y ago
From the paper (https://storage.googleapis.com/sdc-prod/v1/safety-report/Waymo-Public-Road-Safety-Performance-Data.pdf https://storage.googleapis.com/sdc-prod/v1/safety-report/Way...): "There were also no collisions (actual or simulated) in which the Waymo Driver struck a pedestrian or cyclist. There were three events (one actual, two simulated) in which the Waymo vehicle was struck by a pedestrian or cyclist. In each instance, the Waymo Driver decelerated and stopped, and a pedestrian or cyclist made contact with the right side of the stationary Waymo vehicle while the pedestrian or cyclist was traveling at low speeds." "in each of the 3 actual and simulated events in which a pedestrian or cyclist struck the Waymo vehicle at low speed, the Waymo vehicle had decelerated and stopped immediately prior to the contact or simulated contact in a way that may have differed from the cyclist’s or pedestrian’s expectations. This illustrates a key challenge faced by AVs operating in a predominantly human traffic system and underscores the importance of driving in a way that is interpretable and predictable by other road users." (Disclosure: I work for Google, a sibling company of Waymo, speaking only for myself)
- 0xB31B1B 6y ago
  As a cyclist, this says nothing to me about the safety of these cars. I was biking and I "struck" a car that was heading opposite my direction and making a left hand turn across my path in the bike line and crossing my right of way. I struck the car on the side near the front passenger wheel at "slow speed" and was thrown over the hood of the car and then onto the ground. This incident was the cars fault for making a turn without consideration of my right of way even though I, the cyclist, was the one who struck the car. An SDC that freezes in front of cyclists is still dangerous to cyclists.
- alextheparrot 6y ago
  They just don’t drive like humans. I almost walked into the side of a Cruise vehicle a few years back because the car did a double stop at a stop sign when I expected it to keep moving forward after the first stop. There was no other traffic and I wasn’t distracted - but it just stopped in the middle of the crosswalk after starting to go forward. Pedestrians assume certain things that are not necessarily true for self-driving vehicles, just as if I started acting erratically as a pedestrian it would really mess with cars. All in all, though, I’d much rather take that failure case (At least when walking) over the human failure case of hitting me while turning.
ljhsiung 6y ago
Firstly, this isn't direct at you, more at the title. I think the title of "47 events" is greatly misleading. While the simulations are definitely incredibly valuable, it's misleading to use the figures from simulations in any argument for how careless Waymo is. In fact, I think the heavy usage of simulation to model realistic events should be a plus, rather than how the title makes it seem like a bad thing. Secondly, If you read Table 1 [1] of the paper, you'll see that there was only 1 actual pedestrian event (and I guess 2 simulated events), and that was a person walking into a parked car (it's funny that they could measure the walking speed of the person at 2.7mph). Lastly, in Table 2 they enumerate all the events that "were the fault of the other human driver". Most of them are "failure to yield", for example-- "failure to yield to a vehicle approaching from the left while making a right turn at an unsignalized intersection." This touches on your bit on driver's responsibility. While it is true that Waymo isn't at fault, it makes me wonder how much "defensive driving" Waymo is including in their models. In the case above, without video/full context, you don't fully know whether it was an avoidable accent or not by, say, swerving to the other lane, or if that option wasn't possible due to another car. It'd be incredibly interesting to know the exact number corrective actions taken/attempted and number of probable accidents avoided. [1] https://storage.googleapis.com/sdc-prod/v1/safety-report/Waymo-Public-Road-Safety-Performance-Data.pdf https://storage.googleapis.com/sdc-prod/v1/safety-report/Way...
- ghaff 6y ago
  Fair enough. I did not read the actual paper. My bad. >that was a person walking into a parked car (it's funny that they could measure the walking speed of the person at 2.7mph). And heh. I can't say the car's likely at fault in that case although it may still have behaved in a way the pedestrian didn't expect. And as someone else noted, some of the language I was quoting was from The Verge. The actual report is much more just the facts. To your broader point, a human driver could probably obey every traffic law and get into tons of at least minor accidents that would be technically the other driver's fault.
- JackC 6y ago
  > I think the title of "47 events" is greatly misleading If I understand correctly, it's not the title that's misleading but the term "simulation." They're only talking about simulating what would have happened if the safety driver didn't intervene in real life, not simulations generally. A better word would be predictions -- they predict that there would have been 47 events, but safety drivers were able to prevent the majority of them.
chrisseaton 6y ago
> the pedestrian blaming at least strikes me as a bit tone deaf. Of course, pedestrians can do really stupid things. But, generally speaking, drivers have a lot of responsibility to still avoid hitting them under most circumstances Can you explain exactly what you think a driver is supposed to do to stop a pedestrian literally just walking into your stationary car? If you’re responsible for avoiding that, you tell me what you would have done?
- sidibe 6y ago
  By making the car act more like people expect it to From the paper (which doesn't really pedestrian-blame as GP suggests) > In each instance, the Waymo vehicle had decelerated and stopped immediately prior to the contact or simulated contact in a way that may have differed from the cyclist’s or pedestrian’s expectations
  - chrisseaton 6y ago
    But > responsibility to still avoid hitting them They didn't hit them at all!
    - Dylan16807 6y ago
      Okay? But that wasn't the question you asked. Part of the answer to your question is that cars should neither hit pedestrians nor do erratic things that set up a collision.
  - michael1999 6y ago
    People also expect drivers to kill 20k per year. I don't think Waymo is likely to sign up to that goal whole-heartedly.
- alextheparrot 6y ago
  If you’ve ever almost walked into one the failure case is really clear. They use stopping as a default mitigation strategy when something unpredictable happens - this isn’t always a good solution especially when suddenly blocking transit (Crosswalks, bike paths, railways, traffic) instead of just crossing it. This is a failure case humans can get cited for - I hit a car in the bike lane once when they did a left turn in front of me and it was legally the driver’s fault for blocking that path. It is just another characteristic they need to figure out in a complex problem.
heavyset_go 6y ago
I'm from an area where pedestrians have the right of way everywhere, so this isn't reassuring.

withinboredom 6y ago

I wonder if the AI is going to overfit traffic patterns and behavior. The rest of the world doesn’t drive like they’re in Phoenix.

scep12 6y ago
I think it's reasonable to expect that overfitting is on the radar for Waymo (no pun intended) given that 1) their parent/sibling company's expertise in the domain and 2) that they've been extensively training outside of Phoenix both in real-world and simulation. To continue down this path, I wonder how easy/hard it is for the Waymo "driver" to adjust to driving on the other side of the road.
- stevesimmons 6y ago
  > driving on the other side of the road just flip the X axis of the camera input!
IggleSniggle 6y ago
I think the mid-game probably needs to be regional behavior fitting anyway.
jeffbee 6y ago
Is there any evidence that the Waymo approach involves "the AI" at all? I think you're projecting the approach of other, failed self-driving efforts onto Waymo.
- computerphage 6y ago
  Are you suggesting that Waymo might not use machine learning? What do you imagine they do instead?
  - jeffbee 6y ago
    No, I understand they use NNs for object classification and so forth, but compare their approach to Tesla, where the overall architecture is camera → a miracle → steering actuators. It seems from the outside that Waymo relies less on the black box.

OnlineGladiator 6y ago

> Waymo says its vehicles were involved in 47 “contact events” with other road users, including other vehicles, pedestrians, and cyclists. Eighteen of these events occurred in real life, while 29 were in simulation. If they drove 6,100,000 real miles and ~5,000,000,000 simulation miles, and had 18 real contacts and ~29 simulated contacts - that strongly indicates their simulation is not representative of reality. Edit: My (old, now changed) numbers were wrong but my theory was right! So they drove 6,100,000 real miles and had 18 real contacts. Between June 2019 and April 2020 they drove ~5,000,000,000 simulated miles and had ~29 simulated contacts. So their simulator is 3 orders of magnitude safer than the real world. The only number that's really fuzzy here is 29 - the number of simulated contacts (because the dates don't match up perfectly, as this report from Waymo started counting in January 2019). But no matter how you slice it, there is clearly no way the simulated miles are representative of reality. https://techcrunch.com/2019/07/10/waymo-has-now-driven-10-billion-autonomous-miles-in-simulation/ https://techcrunch.com/2019/07/10/waymo-has-now-driven-10-bi... https://blog.waymo.com/2020/04/off-road-but-not-offline--simulation27.html https://blog.waymo.com/2020/04/off-road-but-not-offline--sim...

spankalee 6y ago
That's not what the article says. They drove 6.1 million miles autonomous with a safety driver.
jartelt 6y ago
The 6.1M miles was self-driving in real life, with a safety driver in the car. The simulation portion only comes into play if the safety driver intervenes in a real life situation. In this case, they go back and simulate what would have happened in the safety driver did nothing.
mamon 6y ago
Where did you get those numbers from? according to the article 6.1M miles was driven on the roads with safety drivers, and additional 65000 miles were driven without safety drivers. "Simulated contacts" are when safety drivers engages, and Waymo investigates what would happen if they did not, so this is as close to the real life as it gets.

317070 6y ago

Does someone know what a "simulated collision" is? How is it relevant? Or are those still collisions in the real world? EDIT: it is in the article, I read it too fast: > The company says it also counts events in which its trained safety drivers assume control of the vehicle to avoid a collision. Waymo’s engineers then simulate what would have happened had the driver not disengaged the vehicle’s self-driving system to generate a counterfactual, or “what if,” scenario. The company uses these events to examine how the vehicle would have reacted and then uses that data to improve its self-driving software. Ultimately, these counterfactual simulations can be “significantly more realistic” than simulated events that are generated “synthetically,” Waymo says.

adrr 6y ago
Simulated collision is an important stat. It shows how much improvement the AI needs to obtain to be on-par with a human driver.
- uluyol 6y ago
  Almost, it should how much improvement the AI needs to avoid collisions a human driver did avoid. The AI might avoid collisions that the human didn't. In other words, you need 0 simulated collisions to be strictly better than a human driver (in the same setting). Not ~equal with sometimes better and sometimes worse.
rconti 6y ago
I still don't understand "in simulation", though, because of this line: > But the company highlighted eight incidents that it considered “most severe or potentially severe.” Three of these crashes occurred in real life and five only in simulation. Airbags were deployed in all eight incidents.
- rightbyte 6y ago
  That is quite confusing, yes. Did they mean 5 simulated air bags?

jeffbee 6y ago

18 crashes in 6 million miles is pretty good. Middle-aged human drivers cause that rate of police-reportable crashes, and one injury per million miles.

ilaksh 6y ago
But Waymo did not cause the crashes.

mdorazio 6y ago

It's great to see actual data reported since so many AV companies hide it behind as many walls and fluffy press releases as possible. I think this speaks to the maturity of Waymo's system at this point. Two key paragraphs for me personally: "The most common type of crash involving Waymo’s vehicles was rear-end collisions. Waymo said it was involved in 14 actual and two simulated fender-benders, and in all but one, the other vehicle was the one doing the rear-ending." -> This is consistent with previous reports from Waymo. If one of their vehicles gets in an accident, it's extremely likely that the other (human) driver was at fault. Of course, this is somewhat compounded by the fact that Waymo vehicles actually try to follow road laws like attempting to stop for yellow lights and making full stops at stopsigns, which human drivers often don't do (and may not expect other vehicles to do). "The one incident where Waymo rear-ended another vehicle was in simulation: the company determined that the AV would have rear-ended another car that swerved in front of it and then braked hard despite a lack of obstruction ahead — which the company says was “consistent with antagonistic motive.” (There have been dozens of reports of Waymo’s autonomous vehicles being harassed by other drivers, including attempts to run them off the road.)" -> This doesn't surprise me really, but is still pretty sad. Many drivers are just assholes and AVs are going to be an easy target for such people.

xxpor 6y ago
>which human drivers often don't do (and may not expect other vehicles to do). This presents an interesting problem. It's obviously easier to program something to follow the law, given it's unambiguous. But the question is what are we optimizing for? The fewest crashes? That's probably the right thing to do given crashes are bad. In that case, isn't it better to do what people would expect other cars to do? But are Waymo constrained by the fact that if a self-driving car is programmed to get a ticket they could be held liable? Probably. I think the moral side of self-driving cars is just as hard or a harder problem than the technical side, and we haven't made the decisions as a society that we need to. If the government doesn't step up soon to lay out how this is going to work, the corporations will. And guess what: they'll choose whatever costs them the least amount of money. Not what's best for society.
- 6y ago

mola 6y ago

When I cross the road (to get to the other side) I use eye contact as a very strong signal for my safety. I wonder what will replace this.

JoshTko 6y ago
Waymo can signal that it sees you by slowing down a good distance away.

harlanji 6y ago

Waymo does simulations? I have to update my surveillence program design. Pretty sure they’re able to track me, and do. I’m a vocal activist against their data collection, have approached them and taken photos to make the drivers uncomfortable and tweeted about marking them with red paintballs to indicate recording (goal: make them signal their collection activity). Literally every day I am greeted by patterns of Waymo vans, like 2 the moment I leave and at the majoroty of my destinations one+ buzzes me. They box me in at stoplights, park at my destinations. I know it sounds crazy, but a pretty simple program with a few compartmentalized functions would enable the legal team to feed my phone carrier loction stream into the dispatching system; could also be an element of prediction as well. None of the drivers or engineers would know what they’re building. The end result would be a surveillance system that can follow targets. The diagram is just pencilled on paper, but I’ve been deep enough in Silicon Valley to say it’s plausible. I also made a short story on SoundCloud, same handle. What’s the economic value of this data? An element of my message is also that they’re profiting off of homeless people who can’t opt out; eg. they could be targets for training a prediction system, since they will always be available and be in the same few locations. I want an opt-out function. I’ve been meaning to fugure out how to CCPA request their data, with my photo, license plate, photos of my car and bike. Lawyer in the house? Biz@harlanji.com

maliker 6y ago

That's around 7.71 crashes/million miles driven by Waymo. By comparison, US national statistics from 2018 [1] are around 2.08 crashes/million miles driven. Seems like they're far from human performance. There are ton of ways to slice this data, though. I used national stats for accidents where the police showed up; maybe Waymo's data includes no-police accidents. It would also be interesting to compare injury statistics. And of course this is Phoenix-specific data from Waymo, which is a place with good road conditions. [1] https://cdan.nhtsa.gov/tsftables/National%20Statistics.pdf https://cdan.nhtsa.gov/tsftables/National%20Statistics.pdf (edited to include simulated crashes, which were only averted through human safety driver intervention.)

brixon 6y ago
Did you count all incidents or only the ones that happened in real life and not the simulations?
- maliker 6y ago
  I omitted the simulations in the Waymo count. Good idea to add them in?
  - IshKebab 6y ago
    Yeah you should include them because they were only prevented by a safety driver who won't exist in the long term.
    - maliker 6y ago
      Thanks! I missed that definition in the article. Updated the ratios. So it's actually 3.7x more accidents not 1.5x. Definitely changed my conclusion.
- danielvf 6y ago
  Simulations counts were times when a safety driver took control of the vechile, and post-incident simulations with the same data show that the AI would have crashed the car without the safety driver. So fair to include them.

csours 6y ago

If you dig in and watch the vehicles, it's clear that the next barrier is communication with other road users pedestrians, cyclists, drivers (and passengers). Pedestrians don't know where the car is trying to go or when it will start or stop. Drivers get annoyed and brake check the vehicles. Drivers don't understand what the car is about to do next and drive erratically to get around it.

jeffbee 6y ago
People who "get annoyed and brake check the vehicles" should be apprehended and launched into the sun. There's no excuse for behavior like that.

hacker245678910 6y ago

wowwwww

deleted 6y ago

[deleted]

Hitton 6y ago

Looks nice, but I'm little sceptical about the rear end collisions. From previous videos of self-driving cars (I don't remember whether it was Waymo) I have seen, they tend to suddenly stop for seemingly no reason. And although the driver has duty to keep safe distance, from point of view of other drivers it might seem no different than "anatagonistic motive" mentioned in the article.

ilaksh 6y ago
Check out JJ Ricks's recent Waymo vehicles. There is no evidence of that being the case today.
- zaroth 6y ago
  The evidence is the rear-end collisions cited by OP.

Animats 6y ago

That's comparable to the autonomous vehicle accident reports Google files with the California DMV. Minor rear-ending at low speed is the most common problem. The usual situation is where the Waymo vehicle has an obstructed line of sight at an intersection and enters the intersection very slowly. Then the system sees cross traffic and stops. The human-driven vehicle, following the Waymo vehicle, then fails to stop fast enough. There's one intersection in Mountain View which has a tree in the median high enough to block the LIDAR but clear at window height. That causes a very cautious intersection entrance. Waymo vehicles have been rear-ended twice there. As LIDAR units get cheaper and more are fitted, that problems should be fixed. Autonomous vehicles may need a "Back Off" signal, like flashing the brake lights at ultra-bright levels when someone gets too close. "The one actual, non-simulated angled collision occurred when a vehicle ran a red light at 36 mph, smashing into the side of a Waymo vehicle that was traveling through the intersection at 38 mph." Not Waymo's fault. I wonder if Waymo went to court, with full video of the driver running the red light. This is encouraging.

OliverGilan 6y ago

I'm not well versed in the different self driving techniques but isn't 6.1 miles a little low compared to say Tesla which is estimated to have autopilot miles in the billions [0]? [0]: https://lexfridman.com/tesla-autopilot-miles-and-vehicles/#:~:text=The%20following%20is%20a%20plot,Tesla%20vehicles%3A%2022.5%20billion%20miles https://lexfridman.com/tesla-autopilot-miles-and-vehicles/#:...

forgot-my-pw 6y ago
Waymo did 6.1M miles in Phoenix, Arizona alone. Waymo did a lot of simulated driving, they claim about 200M miles a day: https://blog.waymo.com/2020/04/off-road-but-not-offline--simulation27.html https://blog.waymo.com/2020/04/off-road-but-not-offline--sim...

loosescrews 6y ago

The severity scale used in the paper is backwards. Their S0 means "no injury expected" and their S3 means "possible critical injuries expected". Google's Issue Tracker [1] uses the opposite scale [2]. [1] https://issuetracker.google.com/ https://issuetracker.google.com/ [2] https://developers.google.com/issue-tracker/concepts/issues https://developers.google.com/issue-tracker/concepts/issues

abirkill 6y ago
They're using the ISO 26262 scale for severity, not their own: https://en.wikipedia.org/wiki/ISO_26262#Part_9:_Automotive_Safety_Integrity_Level_(ASIL)-oriented_and_safety-oriented_analysis https://en.wikipedia.org/wiki/ISO_26262#Part_9:_Automotive_S...

just_steve_h 6y ago

I believe that the data-collection should be publicly escrowed for safety-critical research such as for autonomous vehicles, therapeutics, and so on. It's too easy for employees to subtly edit or shape data to please their managers / directors / shareholders. It can be as subtle as "cleaning" the data by removing records with "noisy" or "unreliable" data, or "correcting" outliers. In the Waymo case, I consider self-disclosure to be self-serving.

ummonk 6y ago

Waymo cars could be following the law and still be extremely difficult for other drivers to deal with. E.g. I don't know how much they've improved since but a few years back I was in a short merge for my daily commute and there was a Waymo car in the lane I was trying to merge into which just drove parallel to me without speeding up or slowing down as most human drivers do to make way. This was the most difficult merge I had experienced on that spot in years of driving that same commute.

Jabbles 6y ago

"Only" 74,000 miles in driverless mode. That is less than I expected. Executive Summary https://storage.googleapis.com/sdc-prod/v1/safety-report/Waymo-Safety-Methodologies-and-Readiness-Determinations.pdf https://storage.googleapis.com/sdc-prod/v1/safety-report/Way...

bgee 6y ago
Google engineer not working on Waymo, opinions are my own. I think "driverless" here is different from "self-driving", i.e. 'Waymo uses “driverless” here to refer to operations in which the ADS controls the vehicle for the entire trip without a human driver (whether in the vehicle or at a remote location) expected to assume any part of the driving task.' at page 5 of that report. I think this driverless mode is only available to public on Oct 8th [0]. [0]: https://blog.waymo.com/2020/10/waymo-is-opening-its-fully-driverless.html https://blog.waymo.com/2020/10/waymo-is-opening-its-fully-dr...
- Jabbles 6y ago
  That's how I interpreted it too, but I'm still surprised it's not 10x. I would have thought that paying the drivers is a major expense, and collecting "real" data 24/7 would be a priority.

warxmike 6y ago

Geohot is gonna love this one

twilio 6y ago

> Many of Waymo’s competitors, including Argo, Aurora, Cruise, Zoox, Nuro Since these are the only competitors that are directly mentioned, is it safe to say that they're closest in performance to Waymo?