19 ms·
Normalized crash data shows Autopilot is much less safe than Tesla claims
- _ea1k 5y agoIMO, the headline is a boring and editorialized take on the situation. Even when the data is normalized in this way, active safety only is 55 crashes per 100M miles and AP is ~42 crashes per 100M miles in the most recent quarter. While the most recent quarter could be an outlier, some other quarters show a similarly dramatic difference.
- gnicholas 5y agoUnderlying study is here. [1] Full disclosure, I co-wrote the piece for The Daily Beast [2] that originally suggested that Tesla's methodology was seriously flawed. I am not a Tesla-hater though — I just thought it was odd that the company was playing quite so fast and loose with their safety claims. 1: https://engrxiv.org/preprint/view/1973/3986 https://engrxiv.org/preprint/view/1973/3986 2: https://www.thedailybeast.com/how-tesla-and-elon-musk-exaggeraged-safety-claims-about-autopilot-and-cars https://www.thedailybeast.com/how-tesla-and-elon-musk-exagge...
- Natsu 5y agoThe study says this: > An estimated 15% of injury crashes and 24% of property damage-only crashes are never reported to police (M. Davis and Company, Inc., 2015), while9% of injury crashes and 24% of property damage-only crashes are reported but not logged (Blincoe et al., 2015). Even with robust data, establishing the statistical significance of automated vehicle safety can be expensive. Kalra and Paddock (2016)demonstrated that establishing that an AV has a fatal crash rate equivalent to the national average with 95% confidence would require driving a fleet of 100 vehicles continuously for 12.5 years. I presume there's an adjustment for this in the figures you compare to, but I am having trouble finding it, though that may be my own failure so I wanted to ask if you could help me find how that's adjusted for?
- TedDoesntTalk 5y agoWhy would injury crashes go unreported?
- sokoloff 5y agoIf I had a single-car crash with minor injuries, why would I report it? It seems like there’s limited upside to me as the (obviously at-fault) driver. Go get medical care and go get my car fixed. (I drive cars cheap enough to replace and so do not insure them against collisions that are my fault.) Drivers driving without a valid license, without insurance, under the influence, wanted by the law, etc would have all kinds of reasons to not report a crash.
- deleted 5y ago[deleted]
- vanattab 5y agoI got hit a few weeks ago while walking my dog (dog was fine). I thought I had just bruised my tailbone so I waved the driver on after a short chat and never reported it. In hindsight I wish I had as it seems like my tailbone was fractured and not just bruised but it definitely happens.
- LorenPechtel 5y agoPolice responding to the actual crash scene, useful. Police report filed after the fact, only if there's some need for said report. And not all injuries are immediately apparent. 3 1/2 years ago I got a first-person view of what it would be like to be on the receiving end of a PIT maneuver courtesy of a woman who didn't look adequately. (Yes, when you looked left the road was empty--I saw that also and turned into that empty spot the block before you!) At the time the only injury I was aware of was a very minor scrape, not even worthy of a bandaid. Later I discovered a pulled muscle--it only hurt when I did certain things. I also used to know a woman who thought it was nothing more than a fender-bender, went to the doc a couple of days later because her neck was bothering her. Doc sent her straight to the hospital--incomplete cervical fracture, one wrong move and she could have dropped dead.
- jacquesm 5y ago> I just thought it was odd that the company was playing quite so fast and loose with their safety claims. That's because it's marketing, not science.
- sandworm101 5y agoAnd because it's Tesla, not Honda. Every car company does marketing. Few make over-the-top safety claims about their products. This is the company that decided to have its cars not stop at stop signs. Normal rules don't apply to Tesla. https://www.bbc.com/news/technology-60230072 https://www.bbc.com/news/technology-60230072 "In recall documents, the electric vehicle maker notes that if certain conditions are met, the "rolling stop" feature is designed to allow the vehicle to travel through all-way-stop intersections at up to 5.6mph without coming to a complete stop."
- omgwtfbyobbq 5y agoIt's not like Honda doesn't engage in puffery. Most retailers make over-the-top claims regarding their products, including safety. https://www.latimes.com/business/autos/la-fi-honda-fine-20150109-story.html https://www.latimes.com/business/autos/la-fi-honda-fine-2015...
- NikolaNovak 5y agoI mean... I am "Tesla Hater" in that I don't like the direction they're proceeding in; but every human being I ever met in real actual life does rolling stops under some conditions. Last time I can confidently state that "I never rolled through a stop at very slow speed" is when I was 17, and only NOW do I realize how much of an annoying bum I was :P. So if a car does a rolling stop through a stop sign when there are no other cars obstacles or people, my thought is 1. yes that's a breach of law as written 2. It's what everybody else safely does. Kind of like in many jurisdictions, police will gently talk to you if you drive 60kph on the freeway, or even if you're doing 90kph in the left lane. Yes it's legal, no it isn't safe.
- sandworm101 5y ago
- andrepd 5y ago>I am not a Tesla-hater though — I just thought it was odd that the company was playing quite so fast and loose with their safety claims. If you're still surprised by this you've not been paying attention to how Telsa operates.
- mdoms 5y agoHe never said he was surprised.
- maxdo 5y ago
- simplyaccont 5y ago>Facts: Tesla was the first to introduce such wide range of safety features in every car they sold. They don't make safety a premium feature. They invest in own crash test facilities. Only Volvo is kind of on par. Volvo built their crash test facilities before Tesla came to existence.
- FireBeyond 5y ago> Facts: Tesla was the first to introduce such wide range of safety features in every car they sold. Please itemize these "wide range of safety features" for us that other manufacturers do not have or did not have until Tesla innovated them. > They invest in own crash test facilities. Only Volvo is kind of on par. Audi owns one, for just one.
- deleted 5y ago[deleted]
- maxdo 5y agoSure, https://www.tesla.com/safety https://www.tesla.com/safety https://www.youtube.com/watch?v=9KR2N_Q8ep8 https://www.youtube.com/watch?v=9KR2N_Q8ep8 Tesla roll out cars in generations. So just take a $40k Model 3/Y platform was introduced in 2017 and compare it with any car been introduced that year. My friend bought a BMW in 2019 for similar money, it doesn't have half of that. Maybe you can have them but you have to buy extra packages for extra money. Tesla don't sell safety features for extra price. The only extra feature is Autopilot. But hardware, camera's , lane departure avoidance, blind spot collision warning etc in every car. You can even take such a simple feature as forward collision warning system. When you drive it is super accurate it uses the same data and models from Autopilot to predict when there is a crash. If your slowing down dynamic is not good or the car in front/back change it dynamics it will warn you immediately. Only this feature saved me several times from collision. I'm not a good driver. As I said in 2022 there are many car brands that catching up in some cases they doing better. And this is good. Human is a bad, constantly distracted driver. Technology is the only way to make our life safer.
- contravariant 5y agoSo to summarize: - Freeways account for 93% of distance travelled with Autopilot - Freeways account for 28% of the distance travelled by regular drivers. - "vehicles on non-freeways crashed 2.01 times more often per mile" a quick back of the envelop calculation suggests merely driving on the freeway lowers accidents per mile by ~40%, which accounts for most of the difference between Autopilot and regular drivers.
- avs733 5y agoso in sum, Thomas Bayes just kicked Tesla square in the ass?
- nwiswell 5y agoSquare in the frunk.
- adolph 5y agoThere are additional confounding variables noted in the paper: - Age of driver - Specific highways used (congestion, road conditions) - What is classified as a highway Overall this is pretty good news for "autopilot" systems. It gives some humans an ego-boost as they clutch to steering wheels like John Henry. It gives others interested in a more relaxed driving experience the confidence that the computer isn't going to kill them any worse than they already do.
- AtlasBarfed 5y agoAnd you know what? All I want out of the first fifteen years of self-driving is a really good highway autopilot. As in one I can hopefully sleep at the wheel on a late night trip between midwestern US cities. John Henryism can navigate the suburbs, stroads, and drivethrus. If they achieve a backup safety system that monitors the humans, well, ok great. But right now, I want long distance automation, and I can handle the messy last mile. Highways are much easier to get uniform infrastructure convergence: some sensors, PSA warnings, better signs + markings + construction notices, etc. I think Uber making as big a splash at it did sidetracked this low hanging fruit. IMO that was where the bootstrap bucks were, because you would have gotten freight/shipping to pay the cost of the next gen software, and would have provided the impetus for the convergent infrastructure. Alas, they are obsessed with robotaxis.
- deleted 5y ago[deleted]
- RyEgswuCsn 5y agoSo it sounds like it's Simpson's paradox [1] at work. I had always wondered why many people are slow to accept self-driving cars despite the claims of them being statistically safer. I think that explains it --- to make consumers confident about the self-driving technologies, it is not enough for them to be able to handle the "easy" kind of driving (e.g. on highway and under relatively good visibility conditions) better than humans; they need to demonstrate that they can drive better than humans in the most challenging driving environments too. I guess this is the classic scenario where human intuition is defeated by carefully presented statistics. [1]: https://en.wikipedia.org/wiki/Simpson%27s_paradox https://en.wikipedia.org/wiki/Simpson%27s_paradox
- wilde 5y agoI think traditional auto makers have done a much better job of communicating to consumers what the tech can actually do. We don’t have self driving cars (which implies that they can drive in those harder conditions). We have cars that are good at lane keeping on freeways. I wish Tesla would celebrate that victory rather than double down on lies.
- deleted 5y ago[deleted]
- oblio 5y agoThe thing is, you don't even need Tesla "self driving" for that. Adaptive cruise control, tech we've had for what, 20+ years? is more than enough for most highways, it takes out ~80% of the stress/boredom of driving, especially since lane changes are minimal and curves are gentle. The hard parts of driving (aka not highways) are the real problem, as expected. We solved the easy parts in 2000 and who knows if we'll solve the hard parts by 2050.
- CSSer 5y agoI have a car with adaptive cruise control and lane assist, and most of my morning commute (on days when I go into the office) is driven on the highway. Most of the time I don't turn it on because I'm in a self-perceived hurry to get to work, so I forget. You know what would be really nice? Something that detects when it's safe and reasonable to suggest turning on adaptive cruise control so I use it more. Maybe something like detecting I've driven a few miles at a consistent rate of speed above a certain threshold. The hard parts are definitely the problem, but there are lots of easy wins left on the table too.
- julianz 5y agoSo that Daily Beast article has been up for more than 5 years with nobody fixing the horrible typo in the headline? Wow.
- rosndo 5y agoDaily Beast, not the Economist.
- ricardobeat 5y ago> naturalistic driving study of 28 vehicles The study seems to extrapolate from this. Tesla has something like 10 billion miles total driven on Autopilot, on a fleet of a million cars.
- dsl 5y agoThis is the dataset: https://insight.shrp2nds.us/documents/shrp2_background.pdf https://insight.shrp2nds.us/documents/shrp2_background.pdf It covers 5.4 million trips in 3,400 vehicles with digitally collected records and vehicle data and has been independently validated by the Virginia Tech Transportation Institute and Department of Transportation.
- ricardobeat 5y agoThe 93% highway driving figure, which is used to normalize Tesla numbers, and then compare agains the NHS dataset, is coming from that 28 Tesla cars study from 2018, or did I read that wrong?
- Robotbeat 5y agoEven after adjusting, it does look like autopilot is slightly safer (although not a strong enough effect to say for sure). It doesn't appear to be worse, so good enough to allow it, IMHO. I actually wonder if there's another effect: Drivers only eligible for beta FSD if they meet a super high safety score (which isn't a perfect metric, but probably better than nothing). That incentivizes good driving even when FSD not engaged. Of course, that's a recent thing and hasn't shown up in data.
- ubercore 5y agoAfter correcting for age, it becomes less safe
- lordnacho 5y agoWow, adjusted for driver age the difference vanishes. That's quite an interesting observation, and it shows the importance of proper statistics. It also raises questions about companies producing their own stats, it's often only when someone takes the time to dig into it that we discover this kind of thing. I am not holding my breath waiting for the common consumer to understand statistical critical thinking though. I sometimes catch myself forgetting the due diligence even after many years of forcing myself to think about stats. Perhaps some sort of ombudsman could do this, pointing out when stats are lying to people.
- djanogo 5y agoThey should adjust for car-age and price.
- ProAm 5y agoIf you have taken a course in statistics you'll know its always possible to make the numbers looks favorable to your cause. It would take several unrelated blind third parties to do this correctly and no public company will ever let that happen, especially Tesla.
- kadoban 5y agoThey should be forced to let that happen. It's crucial for public safety.
- ProAm 5y agoPublic knowledge of safety affects stock price and brand recognition. Will never happen.
- gretch 5y agoThis is an arrogant amount of speculation based on nothing more than personal intuition. In fact, there are plenty of examples where public knowledge of safety is published by mandate of the government. Here is an example of FAA flight data and you can see details about incidents across airlines: https://www.faa.gov/data_research/accident_incident/ https://www.faa.gov/data_research/accident_incident/ I'm not going to spend time digging it up, but I'm pretty sure there are just as many sources of data on other important things such as pharmaceuticals e.g. the covid vaccine. So no, just because 'stock price go down' doesn't mean a thing will never happen
- gameswithgo 5y agoEDIT: After reading the full paper and not just the twitter graphs, some of what I said is wrong. That is, they also controlled for age, and when controlling for age and road type, autpilot was just worse. On the one hand I had the same suspicion about the unadjusted data, on the other hand the fact that autopilot isn't worse is pretty promising! However, average human rates include people driving drunk, tired, elders, teenagers etc. So at 43 when I am driving sober, I'm probably safer not using autopilot. When I was 19 probably safer to use autopilot.
- gnicholas 5y ago> the fact that autopilot isn't worse is pretty promising! Presumably drivers take over when it makes mistakes, which tilts the stats a bit. > When I was 19 probably safer to use autopilot. As a parent, I wonder about what safety tech I would let my kids use when learning to drive. I want to make sure they fully learn how to drive, which makes me think I shouldn't let them use too much semi-autonomous tech. At the same time, it seems foolish to tell them not to use blind spot monitors, or to even expect them to "check their blind spot" the old-fashioned way if their cars have monitors. Interestingly, my father (in his 70s) tells me he'll never trust blind spot monitors, so I'm seeing a generational difference even between him and me. Too bad, since older drivers probably could benefit the most from not having to look back when changing lanes. Older eyes take longer to re-focus, so it's more likely an older driver would miss something happening ahead on account of having looked rearward.
- rzimmerman 5y agoI also think it will be important for any future teenager of mine to learn to drive without the regenerative braking that Teslas default to. It’s a much better driving experience IMO, but I worry a new driver won’t learn the “oh no hit the brakes!” reflex if they rarely use the brake pedal. I’d probably force them to learn with the car set to “normal” ICE/idle—style for the first few months.
- Psyonic 5y agoIs there actually a "normal" style? I've only done a test drive but it seemed like there were various levels of automatic braking, but none that actually allowed you to coast.
- rurp 5y agoI'm not too surprised by this and bet a lot of HN folks suspected that this was the case, but I'm really glad to see this come out and hope it gets a lot of visibility. Controlling for confounding variables is a complicated and often subtle process that many people don't have an intuitive grasp of, so Elon's BS claims have probably been very convincing for many.
- endisneigh 5y agoAre there stats that even show that any self driving system is safer than the median driver? Basically accidents involve two people. Said accidents have responsibility divided between both individuals. A hypothetically perfect driver could still be involved in fatal accidents by virtue of the other driver. The worse drivers obviously will cause these accidents. The questions are three fold: 1. Are self driving vehicles safer than the median driver? 2. What’s the ratio between at fault and not at fault for any accidents the median driver gets in? 3. What’s the same ratio for self driving cars? Normalizes for location, of course.
- M2Ys4U 5y ago>Basically accidents involve two people. Not always, no. It's perfectly possible to crash a car with nobody else around.
- bagels 5y agoOr for one car to crash in to many cars.
- spywaregorilla 5y agoThe median driver is a difficult thing to study. 77% of people have experienced an accident, but I think fewer than 50% of people have been at fault for one. That's not to say they wouldn't be at fault if they drove more, but the median has a clean record. Hard to know who to exclude.
- yrral 5y agoHow about the predominately male ownership of teslas causing higher accident rates. I see the paper adjusts for age and road type but missing gender is a pretty big slip-up? I didn't read the paper fully but grepped for "gender" and "sex" and couldn't find anything.
- gnicholas 5y agoGood question! According to one study in LA, [1] men cause 60% of accidents. I do wonder, however, the extent to which this gap fades with age. Given that most Tesla owners are not super young, it might not make as much of a difference as the 60% number (assuming it is nationally representative) would indicate. 1: https://xtown.la/2019/12/12/women-drivers-maybe-we-need-more-of-them/ https://xtown.la/2019/12/12/women-drivers-maybe-we-need-more...
- yrral 5y agoNote when looking at 40% vs 60% of accidents (assuming males and females drive the same amount) the % difference seems small, only 20%, but actually this means males cause 50% more accidents than females do (60/40=1.5).
- user123abc 5y ago[flagged]
- nlowell 5y agoI don't know if we can assume males and females drive the same amount.
- user123abc 5y agoMen also drive 63% more miles than women. https://www.carinsurance.com/Articles/average-miles-driven-per-year-by-state.aspx https://www.carinsurance.com/Articles/average-miles-driven-p...
- smnrchrds 5y agoKeep in mind that men drive 16,500 miles per year on average, while women drive 10,100 miles. There is a noticeable gender gap in "accidents per year", but not in "accidents per mile". The paper compares accidents per mile, so there should be little gender effect in the results. https://www.fhwa.dot.gov/ohim/onh00/bar8.htm https://www.fhwa.dot.gov/ohim/onh00/bar8.htm
- deleted 5y ago[deleted]
- borski 5y agoWe need a startup that can test and correct dubious statistical claims, or verify good ones. I would definitely pay for that.
- Someone1234 5y agoThat would be cool, but companies are often not forthcoming with the raw data needed to do such analysis independently (particularly if it could make them look bad).
- borski 5y agoSure, but would they be if they could be “independently verified” by a startup like this? It might be a PR win.
- JoeAltmaier 5y agoIs it eggregiously worse than normal human pilots? If not, then that's an accomplishment in itself.
- spywaregorilla 5y agoIt's marginally better
- JoeAltmaier 5y agoRight! A few years ago, claiming that a robot car would drive as safely as a human driver would have been an incredible claim. Just saying.
- thomaskcr 5y agoIt would have been, Musk didn't think so and felt the need to embellish/exaggerate - his habit of exaggerating/lying I think is more on trial than the actual Tesla vehicles here.
- brk 5y agoNobody working in machine vision is surprised by this. We are a LONG way from having self-driving vehicles as a general thing, particularly when Tesla wants to rely on MV as their source of environment analysis.
- starwind 5y agoMy last job we did CV mostly focused on satellite images. When someone on the team mentioned she had a Tesla the director who set up the team was like "you see the results we get here. Why do you feel safe getting in a self-driving car?"
- paxys 5y agoI have felt since day 1 that "autopilot is better than the average driver!" is the most misleading fact/statistic in the industry. The "average driver" involved in an accident includes drunk drivers, road ragers, habitual speeders, teenagers, people driving in bad conditions (rain, snow), people driving without sleep and several other risky groups that most people buying Teslas aren't part of. To be worth it, the car has to reduce my personal accident risk under the conditions I usually drive, otherwise I prefer to keep my own hands behind the wheel over handing over control to an algorithm.
- Xylakant 5y agoI think it's fair to compare to the average driver. You may be not average by virtue of not being a teenager and not driving drunk, but you're not magically immune to being run over or crashed into by one of those. So even if a hypothetical FSD capable car would not drive one yota better than you do, it cwould still make you safer by virtue of making the surrounding environment safer.
- paxys 5y agoSure the worst drivers out there utilizing assist tech like autopilot would benefit everyone, but data shows that they aren't the ones spending money on fancy cars. My comment was about me making a decision for myself. Plus in your hypothetical the autopilot can drive at least as well as me, which is also a big assumption and not something I believe with what I have seen so far.
- djanogo 5y agoIt can't be just any "average driver", they should be compared against "average driver" who just bought $40-50K NEW car.
- sandworm101 5y ago>> "average driver" who just bought $40-50K NEW car. Driver and buyer/owner are different things. Few teenagers ever buy a Tesla, but they certainly drive them. A car cannot only be safe in the hands of the rich initial buyer who lives in a nice climate. It must be safe in all the other people who that owner may let drive. It must also be safe for subsequent second and third owners, people who might not be wealthy enough to stay home when it rains or when the autopilot thinks conditions are too rough. (It was -35c on my drive to work this morning. Dark. Ice fog then blowing snow.)
- savant_penguin 5y agoSo tesla owners belong to some specific age group and that makes it safer?
- garbageT 5y agoPre-print research based off of unpublished paper/dataset and “personal correspondence” makes this questionable. Why are we sharing this before knowledgeable and relevant reviewers get a hold it? I’ll wait for the peer-reviewed version.
- kevingadd 5y agoSo Tesla marketing gets the benefit of the doubt, but research doesn't?
- hunterb123 5y agoThat was never said. GP could doubt both. I don't think his request is out of the ordinary here: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=peer%20review&sort=byPopularity&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
- nja 5y agoI was just watching this video of a Tesla trying and miserably failing to drive itself through the streets of Boston: https://twitter.com/TaylorOgan/status/1488555256162172928 https://twitter.com/TaylorOgan/status/1488555256162172928 A number of these near-crashes seem to be pretty odd errors (a couple of times it just decided to drive on the wrong side of the street?) -- albeit ones that one _could_ see drivers unfamiliar with the area making. If a human driver started driving on the wrong side of a street, etc -- at least in Boston -- they would be greeted by a chorus of horns and rude gestures from those around, which would hopefully indicate to them that they were in error. Brings up an interesting question: does anyone know if Teslas are monitoring for surrounding horns/etc? Another common problem in Boston is being blocked by geese crossing the road. Are Teslas able to detect such low, slow-moving obstacles? It seems like it's not super feasible to rely on self-driving vehicles in a dense city like Boston, but maybe such features could be geofenced to areas like highways?
- JoshTko 5y agoTo be fair Boston city roads are like horse carriage routes that were converted to streets and will probably one of the last places FSD optimizes for.
- nja 5y agoInterestingly, the routes in that video are in one of the few parts of Boston with a street grid (Southie: https://twitter.com/TaylorOgan/status/1488555262655045637 https://twitter.com/TaylorOgan/status/1488555262655045637 ) -- so I wonder how much _worse_ it is on the cowpath roads elsewhere...
- petee 5y agoGood point, Boston is a great real-world fuzzer. If they spent their time making it work there they'd have less issues elsewhere
- mijamo 5y agoThose roads seem much easier to drive on than basically any city I've been in Europe. Paris is 10x more complex than that for instance. If FSD can't handle that there's really not of a use case apart from highways in Europe. To give you an idea of the incredible features in some cities, I have recently driven in Rennes where they found a good idea to makes roads with one single central lane for cars with wide bicycle path on each side. You are supposed to drive in the center and if a car comes in the other direction each car needs to go on the bicycle lane on their respective side of the road while giving priority to bicycles. I wonder how FSD would handle that...
- hartator 5y agoAlso feel they compare Tesla with autopilot vs. other cars. Instead of Tesla with autopilot vs. Tesla without autopilot. As Teslas have way better crash tests but doesn't mean autopilot is safer.
- maxerickson 5y agoBeing safer in a crash doesn't make it okay to crash.
- pengaru 5y agoCruise-control (including Autopilot) is only applicable to a subset of ideal driving conditions anyways. You'd make the comparison of cruise-control crash rates vs. general driving in all conditions if you were trying to abuse statistics for marketing your gimmick cruise-control feature. It's unclear to me how you'd even attempt to normalize the data when you don't have national crash rates for cruise-control only miles in non-Tesla vehicles. I saw mention that one of the TLA's investigating Teslas crashing into emergency vehicles subpoenaed all manufacturers of vehicles sold w/L2 cruise-control features for their safety data. Maybe that will produce some normalized-enough data for making a meaningful comparison.
- deleted 5y ago[deleted]
- spywaregorilla 5y agoI find this headline to be pretty misleading. * It doesn't find anything to suggest Autopilot is less safe than claimed. It finds reasons to believe Autopilot's improvement on safety relative to humans is less than claimed * Under both the road adjusted model; and the road+age adjusted model, Autopilot outperforms the average driver on average. Not by much, but its there. * Everyone here seems to be assuming that the takeaway is that Autopilot is less safe than humans, but there's no evidence for that. If the numbers here are to be believed, I would take it as a good sign for tesla, and by definition an advantage to people who are otherwise poor drivers? * Question: Why do crashes per million miles have such large swings in the baseline? It's weird that it varies so heavily by quarter without any obvious seasonal patterns to me. edit: mildly annoyed at getting downvoted for seeming to be the only person here noting that the blue line is on average below the red line on every chart shown.
- dboreham 5y agoPerhaps the downvotes are because most people would be apprehensive about putting their life in the hands of an "average" safe driver that's built from software? I probably wouldn't allow any other average safe human to drive me, if the alternative of me driving myself is available.
- spywaregorilla 5y agoYou never take ubers on the grounds of safety? Because that is, on average, an average human. Either way, I'm not too impressed. I wouldn't purchase Autopilot personally for similar safety reasons. But let's not twist the stats to suggest they're actually much worse than they are.
- tim333 5y agoIt's not actually. If you look at the stats professional drivers crash far less than the average driver per mile. More experienced, not drunk etc.
- natch 5y ago
- irthomasthomas 5y agoHow is this still front page? Somebody sleeping? In case you missed them, there have been 5 tesla stories nuked from the front page today: 'Self driving' Tesla fails miserably on the streets of South Boston 30 points 22 comments https://news.ycombinator.com/item?id=30173566 https://news.ycombinator.com/item?id=30173566 Tesla Australia admits its sales figures were wrong 39 points 14 comments https://news.ycombinator.com/item?id=30176635 https://news.ycombinator.com/item?id=30176635 Tesla drivers report a surge in ‘phantom braking’ 42 points 79 comments https://news.ycombinator.com/item?id=30179681 https://news.ycombinator.com/item?id=30179681 Self-driving Tesla does 'the craziest things you can imagine' 96 points 172 comments https://news.ycombinator.com/item?id=30178168 https://news.ycombinator.com/item?id=30178168
- deleted 5y ago[deleted]
- contravariant 5y agoLack of the word 'Tesla' in the title probably does it. As well as not tripping the flame war detector probably. We'll see how this thread fares.
- LeoPanthera 5y agoThere are valid criticisms of Tesla, this normalized data story is probably one of them, but the fans and anti-fans of Tesla are so equally rabid that it is nearly impossible to have a rational conversation about it. Expressing a view in either direction results in an instant pile-on of trolls and flames.
- toomuchtodo 5y agoThere are valid criticisms, and there is being...overly enthusiastic. https://news.ycombinator.com/submitted?id=camjohnson26 https://news.ycombinator.com/submitted?id=camjohnson26
- irthomasthomas 5y agoSo Tesla news can be discussed anywhere except HN? The reason HN is exceptional is that it has a lot of readers working on or near AI, who will quickly call out the B.S. in Tesla PR. They know this, so all Tesla stories get nuked.
- caditinpiscinam 5y agoHow is it that Tesla (and other companies developing self-driving capabilities) are allowed to deploy their products in the real world? We make people take a test to get a drivers license -- why don't we have requirements for self-driving systems? We require that cars pass routine mechanical inspections -- why isn't the software controlling these cars regulated?
- dboreham 5y agoBecause in the context of regulations, they don't claim it to be self-driving.
- FireBeyond 5y agoExactly. Marketing: "Self driving!" "The driver is only in the seat for legal purposes. The car is driving itself." Legal: "Maintain control of the vehicle at all times." Same with Summon. Marketing: "Bring your vehicle to you while dealing with a fussy child!" Legal: "Maintain attention and focus on vehicle at all times. Do not use while distracted."
- osipov 5y ago
- WA 5y agoWhich I don't understand. Marketing is part of what makes the product. You can't simply promise stuff on your website and then put something else in the fine print. If you market the thing as (almost) self-driving, then the fine-print doesn't really matter. It is about customer expectations. Most regulated products require post market surveillance. You have to actively monitor the market and how consumers behave. If they have a misunderstanding about the intended use of your product, you're overpromising. Which is clearly the case here if Tesla did their post market surveillance correctly.
- tim333 5y ago
- omgwtfbyobbq 5y agoTesla should release demographic info if they have it. IIRC, luxury vehicles tend to have lower accident/fatality rates because their buyers tend to be more experienced, mature, and wealthier. I'm also wondering about the assumptions the pre-print makes. https://engrxiv.org/preprint/view/1973/3986 https://engrxiv.org/preprint/view/1973/3986 For example, it's assumed that Tesla's data has the same characteristics as the SHRP 2 NDS data, and that the demographic survey data from Hardman et al is accurate WRT the SHRP 2 NDS data. > The analysis in this paper relies on the assumption that the freeway-to-non-freeway and age group crash ratios found in the SHRP 2 NDS are consistent with the manufacturer’s data, as there are no roadway specific nor age-related factors in the manufacturer safety report. > Although the ages of drivers in Tesla’s crash rate data are unknown, their ages could be estimated from a 2018 demographic survey of 424 Tesla owners (Hardman et al., 2019). But... It seems like Hardman et al are using California specific data. https://escholarship.org/content/qt58t7674n/qt58t7674n_noSplash_eeb18fda33a9479b18bad511cc74904f.pdf?t=qt1u1p https://escholarship.org/content/qt58t7674n/qt58t7674n_noSpl... > In this study, we used a cohort survey of Plug-in Electric Vehicles (PEV) owners in California administered by the authors in November 2019. Respondents had been previously surveyed by the UC Davis Plug-in and Hybrid & Electric Vehicle (PH&EV) Research Center between 2015 and 2018 as part of four surveys in the eVMT project when they originally bought their PEV. Respondents for the four phases of the eVMT survey were sampled from the pool of PEV buyers who had applied for the state rebate from the California Vehicle Rebate Program (CVRP). More than 25,000 PEV owners were surveyed between 2015 and 2018. A total of 15,000 of these respondents gave consent to be re-contacted and were invited for the repeat survey in 2019. In all, 4,925 PEV owners responded to the repeat survey. And... The newer data indicates that age could be positively with more Autopilot use and long-distance travel (also correlated with more Autopilot use), which seems like a confound WRT to the pre-print's conclusions and so on. > Age is negatively correlated, indicating that the lower the driver’s age the higher the odds of reporting more long-distance travel. This suggests Autopilot induces travel among younger Tesla owners. As with the model with all Level 2 automation, users’ income is negatively correlated.
- croes 5y agoI doubt that full self driving is possible in the near future without changes to the cars and the infrastructure to help the autonomous systems.
- petilon 5y agoIt is available today in a limited area: https://techcrunch.com/2021/11/03/cruise-launches-driverless-robotaxi-service-for-employees-in-san-francisco/ https://techcrunch.com/2021/11/03/cruise-launches-driverless...
- hnburnsy 5y agoFrom the paper: (1) Statistics obtained from Hardmanet al.(2019) (2) Statistics obtained from Blanco et al.(2016) (3) Statistics obtained from Transportation Research Board of the National Academy of Sciences (2013) I don't understand how such chronologically disparate sets of data could produce reliable statistics. The Telsa demographic data (1) was from a "2018 demographic survey of 424 Tesla owners"
- ActorNightly 5y ago> Crash rates can be adjusted to account for differences in environment and demographics in different data sets. A sample dataset with acrash rateris exposed to some variableiat a different proportion p than the comparison dataset. In the case study, for example, vehicles running Autopilot were driven on freeways (i) 93% of the time, resulting in pi= 0.93. In the SHRP 2 NDS, only 28% ofvehicle mileage was recorded on freeways, i.e pi= 0.28. In the SHRP 2 NDS data, vehicles on non-freeways crashed 2.01 times more often per mile than vehicles on freeways. The observed Autopilot crash rate can be adjusted to reflect national driving ratesto reflect the crash rate that might be observed if 28% of Autopilot mileage was on freeways and 72% were on non-freeways, assuming that the 2.01 ratio holds for Autopilot. So if humans are 2 times more likely to crash on non-highways than highways, the argument presented here is that the added danger of non-freeways affects autopilot in the same way, degrading its performance. Seems like a far reaching assumption. If there is no data on autopilot performance on rural roads, then you cannot make a claim either way.
- amelius 5y agoCan we have Self-driving Formula 1 before we get these cars on the street please?
- maxdo 5y ago>Although the ages of drivers in Tesla’s crash rate data are unknown, their ages could be estimated from a 2018 demographic survey of 424 Tesla owners (Hardman et al., 2019). Seriously? you trying to claim your method is more fair using 424 people in 2019, when Tesla has demographics of millions cars across the globe? My conclusion: Author is saying it's not fair to compare with some old car. Tesla was one of the first brands to mass adopt this safety features in every Telsa. This is the only brand that include camera, collision warnings in every, even cheapest modification. And they were the first to introduce many of these features. The rest are catching up. Why Tesla can't be proud of that? They give safety to every customer, no matter if it's the cheapest option or $150k+ car. Only volvo invest on par with tesla in testing facilities, in house extended crash tests ets. Autopilot and similar system is safer compares to any car on the highway with old cruise control because they have features that force you to be focused. They can stop you on a traffic light, they can react faster on animal running across the road etc. And Yes all measures like collision warnings, notifications, makes your driving safe. Yes, you can use it without autopilot, but I as a consumer don't care. If I'm buying a car, fact that this car is much safer compares to 10 years old ford makes me more confident in purchase. You can debate a lot on how easy is to avoid some of this safety measures. But if you an idiot you can fall asleep in cruise controlled car as well. Tesla will at least try to wake you up. And will stop on a traffic light. FSD is a good example of an extension of this rational vs emotional thinking. FSD Beta has a very strict focus control. You look not on a road few times you're blocked. You don't hold a wheel few times you blocked. Etc. So even FSD beta in it's current state is a complete garbage that quite often literally drives you into accident the way they enforce your attention fixes everything. FSD beta with this limitations has 60k cars on a road. And 0 major incidents that lead to a major injury or death. Not counting few scratched rims and one time driver got off the road because he over reacted on some maneuver. It's bizarre how much money legacy auto is poring into efforts to make tesla look bad. Before it was infamous data of burning EV's. But if you look at data a single car brand BMW has so 100 times more risk of fire compares to tesla. You don't see paid articles on Times square how dangerous is BMW. Right now all major brands invest in EV and suddenly all this fire incidents disappear. And they switched to Autopilot and FSD. Is there a room for improvement yes. Is the car that can stop on a traffic light, after some fixes for a cop's car, notify about potential collision, avoid automatically some collisions safer. Definitely yes. The rest is just hype, speculations and quite often corruption and paid articles.
- NaturalPhallacy 5y agoBut is it safer than human drivers? This is the real question, because a "perfect" autopilot may be impossible because of cases where it must choose between sacrificing something outside the car vs its occupants and there will be disagreement. Setting the bar at "perfect" is pretty unrealistic given that humans are already pretty bad.
- YaBomm 5y ago
- laomai 5y agoRegarding some of the conversations about programming something to obey the law or obey human convention: - it’s tricky because the people on the roads expect human convention and drive for that - as long as there is a mix of humans that drive by convention, it’s probably best to err on the side of following human convention to some degree Case in point: center lane When I first started using autopilot on two lane roads the car would stay dead center in the lane. If a car was coming towards my in the opposite lane, it began to notice humans would veer away from center to provide more buffer between themselves and the oncoming car. Because I didn’t want to piss other drivers off, I would often disengage and drift to the right of my lane while the oncoming car approached me. If I didn’t do that, there was always a last minute extra drift away from me by the other car.. the conventionally expected buffer distance wasn’t enough to make them feel comfortable due to unexpected (lack of) behavior. It’s not a law / no law issue above. But I have similar experiences navigation into roundabouts with crosswalks in front of them (autopilot stops at empty crosswalk where normal driver would cruise through until the stop line to check for cars in the roundabout — and if none were there might pass through with a rolling stop/check.
- pjkundert 5y agoAn even more serious problem with Tesla's claims regarding the safety of its Autopilot is this: Autopilot failure statistics are non-ergodic. You don't get to "play again" after a catastrophic Autopilot failure. So, Autopilot must not be 5x or 10x better than your own driving -- it has to be, like 100x or 1000x better, in order to warrant you risking your life on it. Trusting an autopilot that is just 10x better than you are? Insane. It randomly drives under a semi-trailer that you would have easily seen and avoided, and you're dead.
- freediver 5y agoAs a Tesla driver, I do not trust the autopilot yet and think it is years away (if ever achievable) from level 5... The reason this study has this outcome can be because of the survivorship bias - people are letting autopilot drive only in easiest road conditions where chance for something to go wrong is smaller to begin with. I would have hard time imagining people are letting autopilot drive regularly in a tight, windy road (at night) or in snow, thus completely preventing those (frequent types of?) accidents from happening under autopilot.
- magicroot75 5y agoI agree with the premise. However, this isn't survivorship bias, it's just a selection bias.
- freediver 5y agoCare to explain the difference in this case?
- kruxigt 5y ago
- femto113 5y agoThis has been obvious for a long time, because if it wasn't true it would be trivial for Tesla to disprove it, and they haven't. Allow me to quote myself (from a discussion about an Autopilot related crash in 2019 [1]) "The fact that Tesla consistently refuses to share full data on identical models driving with autopilot turned on, enabled but not turned on, and not enabled is extremely suggestive that the overall safety rates for autopilot on are not better." [1] https://news.ycombinator.com/item?id=19929793 https://news.ycombinator.com/item?id=19929793
- GuB-42 5y agoOne thing I wonder about the safety of Autopilot (because I have never used a Tesla). Doesn't the Autopilot sometimes just give up in difficult situations? Either explicitly, by stopping behind parked cars or things like that, or worse, by behaving so badly that the driver has to take back control for his own safety. The worst case would be: autopilot doesn't see an obstacle, driver sees it at the last moment, takes back control, but it is too late, and crash. Is it an autopilot crash or a human crash? Or: there is some road work, the autopilot cannot maneuver around it and gives up, the driver takes back control and crashes. It is clearly a human error, but still, the autopilot leave the short part where accidents are the most likely to happen the driver. It things like that happen, it is almost as if the autopilot is to be used only where accidents are the least likely to happen, which can obviously explain why people on autopilot have less accidents.
- olliej 5y agoYes, it does, which is also a great way to avoid "being active in a crash". The model of "a human must being completely focused 100% of the time while also not doing anything to control the vehicle and ready to take control" is not a reasonable model, and is why all the "partial" self-driving rules are nonsense. It also helps statistics both directly: crashes are more likely in the snow and rain, but Teslas don't do self driving in those conditions (that's one of the things this paper addresses). Tesla has in numerous cases blamed the driver for not taking control of the unsafe "self driving" when crashes occurred, how many near misses have occurred due to drivers intervening? Airlines and aircraft manufacturers are required to report events where a pilot how to take over to prevent an accident but Tesla and co don't
- nomoreusernames 5y ago
- _xnmw 5y agoRelevant: McNamara fallacy from yesterday: http://mcnamarafallacy.com http://mcnamarafallacy.com. Numbers are not a substitute for actual embodied knowledge.
- leroman 5y agoInsurance companies most probably have all the data at hand, couldn't coverage price reflect the real world risk as seen by someone with something to lose?
- tim333 5y agoTeslas are pricey to fix so it'd be hard to tell just from the premiums.
- kwhitefoot 5y agoI don't pay much more to insure my Model S than my previous car which had less than half the power and was a lot cheaper to fix if things went wrong. Which suggests to me that the insurance companies here, Norway, don't think that Tesla's are unsafe; or at least that they aren't going to cost the insurer a lot of money. And they do have a lot of experience here, 15 % of private cars on the road here are electric, quite a few of them Teslas because they were in the market from the beginning.
- bratwurst3000 5y agoYeah no shit. There is only one relevant data point and its teslas that got an accident with autopilot and without. That should spill out the lentgh of the track without accident….. for some reason tesla keeps comparing their high end cars with cars from the 90s……
- InTheArena 5y agoNiedermeyer Google this guy before you buy any of his spin. Not saying that Tesla shouldn’t be more transparent, but Niedermeyer is famous for how wrong he has been on Tesla on virtually everything. Like many Tesla partisans (not pro and anti) went way beyond rational discourse long long ago.