5 ms·

Tesla Autopilot failing where there was a fatal car accident [video]

79 points by bobsil1 9y ago

gregoriol 9y ago
There is another one: https://www.youtube.com/watch?v=KEaJW-Mk0Bo https://www.youtube.com/watch?v=KEaJW-Mk0Bo
- danso 9y ago
  Same driver just posted a new video, this time during the day, in which AP behaves correctly: https://www.youtube.com/watch?v=bkDsozA1Pb8 https://www.youtube.com/watch?v=bkDsozA1Pb8
  - gregoriol 9y ago
    That's the scary and also dangerous part: the behavior might be ok at some point, and might fail at another point. Tesla's (and NTSB) investigations will likely find out what went wrong in this particular accident and on this part of the road, and likely will improve the systems. But users are certainly not enough aware of that kind of risk: they might try the Autopilot, see that it works, and trust that if it worked, it will work again and again and again... On a daily commute for example, if it worked many days in a row, and then fails for some reason (technical, external, whatever), the user will likely fail to notice.
    - unityByFreedom 9y ago
      > On a daily commute for example, if it worked many days in a row, and then fails for some reason (technical, external, whatever), the user will likely fail to notice Yup. This is why I won't get into a self-driving car until they're fully driverless. Drivers become less attentive as the system gets better.
  - GoToRO 9y ago
    During the day there is a black gap clearly visible. During the night I imagine that black line just disappears.
  - rohit2412 9y ago
    Since nobody noticed it, it failed with a AP2 car in night time, and succeeded with a AP1 car in daytime
- lamecicle 9y ago
  That is waaay too close for no one to be screaming in the car! I don't understand how the Tesla couldn't see the centre barrier but it can detect when a crash is about it happen, can someone explain that?
  - lazyjones 9y ago
    > I don't understand how the Tesla couldn't see the centre barrier but it can detect when a crash is about it happen, can someone explain that? AFAIK the AP focuses more on moving targets for collision avoidance, stationary objects are harder because there's so many of them you'll never hit since they're above or next to the road, but that's hard to tell until they're very close (can't see where the road leads to 100m away).
    - Piskvorrr 9y ago
      That's the very definition of "you're driving way too fast (and if that's your only detection method, you shouldn't be on the road)".
  - avree 9y ago
    It can't detect when a crash is about to happen. The driver manually pushes the brakes there. I have only worked tangentially with self-driving systems but in general they are very bad at dealing with stationary objects - this is why it can't see the center barrier. This article describes the reason why radar and stationary objects don't mix: https://www.wired.com/story/tesla-autopilot-why-crash-radar/ https://www.wired.com/story/tesla-autopilot-why-crash-radar/
    - amelius 9y ago
      How far ahead can LiDAR detect stationary objects? Will there be enough time on a highway to brake or steer away from suddenly appearing obstacles?
      - unityByFreedom 9y ago
- vosbert 9y ago
  Better quality video of the same video: https://www.youtube.com/watch?v=6QCF8tVqM3I https://www.youtube.com/watch?v=6QCF8tVqM3I
babaganoosh89 9y ago
This seems very hard for Tesla to fix with a simple software patch. If the car is using cameras to look at the lines on the road + radar which can not see stationary objects well, then I don't see how they'd have enough sensor information to fix this reliably.
- ealexhudson 9y ago
  In principle the cameras can see the stationary object like we can, after all, we don't have LIDAR. Fundamentally, though, that does seem like the long route to self driving. Being able to not drive into the proverbial wall seems like Job #1 of a pilot system. Tesla need to be careful they're not not creating a reputation for themselves at this point.
- simion314 9y ago
  They have many cars passing those places, can't they improve the maps they have, in the places where the video inputs and Tesla maps will conflict decide to slow down and ask the human to take over, then analyze the issue and add is as a unit test.
  - Piskvorrr 9y ago
    Improving maps only papers over the issue at this one particular place. What happens when this happens somewhere else, where there's no marking in the map instructing the lane-keeping software "this is a known bug, follow the right lane marker here"?
    - simion314 9y ago
      I don't mean of marking this spot, but soemthing like 1 this is a new road, "autopilot" will not engage but collect data and add it to the map 2 after 100 or other better number of cars passed that point and the maps are updated enough AP can engage 3 if radar+camera detects something that does not match the maps it slows down and asks the human to take over, the maps are used as a way to double check what the AI wants to do. This system will not work on small roads with few Tesla cars on them but it will prevent problems on big roads and collect useful data to add to the unit tests
TheSpiceIsLife 9y ago
It'd be interesting to see other vehicles with adaptive cruise control / lane keeping try this same experiment.
- gambiting 9y ago
  They would fail too, but I'm not aware of any adaptive cruise control from any manufacturer being sold as "autopilot". In either case, I think the biggest(and insurmountable) obstacle to self-driving cars will be the fact that they rely on having clear signs, clear markings, clear lanes, signs not rotated by the wind or vandalised - and it's going to be impossible(due to cost) to maintain our infrastructure to the required level. Yes, Tesla can patch their system to work in this specific road situation - but there will be countless others.
  - simion314 9y ago
    But what happens when you have tons of self driving cars, and it starts snowing, do all this cars get stuck until the snowing stops and how can you clear the roads if the cars are stuck? (I am referring at a scenario where this are actual self driving cars and not drive assist systems with a human behind the wheel)
- dx034 9y ago
  Isn't lane keeping designed to avoid that you accidentally leave the lane? I'm not aware of any OEMs except Tesla marketing it as doing the job for you (except for congestion at low speed). VW writes: > The Lane Assist system is like a friendly co-driver. If it senses your car is drifting out of your driving lane it gently counter-steers the car back on line. They explicitly say that it only corrects you, it doesn't replace steering by the driver.
coolspot 9y ago
Wow. This and one in comments are scary, I could absolutely miss situation like that.
beobab 9y ago
The way the concrete is laid there gives mis-clues. I had a funny brain-wobble whilst watching the video, playing “hunt for the lane”. I guess the car decided on the wrong one, with fatal consequences.
- dx034 9y ago
  When driving on roads with poor markings, I always follow other cars in front of me and in the other lanes to determine where my lane is. It's better to be in sync with the rest of the traffic than being "correct". I'm a bit confused that self driving cars (at least Teslas) appear to ignore that. Even if they were right and it was the correct lane, isn't following it more dangerous than staying in sync with traffic?
  - alkonaut 9y ago
    The autopilot likely thought the other cars took an off-ramp and staid on the highway by following the left lane. I'm surprised there aren't a marked (hatched) area at the junction though. But regardless, the car has to be able to handle even mis-marked or temporary roadwork.
  - Piskvorrr 9y ago
    That's a good way to get off the road - e.g. at night, in the snow. Seen that more than once - the previous car might be going straight because they lost traction in a curve. Do not overestimate the wisdom of the crowds, especially in a crowd of one.
    - dx034 9y ago
      That obviously doesn't apply if there's just one car. But if you have >10 cars in different lanes, I can't see how not following them would help you.
      - Piskvorrr 9y ago
        Fair point; not sure how much the current code is ready for incorporating the collective intent. Another issue: what if the other 10 cars are also self-driving into a wall? With such a low quorum, you are still vulnerable to the same issue as with a single car.
dawnerd 9y ago
Not just a Tesla problem as I recon all cars using lane markings will have done the same. My Passat will constantly try to pull me into off-ramps or whenever theres old paint or other weird markings.
- amelius 9y ago
  When exactly did we start to allow these half-baked products on the road?
  - dx034 9y ago
    Other manufacturers market that product for collision avoidance, not to drive the car for you. These features can help to avoid accidents when the driver is tired or doesn't pay attention.
    - simion314 9y ago
      And then some SV startups decided to abuse people trust and named a lane assist feature as autopilot, put tons of marketing into showing how great the autopilot is and very soon we will no need to have a human driver.
- dx034 9y ago
  But the difference is that other car manufacturers deliberately don't label it Autopilot. They call it "lane assist" (or similar). That makes clear that it keeps you in the lane, not that it drives the car for you. I.e., without proper lanes it cannot work. I don't think Tesla's capabilities are the issue, their marketing is. They should've never called it Autopilot if it's just the standard adaptive cruise control + lane assist that nearly all manufacturers offer.
- jbrooksuk 9y ago
  My VW T-Roc has Lane Assist and it does the same. I can't quite work out if I love it or hate it yet.
  - 9y ago
downandout 9y ago
I get that the system is designed to follow the white line, and that in this nightmarish freeway design, the white line leads straight into the divider. But I thought that the whole point of LIDAR/whatever Tesla uses for object detection was to be able to detect upcoming objects. This seems to have been a problem in the Uber pedestrian crash as well - while it was clear from that video that a human driver probably couldn’t have seen the woman, a LIDAR equipped vehicle should have easily detected that she was there and avoided her (and that Uber vehicle had full LIDAR I believe). So what is going on here? I’m not a radar expert. Is there some fundamental flaw with this technology that is preventing it from “seeing” all objects under all circumstances? Or does the problem arise from faulty driving logic...perhaps the object detection system reports the object, and for whatever reason the system interpreting that data ignores it?
- e98cuenc 9y ago
  Tesla does not use lidar, only regular cameras.
  - Gustomaximus 9y ago
    Tesla used to use radar and camera combo. Has this changed? https://www.tesla.com/en_AU/blog/upgrading-autopilot-seeing-world-radar https://www.tesla.com/en_AU/blog/upgrading-autopilot-seeing-...
  - deleted 9y ago
    [deleted]
  - alkonaut 9y ago
    Any car with distance sensing cruise control or collision avoidance at least has some kind of forward looking radar tech, right? So it should at least be able to panic-break when noticing the mistake? It looked like the car had gone full speed into the barrier.
    - simion314 9y ago
      Tesla has this radar tech but they get some confusing data sometimes so they decided to put code to ignore some of the detections https://www.tesla.com/blog/upgrading-autopilot-seeing-world-radar https://www.tesla.com/blog/upgrading-autopilot-seeing-world-...
sauravt 9y ago
Loss of even 1 life is far greater than the convenience gain for all other passengers combined. Current autopilot technology just isn't ready for the roads yet (some of them like this one in the video constructed with gross incompetence). This and many such future accidents to follow cannot be fixed by simple software glitches, instead, auto driving car manufacturers should be required to construct magnetic road white stripes for major highways and routes.
- Scaevolus 9y ago
  It's not just convenience. Human drivers make mistakes and die all the time. If every car in the US had autopilot and half the rate of fatal crashes (not perfect!), you'd save 18,000 lives each year.
  - rohit2412 9y ago
    If the fatal crashes are prevented with automatic emergency breaking, why drag in rest of autopilot? AEB is a simple assistance technology that's standard on most new vehicles
  - danso 9y ago
    There's no evidence that autopilot has that good of a safety effect yet.
- duncan_bayne 9y ago
  Are you sure? Your argument reduces to absurdity, because if a single life is never an acceptable trade-off for convenience, we should set a global speed limit of 5mph.
  - visakanv 9y ago
    And some people will still probably die at 5mph!
- Piskvorrr 9y ago
  How would that help? The car followed the left stripe (into the divider), what difference would it make if it were magnetic?
duncan_bayne 9y ago
I think the problem is that, as things stand[1], autonomous cars are an "AI hard" problem, in that you'd need to build something of equivalent general intelligence to a human. https://en.m.wikipedia.org/wiki/AI-complete https://en.m.wikipedia.org/wiki/AI-complete [1] One example of how things might change is if societies decided to build "smart roads" to assist AI drivers, and then banned non AI drivers from them.
- simion314 9y ago
  A smart road could be like a train track, completely automated, then you would enter in a cart, input a destination and the centralized system will compute yo2ur route and speeds, this needs a lot more infrastructure but theoretically you could have huge speeds and no stops at intersections, just some small slowdowns to allow carts pass near each other at intersections.
- Piskvorrr 9y ago
  Building the new and shiny is not the problem. Keeping it shiny is.
alkonaut 9y ago
What level of autonomy does Tesla market their latest hardware as? So long as they never label it as being above SAE Level 2 (i.e. don't take your hands of the wheel or eyes off the road ever) then I have no problem with this tech. The only thing they need to fix then is the name of the product. They shouldn't be calling it autopilot as it gives the impression you can answer an email on your phone while the car is doing the commuting for you. In my opinion Tesla and all other manufacturers should perfect their Level 2 tech (and not call it autopilot) and then jump straight to Level 5, even if it means it's 3 decades out. We can't have drivers sleeping at the wheel in level 3/4 cars with patchy software. And that's regardless of whether manufacturers can prove that their tech has markedly fewer accidents than often-failing human drivers. I'm happy to take the risk to crash with a sleepy fleshdriver, but I'm not ok with crashing due to a software bug. I think many share this sentiment - which means there is a double standard that setting the bar much much higher for autonomous systems than they will ever be for humans.
- SCHiM 9y ago
  No, humans cannot operate in the way the product is advertised. It does not work and it invites the human to become complacent and not pay attention. It's a way to weasel and doublespeak your way around shortcomings in the product. A human that doesn't need their hands on the wheel and their eyes on the road 99% of the time will not have their hands on the wheel the 1% of the time that they do.
- OscarCunningham 9y ago
  >I'm happy to take the risk to crash with a sleepy fleshdriver, but I'm not ok with crashing due to a software bug. I find this very hard to understand, especially when you're making the choice on behalf of others. Would you drive your children to school, knowing that you're making them less safe, just so that if they do die it's your fault rather than that of some programmer?
  - tremon 9y ago
    What's the hard to understand part? The programmer is not the one facing a life-or-death decision when a driving hazard occurs -- the driver is.
Gustomaximus 9y ago
1) Given Tesla has been using passive data collection wouldn't a well traveled road like this have clear data that there is a split road but nothing in the middle so geo-position should have stopped this? 2) How did the radar /camera not see the barrier. 3) Give their update capability I'm surprised this section of road hasn't been fixed or updated to stop 'autopilot' for this section.
- improbable22 9y ago
  Automated mapping seems like it ought to be able to solve this. Even if the cameras are less good than a human, all the other cars' cameras could help immensely. Why isn't every Tesla tracking all the lanes it can see, and the behaviour of every option at every exit, and flagging cases where one of the other lanes does something weird, or dangerous, half a mile later?
- makomk 9y ago
  Are they even collecting that kind of data, let alone using it to make decisions in autopilot mode? I know they're using passively collected data to decide whether radar detections of passive objects are actual threats or just road signs, but I'm not sure they've announced anything more advanced than that.
- mshook 9y ago
  2) was best explained by https://news.ycombinator.com/item?id=16239612 https://news.ycombinator.com/item?id=16239612 and in a nutshell is what the wired article mentioned above was about.
sschueller 9y ago
Interesting how the autopilot section on their website for the model s is no longer there as it was in 2016 yet the other items are still there[1]. Or on this page[2], Specifically the section "Full Self-Driving Capability" makes it sound like that car can completely drive itself. [1] https://web.archive.org/web/20160726173047/https://www.tesla.com/models/ https://web.archive.org/web/20160726173047/https://www.tesla... [2] https://web.archive.org/web/20171116234123/https://www.tesla.com/autopilot https://web.archive.org/web/20171116234123/https://www.tesla...
bkovacev 9y ago
Correct me if I am wrong, but the car followed the bolder line instead of a barely visible one? Could it be sunlight + concrete + bad vision of the lane line?
m4tthumphrey 9y ago
UK driver here. If self driving cars rely on lane markings for working out where to sit on a road, how does it work on country lanes or newly laid roads?
danso 9y ago
After last week's crash, when Tesla claimed [0]: > Our data shows that Tesla owners have driven this same stretch of highway with Autopilot engaged roughly 85,000 times since Autopilot was first rolled out in 2015 and roughly 20,000 times since just the beginning of the year, and there has never been an accident that we know of. There are over 200 successful Autopilot trips per day on this exact stretch of road. Do we know how many of those trips were made with the same software version as the victim's? And do we know how many of those trips were in that particular lane? [0] https://www.tesla.com/blog/what-we-know-about-last-weeks-accident https://www.tesla.com/blog/what-we-know-about-last-weeks-acc...
heurist 9y ago
I wonder if autopilot networks will eventually reroute vehicles around bug-prone locations like this. The only problem is recognizing bug-prone locations. It is interesting to think of some geographic locations as wells of danger that are best avoided...
PoppaSmurf 9y ago
I've read the concept that a self driving car cannot react to everything that it sees in front of it because otherwise it would basically be inoperable, ie, it has to disregard things like overhead signs, bridges crossing over the highway etc, because braking for those is unnecessary and possibly dangerous. But, when the car detects these things and decides to disregard them, does it just forget about them? Just thinking back to the Tesla car that ended up going under the truck crossing in its path, supposedly the car didn't react as you would expect because it assumed it was an overhead sign. But I would think that an overhead sign would begin to look different than a truck on the road crossing in front of you as you continue to get closer to it, is the software not capable of at first deciding to disregard something up ahead but then change its mind as it gets closer? Likewise with a concrete stanchion that keeps getting closer and closer? Does the software allow for it to second guess itself?

79 points by bobsil1 9y ago

gregoriol 9y ago

There is another one: https://www.youtube.com/watch?v=KEaJW-Mk0Bo https://www.youtube.com/watch?v=KEaJW-Mk0Bo

danso 9y ago
Same driver just posted a new video, this time during the day, in which AP behaves correctly: https://www.youtube.com/watch?v=bkDsozA1Pb8 https://www.youtube.com/watch?v=bkDsozA1Pb8
- gregoriol 9y ago
  That's the scary and also dangerous part: the behavior might be ok at some point, and might fail at another point. Tesla's (and NTSB) investigations will likely find out what went wrong in this particular accident and on this part of the road, and likely will improve the systems. But users are certainly not enough aware of that kind of risk: they might try the Autopilot, see that it works, and trust that if it worked, it will work again and again and again... On a daily commute for example, if it worked many days in a row, and then fails for some reason (technical, external, whatever), the user will likely fail to notice.
  - unityByFreedom 9y ago
    > On a daily commute for example, if it worked many days in a row, and then fails for some reason (technical, external, whatever), the user will likely fail to notice Yup. This is why I won't get into a self-driving car until they're fully driverless. Drivers become less attentive as the system gets better.
- GoToRO 9y ago
  During the day there is a black gap clearly visible. During the night I imagine that black line just disappears.
- rohit2412 9y ago
  Since nobody noticed it, it failed with a AP2 car in night time, and succeeded with a AP1 car in daytime
lamecicle 9y ago
That is waaay too close for no one to be screaming in the car! I don't understand how the Tesla couldn't see the centre barrier but it can detect when a crash is about it happen, can someone explain that?
- lazyjones 9y ago
  > I don't understand how the Tesla couldn't see the centre barrier but it can detect when a crash is about it happen, can someone explain that? AFAIK the AP focuses more on moving targets for collision avoidance, stationary objects are harder because there's so many of them you'll never hit since they're above or next to the road, but that's hard to tell until they're very close (can't see where the road leads to 100m away).
  - Piskvorrr 9y ago
    That's the very definition of "you're driving way too fast (and if that's your only detection method, you shouldn't be on the road)".
- avree 9y ago
  It can't detect when a crash is about to happen. The driver manually pushes the brakes there. I have only worked tangentially with self-driving systems but in general they are very bad at dealing with stationary objects - this is why it can't see the center barrier. This article describes the reason why radar and stationary objects don't mix: https://www.wired.com/story/tesla-autopilot-why-crash-radar/ https://www.wired.com/story/tesla-autopilot-why-crash-radar/
  - amelius 9y ago
    How far ahead can LiDAR detect stationary objects? Will there be enough time on a highway to brake or steer away from suddenly appearing obstacles?
    - unityByFreedom 9y ago
vosbert 9y ago
Better quality video of the same video: https://www.youtube.com/watch?v=6QCF8tVqM3I https://www.youtube.com/watch?v=6QCF8tVqM3I

babaganoosh89 9y ago

This seems very hard for Tesla to fix with a simple software patch. If the car is using cameras to look at the lines on the road + radar which can not see stationary objects well, then I don't see how they'd have enough sensor information to fix this reliably.

ealexhudson 9y ago
In principle the cameras can see the stationary object like we can, after all, we don't have LIDAR. Fundamentally, though, that does seem like the long route to self driving. Being able to not drive into the proverbial wall seems like Job #1 of a pilot system. Tesla need to be careful they're not not creating a reputation for themselves at this point.
simion314 9y ago
They have many cars passing those places, can't they improve the maps they have, in the places where the video inputs and Tesla maps will conflict decide to slow down and ask the human to take over, then analyze the issue and add is as a unit test.
- Piskvorrr 9y ago
  Improving maps only papers over the issue at this one particular place. What happens when this happens somewhere else, where there's no marking in the map instructing the lane-keeping software "this is a known bug, follow the right lane marker here"?
  - simion314 9y ago
    I don't mean of marking this spot, but soemthing like 1 this is a new road, "autopilot" will not engage but collect data and add it to the map 2 after 100 or other better number of cars passed that point and the maps are updated enough AP can engage 3 if radar+camera detects something that does not match the maps it slows down and asks the human to take over, the maps are used as a way to double check what the AI wants to do. This system will not work on small roads with few Tesla cars on them but it will prevent problems on big roads and collect useful data to add to the unit tests

TheSpiceIsLife 9y ago

It'd be interesting to see other vehicles with adaptive cruise control / lane keeping try this same experiment.

gambiting 9y ago
They would fail too, but I'm not aware of any adaptive cruise control from any manufacturer being sold as "autopilot". In either case, I think the biggest(and insurmountable) obstacle to self-driving cars will be the fact that they rely on having clear signs, clear markings, clear lanes, signs not rotated by the wind or vandalised - and it's going to be impossible(due to cost) to maintain our infrastructure to the required level. Yes, Tesla can patch their system to work in this specific road situation - but there will be countless others.
- simion314 9y ago
  But what happens when you have tons of self driving cars, and it starts snowing, do all this cars get stuck until the snowing stops and how can you clear the roads if the cars are stuck? (I am referring at a scenario where this are actual self driving cars and not drive assist systems with a human behind the wheel)
dx034 9y ago
Isn't lane keeping designed to avoid that you accidentally leave the lane? I'm not aware of any OEMs except Tesla marketing it as doing the job for you (except for congestion at low speed). VW writes: > The Lane Assist system is like a friendly co-driver. If it senses your car is drifting out of your driving lane it gently counter-steers the car back on line. They explicitly say that it only corrects you, it doesn't replace steering by the driver.

coolspot 9y ago

Wow. This and one in comments are scary, I could absolutely miss situation like that.

beobab 9y ago

The way the concrete is laid there gives mis-clues. I had a funny brain-wobble whilst watching the video, playing “hunt for the lane”. I guess the car decided on the wrong one, with fatal consequences.

dx034 9y ago
When driving on roads with poor markings, I always follow other cars in front of me and in the other lanes to determine where my lane is. It's better to be in sync with the rest of the traffic than being "correct". I'm a bit confused that self driving cars (at least Teslas) appear to ignore that. Even if they were right and it was the correct lane, isn't following it more dangerous than staying in sync with traffic?
- alkonaut 9y ago
  The autopilot likely thought the other cars took an off-ramp and staid on the highway by following the left lane. I'm surprised there aren't a marked (hatched) area at the junction though. But regardless, the car has to be able to handle even mis-marked or temporary roadwork.
- Piskvorrr 9y ago
  That's a good way to get off the road - e.g. at night, in the snow. Seen that more than once - the previous car might be going straight because they lost traction in a curve. Do not overestimate the wisdom of the crowds, especially in a crowd of one.
  - dx034 9y ago
    That obviously doesn't apply if there's just one car. But if you have >10 cars in different lanes, I can't see how not following them would help you.
    - Piskvorrr 9y ago
      Fair point; not sure how much the current code is ready for incorporating the collective intent. Another issue: what if the other 10 cars are also self-driving into a wall? With such a low quorum, you are still vulnerable to the same issue as with a single car.

dawnerd 9y ago

Not just a Tesla problem as I recon all cars using lane markings will have done the same. My Passat will constantly try to pull me into off-ramps or whenever theres old paint or other weird markings.

amelius 9y ago
When exactly did we start to allow these half-baked products on the road?
- dx034 9y ago
  Other manufacturers market that product for collision avoidance, not to drive the car for you. These features can help to avoid accidents when the driver is tired or doesn't pay attention.
  - simion314 9y ago
    And then some SV startups decided to abuse people trust and named a lane assist feature as autopilot, put tons of marketing into showing how great the autopilot is and very soon we will no need to have a human driver.
dx034 9y ago
But the difference is that other car manufacturers deliberately don't label it Autopilot. They call it "lane assist" (or similar). That makes clear that it keeps you in the lane, not that it drives the car for you. I.e., without proper lanes it cannot work. I don't think Tesla's capabilities are the issue, their marketing is. They should've never called it Autopilot if it's just the standard adaptive cruise control + lane assist that nearly all manufacturers offer.
jbrooksuk 9y ago
My VW T-Roc has Lane Assist and it does the same. I can't quite work out if I love it or hate it yet.
- 9y ago

downandout 9y ago

I get that the system is designed to follow the white line, and that in this nightmarish freeway design, the white line leads straight into the divider. But I thought that the whole point of LIDAR/whatever Tesla uses for object detection was to be able to detect upcoming objects. This seems to have been a problem in the Uber pedestrian crash as well - while it was clear from that video that a human driver probably couldn’t have seen the woman, a LIDAR equipped vehicle should have easily detected that she was there and avoided her (and that Uber vehicle had full LIDAR I believe). So what is going on here? I’m not a radar expert. Is there some fundamental flaw with this technology that is preventing it from “seeing” all objects under all circumstances? Or does the problem arise from faulty driving logic...perhaps the object detection system reports the object, and for whatever reason the system interpreting that data ignores it?

e98cuenc 9y ago
Tesla does not use lidar, only regular cameras.
- Gustomaximus 9y ago
  Tesla used to use radar and camera combo. Has this changed? https://www.tesla.com/en_AU/blog/upgrading-autopilot-seeing-world-radar https://www.tesla.com/en_AU/blog/upgrading-autopilot-seeing-...
- deleted 9y ago
  [deleted]
- alkonaut 9y ago
  Any car with distance sensing cruise control or collision avoidance at least has some kind of forward looking radar tech, right? So it should at least be able to panic-break when noticing the mistake? It looked like the car had gone full speed into the barrier.
  - simion314 9y ago
    Tesla has this radar tech but they get some confusing data sometimes so they decided to put code to ignore some of the detections https://www.tesla.com/blog/upgrading-autopilot-seeing-world-radar https://www.tesla.com/blog/upgrading-autopilot-seeing-world-...

sauravt 9y ago

Loss of even 1 life is far greater than the convenience gain for all other passengers combined. Current autopilot technology just isn't ready for the roads yet (some of them like this one in the video constructed with gross incompetence). This and many such future accidents to follow cannot be fixed by simple software glitches, instead, auto driving car manufacturers should be required to construct magnetic road white stripes for major highways and routes.

Scaevolus 9y ago
It's not just convenience. Human drivers make mistakes and die all the time. If every car in the US had autopilot and half the rate of fatal crashes (not perfect!), you'd save 18,000 lives each year.
- rohit2412 9y ago
  If the fatal crashes are prevented with automatic emergency breaking, why drag in rest of autopilot? AEB is a simple assistance technology that's standard on most new vehicles
- danso 9y ago
  There's no evidence that autopilot has that good of a safety effect yet.
duncan_bayne 9y ago
Are you sure? Your argument reduces to absurdity, because if a single life is never an acceptable trade-off for convenience, we should set a global speed limit of 5mph.
- visakanv 9y ago
  And some people will still probably die at 5mph!
Piskvorrr 9y ago
How would that help? The car followed the left stripe (into the divider), what difference would it make if it were magnetic?

duncan_bayne 9y ago

I think the problem is that, as things stand[1], autonomous cars are an "AI hard" problem, in that you'd need to build something of equivalent general intelligence to a human. https://en.m.wikipedia.org/wiki/AI-complete https://en.m.wikipedia.org/wiki/AI-complete [1] One example of how things might change is if societies decided to build "smart roads" to assist AI drivers, and then banned non AI drivers from them.

simion314 9y ago
A smart road could be like a train track, completely automated, then you would enter in a cart, input a destination and the centralized system will compute yo2ur route and speeds, this needs a lot more infrastructure but theoretically you could have huge speeds and no stops at intersections, just some small slowdowns to allow carts pass near each other at intersections.
Piskvorrr 9y ago
Building the new and shiny is not the problem. Keeping it shiny is.

alkonaut 9y ago

What level of autonomy does Tesla market their latest hardware as? So long as they never label it as being above SAE Level 2 (i.e. don't take your hands of the wheel or eyes off the road ever) then I have no problem with this tech. The only thing they need to fix then is the name of the product. They shouldn't be calling it autopilot as it gives the impression you can answer an email on your phone while the car is doing the commuting for you. In my opinion Tesla and all other manufacturers should perfect their Level 2 tech (and not call it autopilot) and then jump straight to Level 5, even if it means it's 3 decades out. We can't have drivers sleeping at the wheel in level 3/4 cars with patchy software. And that's regardless of whether manufacturers can prove that their tech has markedly fewer accidents than often-failing human drivers. I'm happy to take the risk to crash with a sleepy fleshdriver, but I'm not ok with crashing due to a software bug. I think many share this sentiment - which means there is a double standard that setting the bar much much higher for autonomous systems than they will ever be for humans.

SCHiM 9y ago
No, humans cannot operate in the way the product is advertised. It does not work and it invites the human to become complacent and not pay attention. It's a way to weasel and doublespeak your way around shortcomings in the product. A human that doesn't need their hands on the wheel and their eyes on the road 99% of the time will not have their hands on the wheel the 1% of the time that they do.
OscarCunningham 9y ago
>I'm happy to take the risk to crash with a sleepy fleshdriver, but I'm not ok with crashing due to a software bug. I find this very hard to understand, especially when you're making the choice on behalf of others. Would you drive your children to school, knowing that you're making them less safe, just so that if they do die it's your fault rather than that of some programmer?
- tremon 9y ago
  What's the hard to understand part? The programmer is not the one facing a life-or-death decision when a driving hazard occurs -- the driver is.

Gustomaximus 9y ago

1) Given Tesla has been using passive data collection wouldn't a well traveled road like this have clear data that there is a split road but nothing in the middle so geo-position should have stopped this? 2) How did the radar /camera not see the barrier. 3) Give their update capability I'm surprised this section of road hasn't been fixed or updated to stop 'autopilot' for this section.

improbable22 9y ago
Automated mapping seems like it ought to be able to solve this. Even if the cameras are less good than a human, all the other cars' cameras could help immensely. Why isn't every Tesla tracking all the lanes it can see, and the behaviour of every option at every exit, and flagging cases where one of the other lanes does something weird, or dangerous, half a mile later?
makomk 9y ago
Are they even collecting that kind of data, let alone using it to make decisions in autopilot mode? I know they're using passively collected data to decide whether radar detections of passive objects are actual threats or just road signs, but I'm not sure they've announced anything more advanced than that.
mshook 9y ago
2) was best explained by https://news.ycombinator.com/item?id=16239612 https://news.ycombinator.com/item?id=16239612 and in a nutshell is what the wired article mentioned above was about.

sschueller 9y ago

Interesting how the autopilot section on their website for the model s is no longer there as it was in 2016 yet the other items are still there[1]. Or on this page[2], Specifically the section "Full Self-Driving Capability" makes it sound like that car can completely drive itself. [1] https://web.archive.org/web/20160726173047/https://www.tesla.com/models/ https://web.archive.org/web/20160726173047/https://www.tesla... [2] https://web.archive.org/web/20171116234123/https://www.tesla.com/autopilot https://web.archive.org/web/20171116234123/https://www.tesla...

bkovacev 9y ago

Correct me if I am wrong, but the car followed the bolder line instead of a barely visible one? Could it be sunlight + concrete + bad vision of the lane line?

m4tthumphrey 9y ago

UK driver here. If self driving cars rely on lane markings for working out where to sit on a road, how does it work on country lanes or newly laid roads?

danso 9y ago

After last week's crash, when Tesla claimed [0]: > Our data shows that Tesla owners have driven this same stretch of highway with Autopilot engaged roughly 85,000 times since Autopilot was first rolled out in 2015 and roughly 20,000 times since just the beginning of the year, and there has never been an accident that we know of. There are over 200 successful Autopilot trips per day on this exact stretch of road. Do we know how many of those trips were made with the same software version as the victim's? And do we know how many of those trips were in that particular lane? [0] https://www.tesla.com/blog/what-we-know-about-last-weeks-accident https://www.tesla.com/blog/what-we-know-about-last-weeks-acc...

heurist 9y ago

I wonder if autopilot networks will eventually reroute vehicles around bug-prone locations like this. The only problem is recognizing bug-prone locations. It is interesting to think of some geographic locations as wells of danger that are best avoided...

PoppaSmurf 9y ago

I've read the concept that a self driving car cannot react to everything that it sees in front of it because otherwise it would basically be inoperable, ie, it has to disregard things like overhead signs, bridges crossing over the highway etc, because braking for those is unnecessary and possibly dangerous. But, when the car detects these things and decides to disregard them, does it just forget about them? Just thinking back to the Tesla car that ended up going under the truck crossing in its path, supposedly the car didn't react as you would expect because it assumed it was an overhead sign. But I would think that an overhead sign would begin to look different than a truck on the road crossing in front of you as you continue to get closer to it, is the software not capable of at first deciding to disregard something up ahead but then change its mind as it gets closer? Likewise with a concrete stanchion that keeps getting closer and closer? Does the software allow for it to second guess itself?