9 ms·

Melbourne hit-and-run blamed on Tesla Autopilot, could set legal precedent

29 points by gravelc 5y ago

ggm 5y ago
The run part: I think they're a young overseas resident who panicked. They returned and handed themselves in to police at the scene. A lot of younger foreigners have no idea what to do in an accident and in their home economy can face retribution, extortion, sometimes at the hands of police. In Australia, you stay, you help the victim, you report the accident. That's the law. In theory all drivers should know this but you get a year on your homeland licence before having to sit a local road rules exam. I have nothing to add regarding self driving. Well I do: I think it's been misnamed and oversold.
- genewitch 5y ago
  > I think it's been misnamed and oversold. Yep. I marvel at the "Machine Learning AI ad algorithms" that try to sell me junk. I get 3 (three!) emails a day from aliexpress about stuff i already bought ... from aliexpress. Same thing with Amazon. I bought a, lets say "toaster" from amazon, and for the next 6 months every weird amazon ad on the wide internet and every time i hit the amazon home page it's trying to sell me toasters. I don't even see it like "it's like a child, it finds something interesting and repeats it" because even a child gives up after a week. maybe a lot of people don't see "AI" as "old tech" - even though some of the first real "advances" - and i say that with a wry smile - were documented 40 years ago or more. It's complete and utter snake oil. There are some cute toys that AI/ML can do, i can clone certain voices passably, for instance. There's "deepfakes" where a screenshot is enough to maybe fool someone not wearing their glasses at 3 meters, on a smart phone screen. There's obviously the AI in video games, etc. But we're a good 25 years out from a truly autonomous, road-worthy, self-driving human transport. Someone will always have to "look over the shoulder" until then. The day i can hire a private self-driving taxi/limo to pick me up at my house and drive me to an arbitrary location, at any time of day or year, is the day i start to tinfoil my house to prevent the grey goo from coming inside. edit: i should note that i would ab-so-loutely love a car service that was cheaper than hiring a limo for a week, if you can even do that. If the car is driving by itself, i can sleep while it gets from interesting place in the US to interesting place, and spend most of my time awake documenting and photographing and interviewing people, rather than hating interstate 10, 20, and 40; which is how most of my "road trips" for the past 20 years have gone. I've traveled this country "a lot", and i rarely have cute stories about places i've been to because i've either been completely wiped and just slept there somewhere, or i was in a hurry to beat the sun/state police/whatever through the state. I'd love to spend a few weeks in wyoming and montana and wisconsin, but if i have to drive it, the only thing i'll publish is a couple of shots of mountains, the kids posing in front of some tourist trap, and that's about it. I passively consume, yet nearly universally ignore and eschew all "AI" news.
  - fxtentacle 5y ago
    Where AI has made really great advancements is all the stuff that you can memorize. Like GitHub Co-Pilot reurgitating John Carmack's source code. Also, stochastic language models have reached the point where they seem like magic, e.g. automated YouTube subtitles and translations, or text to speech. But that all works by memorizing word fragments (CTC) and/or memorizing sentence fragments (n-Gram).
  - krisoft 5y ago
    > But we're a good 25 years out from a truly autonomous, road-worthy, self-driving human transport. It sounds like your reasoning goes like this: my interactions with things which are supposedly AI are bad. Self-driving cars are also some form of AI, therefore self-driving cars must be bad and or far away too. Did I understood you right? The problem with that logic is that there is nothing in common in implementation/architecture/incentives between the things you mention and self-driving technology. Nobody, well nearly nobody, tries to implement self-driving cars in a blackbox “AI” fashion. What I mean is that we don’t just throw sensor data at a neural network, then squint and say “i recon it is going to drive well now”. That would be madness. Most approaches break down the problem into sub-problems covered by sub-systems. The sub-systems are fed information with known error properties and engineered to the specifications. The failure modes are painstakingly traced through and documented. Then in turn assemblies of these subsystems and the whole are reasoned similarly. Fault trees are drawn, the operational domain is considered. The reasoning why the engineers think the system is safe and have the right redundancies in place is more complicated than the code itself. Some of these sub-systems are implemented using what one would call “AI”. Particularly in the object recognition domain that seems to be the state of the art. But the failure modes and shortcomings of these systems are considered and reasoned about the same way you would do the same with a good old-fashioned kalman-filter based sub-system. It is known that they are going to fail in various situations in various ways. The trick is to engineer the whole system such that it still remains safe despite these sub-systems having these characteristics. I’m not saying that we will have safe self-driving soon. What i’m saying is that you can’t reason about self-driving cars by saying “commercial entity X is spamming me with bad marketing crap. People talk about AI behind said marketing crap. Therefore self driving cars are far away.”
- viraptor 5y ago
  > I think they're a young overseas resident who panicked. Ok, I need to call this out. Unless I missed something, there's nothing to suggest that the person involved is an overseas resident. > but you get a year on your homeland licence before having to sit a local road rules exam. P-plates suggests she's doing a local driving licence. If you area foreigner from quite a few countries you just show your existing driving licence and get a Vic one without any exams and without p-plates.
  - prawn 5y ago
    They're likely going on this, which in local media usually implies they're fleeing back to a home country: "Sakshi Agrawal has been granted conditional bail, despite police protesting she poses a flight risk." And/or from another local article: "The 23-year-old, who is on a bridging visa, was released on bail despite concerns from police prosecutors she could flee the country." "The court heard that Ms Agrawal, whose driver’s licence had previously been suspended..."
  - statop 5y ago
    You're right the linked article doesn't mention it. If you search the drivers name and Tesla, other articles do claim that she is not a citizen or permanent resident of Australia and is there on a bridging visa.
    - ggm 5y ago
      Which is exactly what I did: I had read other local news stories about her initial court appearance where a police case was raised she might be a flight risk.
  - postingawayonhn 5y ago
    > Ok, I need to call this out. Unless I missed something, there's nothing to suggest that the person involved is an overseas resident. Just based on the picture I'd be pretty confident she is. I find it fairly easy to pick out young people from Asia who are recent immigrants based on their dress and hairstyles.
- Khaine 5y ago
  Ignorance is no excuse for not following the law
  - voltaireodactyl 5y ago
    It is, however, an extenuating circumstance.
  - auxfil 5y ago
    I hope you don't write software with that kind of logic. You cannot follow a law that you are ignorant of. The word you're looking for is "no defence". Even that is debatable, subject to the reading of relevant applicable laws. Let's leave laws to lawyers and judges, shall we?
- jaclaz 5y ago
  >In Australia, you stay, you help the victim, you report the accident. That's the law. I don't think Law is much different in any country, maybe in some countries it is not respected due to the possibilities of retribution/extortion you mention.
  - throwaway0a5e 5y ago
    It's easy to make low effort quips about the letter of the law but we all know that even in the first world customary practice can be far more nuanced than the letter of the law. And the gap only gets wider from there.
andrewinardeer 5y ago
> A hit-and-run incident in Melbourne yesterday could set a legal precedent for the use of autonomous driving technologies in Australia. There is no precedent to be set. If you are behind the wheel of a car you are responsible for it's operation. 'Autopilot' or not. And hard to say it was the autopilot's fault for driving off and returning to the police station two hours later.
- deepsun 5y ago
  Well, Mercedes takes legal responsibility for its Autopilot, unlike Tesla.
  - natch 5y ago
    Mercedes says they will only take legal responsibility "while the system is active." The system will deactivate and hand control (and liability) right back to the human just in the nick of time if it encounters any marginal conditions (stormy weather, presence of emergency vehicles, etc…). So it looks like just an attempt to one-up Tesla with a marketing gimmick.
    - deepsun 5y ago
      Tesla puts way more marketing BS than others, so I naturally believe others more.
      - natch 5y ago
        Tesla doesn't advertise. Tesla doesn't even have a PR department… they closed it. >Tesla puts way more marketing BS than others link? waiting…
        deepsun 5y ago
        Sure, just one example: Tesla Truck presentation, by Elon: [1] With unbreakable windows (forbidden for safety, otherwise there would not be carjackings anymore). [1]: https://www.youtube.com/watch?v=m7atGkba-Z8 https://www.youtube.com/watch?v=m7atGkba-Z8
barbs 5y ago
In Melbourne, quite often there are 4 lane roads (2 lanes in each direction) with tramlines in the inner two lanes going in each direction. At the tram stops, passengers need to walk across the outer lane to get to/from the sidewalk, so cars need to stop behind the tram even if they are in the outer lane and not directly behind the tram so that they can make way for passengers. Having moved here from Sydney a few years ago, where there are no trams in this configuration, this took a lot of getting used to, since if I was in the outer lane my natural instinct was to keep driving as the road ahead seems clear at first. I'm not sure how common this is globally, but I wonder if Tesla's autopilot is configured for this edge case? I imagine it would be difficult to program since it would detect the tram as simply another vehicle and the road ahead as clear? Then again, I suppose it should have detected the pedestrian in any case?
- ajayyy 5y ago
  This is a thing in Toronto. There also is a similar senario for all school buses where traffic in both directions must stop. But that might be detectable by the stop sign.
- seanmcdirmid 5y ago
  Tesla autopilot isn’t available in Australia, so probably not. Also, isn’t autopilot just for controlled access highways, unless they were using a beta?
  - onethought 5y ago
    Autopilot and FSD (non beta) are both available in Australia. Autopilot is standard on all models.
- onethought 5y ago
  If you have FSD your car (essential enhanced autopilot) will sometimes detect the little stop sign that pops out of the old Melbourne trams. New ones I haven’t seen. But regular autopilot wouldn’t do anything.
tobyjsullivan 5y ago
I didn’t realize Autopilot disabled the vehicle’s brakes. This poor driver couldn’t stop even after the collision. Seems like a terrible design choice.
- mlindner 5y ago
  Autopilot does not disable vehicle brakes. In fact vehicle brakes immediately disables Autopilot. If they're actually claiming that they had no brakes then that makes it likely they're lying about using Autopilot at all.
  - klabb3 5y ago
    Parent was being sarcastic :)
mlindner 5y ago
The key word here is "blamed" as this has likely nothing to do with Autopilot. The person panicked and fled the scene and then made up stories to pretend they were not the cause of the accident.
jfoster 5y ago
1. A driver who drove off after an accident claims they were using Autopilot at the time. 2. The road photographed in the article isn't a highway, and so I'm not sure it would be accepted by regular Autopilot. 3. Whilst FSD might accept that road, FSD isn't available in Australia. I think the driver's claim that Autopilot was in use needs to be questioned.
- JimmyAustin 5y ago
  Autopilot operates on normal roads. It’s essentially lane following and dynamic cruise control.
  - jfoster 5y ago
    Melbourne doesn't generally have what Autopilot would consider "normal" roads, though. The incident involved someone getting off a tram, so the road would have looked something like this: https://en.wikipedia.org/wiki/Brunswick_East,_Victoria#/media/File:Part-time_tram_lane_on_Nicholson_Street,_Brunswick_East.jpg https://en.wikipedia.org/wiki/Brunswick_East,_Victoria#/medi... In the future, perhaps Teslas will handle this well, but you only need to watch a few videos of them encountering similar markings on YouTube to know that this type of road currently results in massive confusion for a Tesla. Example: https://www.youtube.com/watch?v=sbSDsbDQjSU https://www.youtube.com/watch?v=sbSDsbDQjSU
    - onethought 5y ago
      Elon just posted even FSD beta doesn’t handle streetcars (trams?) in Canada very well. So it’s probably a big gap in their training set… I guess SF trolleys are quite different or mapped or something.
      - jfoster 5y ago
        Yeah, it probably views them as a big truck or something. In any case, I would be really surprised if regular (non-FSD) Autopilot would be happy with the markings on a Melbourne street that has tram lines.

29 points by gravelc 5y ago

ggm 5y ago

The run part: I think they're a young overseas resident who panicked. They returned and handed themselves in to police at the scene. A lot of younger foreigners have no idea what to do in an accident and in their home economy can face retribution, extortion, sometimes at the hands of police. In Australia, you stay, you help the victim, you report the accident. That's the law. In theory all drivers should know this but you get a year on your homeland licence before having to sit a local road rules exam. I have nothing to add regarding self driving. Well I do: I think it's been misnamed and oversold.

genewitch 5y ago
> I think it's been misnamed and oversold. Yep. I marvel at the "Machine Learning AI ad algorithms" that try to sell me junk. I get 3 (three!) emails a day from aliexpress about stuff i already bought ... from aliexpress. Same thing with Amazon. I bought a, lets say "toaster" from amazon, and for the next 6 months every weird amazon ad on the wide internet and every time i hit the amazon home page it's trying to sell me toasters. I don't even see it like "it's like a child, it finds something interesting and repeats it" because even a child gives up after a week. maybe a lot of people don't see "AI" as "old tech" - even though some of the first real "advances" - and i say that with a wry smile - were documented 40 years ago or more. It's complete and utter snake oil. There are some cute toys that AI/ML can do, i can clone certain voices passably, for instance. There's "deepfakes" where a screenshot is enough to maybe fool someone not wearing their glasses at 3 meters, on a smart phone screen. There's obviously the AI in video games, etc. But we're a good 25 years out from a truly autonomous, road-worthy, self-driving human transport. Someone will always have to "look over the shoulder" until then. The day i can hire a private self-driving taxi/limo to pick me up at my house and drive me to an arbitrary location, at any time of day or year, is the day i start to tinfoil my house to prevent the grey goo from coming inside. edit: i should note that i would ab-so-loutely love a car service that was cheaper than hiring a limo for a week, if you can even do that. If the car is driving by itself, i can sleep while it gets from interesting place in the US to interesting place, and spend most of my time awake documenting and photographing and interviewing people, rather than hating interstate 10, 20, and 40; which is how most of my "road trips" for the past 20 years have gone. I've traveled this country "a lot", and i rarely have cute stories about places i've been to because i've either been completely wiped and just slept there somewhere, or i was in a hurry to beat the sun/state police/whatever through the state. I'd love to spend a few weeks in wyoming and montana and wisconsin, but if i have to drive it, the only thing i'll publish is a couple of shots of mountains, the kids posing in front of some tourist trap, and that's about it. I passively consume, yet nearly universally ignore and eschew all "AI" news.
- fxtentacle 5y ago
  Where AI has made really great advancements is all the stuff that you can memorize. Like GitHub Co-Pilot reurgitating John Carmack's source code. Also, stochastic language models have reached the point where they seem like magic, e.g. automated YouTube subtitles and translations, or text to speech. But that all works by memorizing word fragments (CTC) and/or memorizing sentence fragments (n-Gram).
- krisoft 5y ago
  > But we're a good 25 years out from a truly autonomous, road-worthy, self-driving human transport. It sounds like your reasoning goes like this: my interactions with things which are supposedly AI are bad. Self-driving cars are also some form of AI, therefore self-driving cars must be bad and or far away too. Did I understood you right? The problem with that logic is that there is nothing in common in implementation/architecture/incentives between the things you mention and self-driving technology. Nobody, well nearly nobody, tries to implement self-driving cars in a blackbox “AI” fashion. What I mean is that we don’t just throw sensor data at a neural network, then squint and say “i recon it is going to drive well now”. That would be madness. Most approaches break down the problem into sub-problems covered by sub-systems. The sub-systems are fed information with known error properties and engineered to the specifications. The failure modes are painstakingly traced through and documented. Then in turn assemblies of these subsystems and the whole are reasoned similarly. Fault trees are drawn, the operational domain is considered. The reasoning why the engineers think the system is safe and have the right redundancies in place is more complicated than the code itself. Some of these sub-systems are implemented using what one would call “AI”. Particularly in the object recognition domain that seems to be the state of the art. But the failure modes and shortcomings of these systems are considered and reasoned about the same way you would do the same with a good old-fashioned kalman-filter based sub-system. It is known that they are going to fail in various situations in various ways. The trick is to engineer the whole system such that it still remains safe despite these sub-systems having these characteristics. I’m not saying that we will have safe self-driving soon. What i’m saying is that you can’t reason about self-driving cars by saying “commercial entity X is spamming me with bad marketing crap. People talk about AI behind said marketing crap. Therefore self driving cars are far away.”
viraptor 5y ago
> I think they're a young overseas resident who panicked. Ok, I need to call this out. Unless I missed something, there's nothing to suggest that the person involved is an overseas resident. > but you get a year on your homeland licence before having to sit a local road rules exam. P-plates suggests she's doing a local driving licence. If you area foreigner from quite a few countries you just show your existing driving licence and get a Vic one without any exams and without p-plates.
- prawn 5y ago
  They're likely going on this, which in local media usually implies they're fleeing back to a home country: "Sakshi Agrawal has been granted conditional bail, despite police protesting she poses a flight risk." And/or from another local article: "The 23-year-old, who is on a bridging visa, was released on bail despite concerns from police prosecutors she could flee the country." "The court heard that Ms Agrawal, whose driver’s licence had previously been suspended..."
- statop 5y ago
  You're right the linked article doesn't mention it. If you search the drivers name and Tesla, other articles do claim that she is not a citizen or permanent resident of Australia and is there on a bridging visa.
  - ggm 5y ago
    Which is exactly what I did: I had read other local news stories about her initial court appearance where a police case was raised she might be a flight risk.
- postingawayonhn 5y ago
  > Ok, I need to call this out. Unless I missed something, there's nothing to suggest that the person involved is an overseas resident. Just based on the picture I'd be pretty confident she is. I find it fairly easy to pick out young people from Asia who are recent immigrants based on their dress and hairstyles.
Khaine 5y ago
Ignorance is no excuse for not following the law
- voltaireodactyl 5y ago
  It is, however, an extenuating circumstance.
- auxfil 5y ago
  I hope you don't write software with that kind of logic. You cannot follow a law that you are ignorant of. The word you're looking for is "no defence". Even that is debatable, subject to the reading of relevant applicable laws. Let's leave laws to lawyers and judges, shall we?
jaclaz 5y ago
>In Australia, you stay, you help the victim, you report the accident. That's the law. I don't think Law is much different in any country, maybe in some countries it is not respected due to the possibilities of retribution/extortion you mention.
- throwaway0a5e 5y ago
  It's easy to make low effort quips about the letter of the law but we all know that even in the first world customary practice can be far more nuanced than the letter of the law. And the gap only gets wider from there.

andrewinardeer 5y ago

> A hit-and-run incident in Melbourne yesterday could set a legal precedent for the use of autonomous driving technologies in Australia. There is no precedent to be set. If you are behind the wheel of a car you are responsible for it's operation. 'Autopilot' or not. And hard to say it was the autopilot's fault for driving off and returning to the police station two hours later.

deepsun 5y ago
Well, Mercedes takes legal responsibility for its Autopilot, unlike Tesla.
- natch 5y ago
  Mercedes says they will only take legal responsibility "while the system is active." The system will deactivate and hand control (and liability) right back to the human just in the nick of time if it encounters any marginal conditions (stormy weather, presence of emergency vehicles, etc…). So it looks like just an attempt to one-up Tesla with a marketing gimmick.
  - deepsun 5y ago
    Tesla puts way more marketing BS than others, so I naturally believe others more.
    - natch 5y ago
      Tesla doesn't advertise. Tesla doesn't even have a PR department… they closed it. >Tesla puts way more marketing BS than others link? waiting…
      - deepsun 5y ago
        Sure, just one example: Tesla Truck presentation, by Elon: [1] With unbreakable windows (forbidden for safety, otherwise there would not be carjackings anymore). [1]: https://www.youtube.com/watch?v=m7atGkba-Z8 https://www.youtube.com/watch?v=m7atGkba-Z8

barbs 5y ago

In Melbourne, quite often there are 4 lane roads (2 lanes in each direction) with tramlines in the inner two lanes going in each direction. At the tram stops, passengers need to walk across the outer lane to get to/from the sidewalk, so cars need to stop behind the tram even if they are in the outer lane and not directly behind the tram so that they can make way for passengers. Having moved here from Sydney a few years ago, where there are no trams in this configuration, this took a lot of getting used to, since if I was in the outer lane my natural instinct was to keep driving as the road ahead seems clear at first. I'm not sure how common this is globally, but I wonder if Tesla's autopilot is configured for this edge case? I imagine it would be difficult to program since it would detect the tram as simply another vehicle and the road ahead as clear? Then again, I suppose it should have detected the pedestrian in any case?

ajayyy 5y ago
This is a thing in Toronto. There also is a similar senario for all school buses where traffic in both directions must stop. But that might be detectable by the stop sign.
seanmcdirmid 5y ago
Tesla autopilot isn’t available in Australia, so probably not. Also, isn’t autopilot just for controlled access highways, unless they were using a beta?
- onethought 5y ago
  Autopilot and FSD (non beta) are both available in Australia. Autopilot is standard on all models.
onethought 5y ago
If you have FSD your car (essential enhanced autopilot) will sometimes detect the little stop sign that pops out of the old Melbourne trams. New ones I haven’t seen. But regular autopilot wouldn’t do anything.

tobyjsullivan 5y ago

I didn’t realize Autopilot disabled the vehicle’s brakes. This poor driver couldn’t stop even after the collision. Seems like a terrible design choice.

mlindner 5y ago
Autopilot does not disable vehicle brakes. In fact vehicle brakes immediately disables Autopilot. If they're actually claiming that they had no brakes then that makes it likely they're lying about using Autopilot at all.
- klabb3 5y ago
  Parent was being sarcastic :)

mlindner 5y ago

The key word here is "blamed" as this has likely nothing to do with Autopilot. The person panicked and fled the scene and then made up stories to pretend they were not the cause of the accident.

jfoster 5y ago

1. A driver who drove off after an accident claims they were using Autopilot at the time. 2. The road photographed in the article isn't a highway, and so I'm not sure it would be accepted by regular Autopilot. 3. Whilst FSD might accept that road, FSD isn't available in Australia. I think the driver's claim that Autopilot was in use needs to be questioned.

JimmyAustin 5y ago
Autopilot operates on normal roads. It’s essentially lane following and dynamic cruise control.
- jfoster 5y ago
  Melbourne doesn't generally have what Autopilot would consider "normal" roads, though. The incident involved someone getting off a tram, so the road would have looked something like this: https://en.wikipedia.org/wiki/Brunswick_East,_Victoria#/media/File:Part-time_tram_lane_on_Nicholson_Street,_Brunswick_East.jpg https://en.wikipedia.org/wiki/Brunswick_East,_Victoria#/medi... In the future, perhaps Teslas will handle this well, but you only need to watch a few videos of them encountering similar markings on YouTube to know that this type of road currently results in massive confusion for a Tesla. Example: https://www.youtube.com/watch?v=sbSDsbDQjSU https://www.youtube.com/watch?v=sbSDsbDQjSU
  - onethought 5y ago
    Elon just posted even FSD beta doesn’t handle streetcars (trams?) in Canada very well. So it’s probably a big gap in their training set… I guess SF trolleys are quite different or mapped or something.
    - jfoster 5y ago
      Yeah, it probably views them as a big truck or something. In any case, I would be really surprised if regular (non-FSD) Autopilot would be happy with the markings on a Melbourne street that has tram lines.