8 ms·

Cruise Knew Its Self-Driving Cars Had Problems Recognizing Children

oldgradstudent 3y ago
> According to one safety memo, Cruise began operating fewer driverless cars during daytime hours to avoid encountering children, a move it deemed effective at mitigating the overall risk without fixing the underlying technical problem. In August, Cruise announced the cuts to daytime ride operations in San Francisco but made no mention of its attempt to lower risk to local children. Edit: looks like Cruise is not long for this world. Insiders seem to have lost faith and are starting to leak. I'd bet it simply cannot withstand any serious scrutiny into its safety levels, practices, and culture.
- nrfulton 3y ago
  For Hackers, the money quote, IMO, is the one about competing technical cultures within Software^1: > Koopman, who has a long career working on AV safety, faulted the data-driven culture of machine learning that leads tech companies to contemplate hazards only after they’ve encountered them, rather than before. I haven't yet figured out how to effectively and efficiently communicate this mindset shift to folks educated primarily in ML culture. I am not sure I ever will. The closest I've come to an elevator pitch of the mindset shift is something like: "when human lives on the line and you're taking an absolutely massive number of samples IRL, doesn't it make sense to stop thinking in terms of analysis/probability and start thinking in terms of topology/nondeterminism? Ie, when you sample A LOT, unlikely shit happens. Manageable risks if you're selling ads I suppose. But not so acceptable if you're deciding whether a giant piece of unforgiving steel saw a bollard or your daughter." [^1]: I read the quote-block in your post and immediately thought "Koopman". Then read the article and, sure enough, they're quoting Koopman. What a tragedy. The message was not only out there, but out there so loudly that the vague shape of the warning has a particular person's name attached to it in my mind. Yet, here we are.
  - steveBK123 3y ago
    Autonomous car devs seem to lack a mindset of "things that never happened before happen all the time" No matter how many miles your car drives and how much data it collects, it will encounter a novel situation on the road. Unless it has higher levels of context / overriding safeguards / etc, a data driven only ML approach is going to fail dangerously. One favorite example is the year old video of Tesla FSD attempting to unprotected left turn thru an oncoming trolley car while the center display 3D rendered the trolley car in motion. Clearly there is no overriding safety guardrail model above the path finding model. If the car can 3d render the object it is aware it exists. And so we go on being perpetually "five years away" from self driving.
    - schiffern 3y ago
      > a data driven only ML approach is going to fail https://en.wikipedia.org/wiki/Nirvana_fallacy https://en.wikipedia.org/wiki/Nirvana_fallacy
      - nrfulton 3y ago
        > https://en.wikipedia.org/wiki/Nirvana_fallacy https://en.wikipedia.org/wiki/Nirvana_fallacy This is a common counter-argument, and sounds reasonable on face, and is even true in the limit. But we're not anywhere close to the limit. Reminding the dear reader of the actual context clears this point up quickly: >> One favorite example is the year old video of Tesla FSD attempting to unprotected left turn thru an oncoming trolley car while the center display 3D rendered the trolley car in motion. We're not asking for Nirvana. We're asking for not throwing up one's hands and declaring that perfect CV is impossible as one mows down pedestrians and trolley cars that one's CV system is clearly capable of identifying. For example.
  - oldgradstudent 3y ago
    Koopman is great, I've been considering starting a fan club ever since I've heard him on some podcast a few years ago. BTW, just recently I needed to choose a CRC polynomial and discovered his work on that field as well. https://news.ycombinator.com/item?id=38172322 https://news.ycombinator.com/item?id=38172322
ra7 3y ago
There were clear signs that Cruise was rushing things with their expansion, but I didn’t know the performance was this bad. Based on developments from the past few days [1], their safety culture seems to be closer to Tesla’s than it is to Waymo’s. [1] https://news.ycombinator.com/item?id=38002752 https://news.ycombinator.com/item?id=38002752
- oldgradstudent 3y ago
  We'll see about Waymo's safety culture after their first serious incident. Transparency is sorely lacking in this sector.
  - chpatrick 3y ago
    The fact that there hasn't been one after years of driving is a pretty good sign no?
    - oldgradstudent 3y ago
      They barely drove a few million miles in ideal conditions and where children are relatively uncommon. Edit: removed a Cruise incident.
      - unshavedyak 3y ago
        Your link is a Cruise (according to the X/Tweet/Thing), but it sounded like you were using that as evidence of faulty Waymo behavior.
        oldgradstudent 3y ago
        Yeah, thanks. Got distracted.
      - ramraj07 3y ago
        I saw numerous Reddit posts of self driving cars causing mayhem. Every single one of them was a Cruise car. Waymo is doing something right. If the cruise fiasco forces more regulation that’s only a good thing. Hopefully waymo passes them all.
_ea1k 3y ago
It sounds more like a path prediction problem than one of not seeing them at all. I'm guessing they expected it to still handle them as well as a human driver would, even if it relied on the adult pedestrian logic. I have doubts as to this being the right choice, but it seems less unreasonable than what I thought from the title.
- saghm 3y ago
  "Our cars could see the kids; they just chose to run into them anyways" is still pretty unreasonable. Relying on tests for one scenario and assuming it works for another is a pretty good way to...well, make cars that hit people, I guess.
  - plagiarist 3y ago
    I wonder if we will see some jail time for at least reckless endangerment.
- oldgradstudent 3y ago
  They were not able to recognize them as kids rather than generic humans. Kids behave differently and more erratically. The details are not that imporant. The main point is that they were aware they were couldn't match human performance on kid safety yet still put their cars on the street. all while talking about the terrible toll of car crashes on children. > Two months ago, Kyle Vogt, the chief executive of Cruise, choked up as he recounted how a driver had killed a 4-year-old girl in a stroller at a San Francisco intersection. “It barely made the news,” he said, pausing to collect himself. “Sorry. I get emotional.” https://www.nytimes.com/2023/11/03/technology/cruise-general-motors-self-driving-cars.html https://www.nytimes.com/2023/11/03/technology/cruise-general...
  - candiddevmike 3y ago
    [flagged]
    - jvanderbot 3y ago
bparsons 3y ago
I was kind of shocked in 2019 to see those cars driving around SF. Most jurisdictions would never allow this in a million years. Why would you allow some tech company to develop its technology at the risk of pedestrian safety? What benefit does this have to the people living in these neighborhoods?
- foobarian 3y ago
  When cab drivers are too poor to afford living in your town you gotta come up with some kind of plan B.
  - ghaff 3y ago
    Raise prices and the people who need/want to take cabs need to deal with it or move elsewhere.
  - asveikau 3y ago
    They just do extremely long commutes from parts very far east. They're already in the car and already used to driving for a long time, may as well add Visalia to SF. (Visalia is especially far, but I had a driver in SF tell me once that's where he drove in from.)
- vkou 3y ago
  > Why would you allow some tech company to develop its technology at the risk of pedestrian safety? If we cared about pedestrian safety, we wouldn't let ~half[1] the drivers on the roads drive. [1] I kid, it's more like 5-10%.
  - happytoexplain 3y ago
    We can't reasonably measure the safety of individuals until they do something concrete, like get caught driving recklessly/drunk/distracted. If a company endangers people because they wanted to hit their timeline or save face or whatever, that's pure evil. They're simply incomparable. We can address both things. The "but people are statistically worse" argument is a horrible, thoughtless "gotcha" in the form of whataboutism that makes light of people's lives.
cratermoon 3y ago
After hearing about how often their cars need a human to intervene, I started wondering about that incident where a Cruise taxi ran over a person that had been thrown into its path, stopped, and then started up again to move to the curb.[1] Did the car stop, notify the mothership, and then have a person direct it to pull out of the flow of traffic? How would we know? If the car moved on its own, that's bad. If the car moved after being contacting a "remote assistant agent", that's bad, too. 1 "The Cruise car ran over her, briefly stopped and then dragged her some 20 feet before pulling to the curb, causing severe injuries." https://www.nytimes.com/2023/11/03/technology/cruise-general-motors-self-driving-cars.html https://www.nytimes.com/2023/11/03/technology/cruise-general...
monero-xmr 3y ago
This is the wider problem of AI in general. It is right some percent of the time. You can only use AI in situations where being wrong randomly, without anyone to sue or blame, is OK. Why people keep trying to use AI to drive cars, I have no idea. It’s like starting with the hardest of hard problems immediately. Fools errand, if you work in self driving cars get out ASAP while the rest of AI is still hot and you can get paid. Because the other stuff is on cusp of the trough of despair given the increasingly desperate emails from salesmen I get from XYZ.ai startups.
- tux1968 3y ago
  If it follows the trajectory chess AI took, it will seem pathetic for quite some time, right up until the moment it outperforms every human on the planet by a wide margin.
  - monero-xmr 3y ago
    Nobody wants to play chess AI. We want to play against humans.
  - steveBK123 3y ago
    Isn't it also like Go though where AI changes the game, and once humans adapt there are adversarial methods that AI trained on human players fail to understand? So AI entering a space changes a space, and humans respond faster to that change. I think this is an underconsidered part of autonomous cars. As soon as every vehicle in Manhattan is autonomous, people will bully them into sitting idle at green lights while walkers, bikers, scooters and everyone else flagrantly jaywalk in front of them. With a human on human interaction there is a chance the driver will try to kill you, or you feel guilt at inconveniencing them. With an AI trained to drive safely, the safest thing to do will accept being bullied. A similar thing will happen in a mixed human & autonomous environment where human drivers will abuse the AI drivers.
    - ghaff 3y ago
      >As soon as every vehicle in Manhattan is autonomous, people will bully them into sitting idle at green lights while walkers, bikers, scooters and everyone else flagrantly jaywalk in front of them. Sounds like pretty much how things work now so long as anything has paused the road traffic--which is all the time.
ajsnigrutin 3y ago
If a (usually lead) construction engineer notices (or is told about) a fault/design issue/problem, knows that it is dangerous, and does nothing about it, then something bad happens because of that, in many cases and many countries, the engineer ends up personally responsible for the outcome. This should be expended to other industries as well.
- batiudrami 3y ago
  It’s all engineering except “software engineering”. You sign it, you own it. The senior techcnical leads who knew about this should face legal repercussions.
  - zardo 3y ago
    When has that happened with any discipline other then civil engineering?
    - batiudrami 3y ago
      It happens routinely in my industry (rail), which is primarily civil, electrical and systems engineering. In all engineering degrees it is impressed upon you that you are responsible for your work and if you behave negligently or outside your competency you can be held accountable for the consequences of your actions.
      - ajsnigrutin 3y ago
        Same with pure electrical... both at technician level (which is relatively simple, just a few standards to keep up with, choosing the right wires and devices) and at pure engineering level, where you have a lot of formulas instead of tables, and if you miscalculate something when connecting something huge, you can bring the whole grid down, stuff can burn, explode, people can get hurt, and you can be personally kept responsible... and that goes doubly so, if you've been warned in advance and knew that there was an issue.
charcircuit 3y ago
The title makes it sound like the car wasn't detecting children as opposed to misclassifying children as adult humans.
- thegrim33 3y ago
  True, but still, the way I drive is different when there's a little child on the side of the road vs. when there's an adult on the side of the road, as there's a higher expectation of one doing something dumb and running into the road than the other, so it not being able to differentiate between the two isn't the best.
  - Veserv 3y ago
    I mean, Tesla gets away with straight up not even recognizing children [1] and running them down in crosswalks in high visibility vests as demonstrated publicly over a year ago. Over a year later and they still have not fixed it or pulled it [2]. Why bother with safety when a criminal disregard for safety makes you the world’s largest car company and the world’s richest person? [1] https://dawnproject.com/dawn-project-safety-test-tesla-full-self-driving-runs-down-child-mannequin-in-school-crosswalk/ https://dawnproject.com/dawn-project-safety-test-tesla-full-... [2] https://www.youtube.com/watch?v=CPMoLmQgxTw https://www.youtube.com/watch?v=CPMoLmQgxTw
SheinhardtWigCo 3y ago
> “Our driverless operations have always performed higher than a human benchmark, and we constantly evaluate and mitigate new risks to continuously improve,” said Erik Moser, Cruise’s director of communications. “We have the lowest risk tolerance for contact with children and treat them with the highest safety priority. No vehicle — human operated or autonomous — will have zero risk of collision.” Notice that this contains nothing of substance: no "human benchmark" is actually defined; nobody was asking if they constantly evaluate risks (do they also remember to breathe in and out every few seconds?); "the lowest risk tolerance" is meaningless, as is "the highest safety priority"; and obviously no vehicle will have "zero risk of collision", just as no vehicle has zero risk of being hit by a meteorite.
- oldgradstudent 3y ago
  Until the leaks started Cruise had zero transparency. Even when reporting was mandated, they twisted the definitions to make them meaningless. Cruise cars spend 2-4% of the time under remote assistance, but Cruise reported one disengagement per 95,000 miles in their most recent DMV report.
steveBK123 3y ago
It's also curious how quickly this gets voted off the front page because it doesn't fit the "AI eating the world" narrative sufficiently.
- preciousoo 3y ago
  I accidentally closed the post and it took me 5 mins to find it. Damn.

> According to one safety memo, Cruise began operating fewer driverless cars during daytime hours to avoid encountering children, a move it deemed effective at mitigating the overall risk without fixing the underlying technical problem. In August, Cruise announced the cuts to daytime ride operations in San Francisco but made no mention of its attempt to lower risk to local children. Edit: looks like Cruise is not long for this world. Insiders seem to have lost faith and are starting to leak. I'd bet it simply cannot withstand any serious scrutiny into its safety levels, practices, and culture.

nrfulton 3y ago
For Hackers, the money quote, IMO, is the one about competing technical cultures within Software^1: > Koopman, who has a long career working on AV safety, faulted the data-driven culture of machine learning that leads tech companies to contemplate hazards only after they’ve encountered them, rather than before. I haven't yet figured out how to effectively and efficiently communicate this mindset shift to folks educated primarily in ML culture. I am not sure I ever will. The closest I've come to an elevator pitch of the mindset shift is something like: "when human lives on the line and you're taking an absolutely massive number of samples IRL, doesn't it make sense to stop thinking in terms of analysis/probability and start thinking in terms of topology/nondeterminism? Ie, when you sample A LOT, unlikely shit happens. Manageable risks if you're selling ads I suppose. But not so acceptable if you're deciding whether a giant piece of unforgiving steel saw a bollard or your daughter." [^1]: I read the quote-block in your post and immediately thought "Koopman". Then read the article and, sure enough, they're quoting Koopman. What a tragedy. The message was not only out there, but out there so loudly that the vague shape of the warning has a particular person's name attached to it in my mind. Yet, here we are.
- steveBK123 3y ago
  Autonomous car devs seem to lack a mindset of "things that never happened before happen all the time" No matter how many miles your car drives and how much data it collects, it will encounter a novel situation on the road. Unless it has higher levels of context / overriding safeguards / etc, a data driven only ML approach is going to fail dangerously. One favorite example is the year old video of Tesla FSD attempting to unprotected left turn thru an oncoming trolley car while the center display 3D rendered the trolley car in motion. Clearly there is no overriding safety guardrail model above the path finding model. If the car can 3d render the object it is aware it exists. And so we go on being perpetually "five years away" from self driving.
  - schiffern 3y ago
    > a data driven only ML approach is going to fail https://en.wikipedia.org/wiki/Nirvana_fallacy https://en.wikipedia.org/wiki/Nirvana_fallacy
    - nrfulton 3y ago
      > https://en.wikipedia.org/wiki/Nirvana_fallacy https://en.wikipedia.org/wiki/Nirvana_fallacy This is a common counter-argument, and sounds reasonable on face, and is even true in the limit. But we're not anywhere close to the limit. Reminding the dear reader of the actual context clears this point up quickly: >> One favorite example is the year old video of Tesla FSD attempting to unprotected left turn thru an oncoming trolley car while the center display 3D rendered the trolley car in motion. We're not asking for Nirvana. We're asking for not throwing up one's hands and declaring that perfect CV is impossible as one mows down pedestrians and trolley cars that one's CV system is clearly capable of identifying. For example.
- oldgradstudent 3y ago
  Koopman is great, I've been considering starting a fan club ever since I've heard him on some podcast a few years ago. BTW, just recently I needed to choose a CRC polynomial and discovered his work on that field as well. https://news.ycombinator.com/item?id=38172322 https://news.ycombinator.com/item?id=38172322

ra7 3y ago

There were clear signs that Cruise was rushing things with their expansion, but I didn’t know the performance was this bad. Based on developments from the past few days [1], their safety culture seems to be closer to Tesla’s than it is to Waymo’s. [1] https://news.ycombinator.com/item?id=38002752 https://news.ycombinator.com/item?id=38002752

oldgradstudent 3y ago
We'll see about Waymo's safety culture after their first serious incident. Transparency is sorely lacking in this sector.
- chpatrick 3y ago
  The fact that there hasn't been one after years of driving is a pretty good sign no?
  - oldgradstudent 3y ago
    They barely drove a few million miles in ideal conditions and where children are relatively uncommon. Edit: removed a Cruise incident.
    - unshavedyak 3y ago
      Your link is a Cruise (according to the X/Tweet/Thing), but it sounded like you were using that as evidence of faulty Waymo behavior.
      - oldgradstudent 3y ago
        Yeah, thanks. Got distracted.
    - ramraj07 3y ago
      I saw numerous Reddit posts of self driving cars causing mayhem. Every single one of them was a Cruise car. Waymo is doing something right. If the cruise fiasco forces more regulation that’s only a good thing. Hopefully waymo passes them all.

_ea1k 3y ago

It sounds more like a path prediction problem than one of not seeing them at all. I'm guessing they expected it to still handle them as well as a human driver would, even if it relied on the adult pedestrian logic. I have doubts as to this being the right choice, but it seems less unreasonable than what I thought from the title.

saghm 3y ago
"Our cars could see the kids; they just chose to run into them anyways" is still pretty unreasonable. Relying on tests for one scenario and assuming it works for another is a pretty good way to...well, make cars that hit people, I guess.
- plagiarist 3y ago
  I wonder if we will see some jail time for at least reckless endangerment.
oldgradstudent 3y ago
They were not able to recognize them as kids rather than generic humans. Kids behave differently and more erratically. The details are not that imporant. The main point is that they were aware they were couldn't match human performance on kid safety yet still put their cars on the street. all while talking about the terrible toll of car crashes on children. > Two months ago, Kyle Vogt, the chief executive of Cruise, choked up as he recounted how a driver had killed a 4-year-old girl in a stroller at a San Francisco intersection. “It barely made the news,” he said, pausing to collect himself. “Sorry. I get emotional.” https://www.nytimes.com/2023/11/03/technology/cruise-general-motors-self-driving-cars.html https://www.nytimes.com/2023/11/03/technology/cruise-general...
- candiddevmike 3y ago
  [flagged]
  - jvanderbot 3y ago

bparsons 3y ago

I was kind of shocked in 2019 to see those cars driving around SF. Most jurisdictions would never allow this in a million years. Why would you allow some tech company to develop its technology at the risk of pedestrian safety? What benefit does this have to the people living in these neighborhoods?

foobarian 3y ago
When cab drivers are too poor to afford living in your town you gotta come up with some kind of plan B.
- ghaff 3y ago
  Raise prices and the people who need/want to take cabs need to deal with it or move elsewhere.
- asveikau 3y ago
  They just do extremely long commutes from parts very far east. They're already in the car and already used to driving for a long time, may as well add Visalia to SF. (Visalia is especially far, but I had a driver in SF tell me once that's where he drove in from.)
vkou 3y ago
> Why would you allow some tech company to develop its technology at the risk of pedestrian safety? If we cared about pedestrian safety, we wouldn't let ~half[1] the drivers on the roads drive. [1] I kid, it's more like 5-10%.
- happytoexplain 3y ago
  We can't reasonably measure the safety of individuals until they do something concrete, like get caught driving recklessly/drunk/distracted. If a company endangers people because they wanted to hit their timeline or save face or whatever, that's pure evil. They're simply incomparable. We can address both things. The "but people are statistically worse" argument is a horrible, thoughtless "gotcha" in the form of whataboutism that makes light of people's lives.

cratermoon 3y ago

After hearing about how often their cars need a human to intervene, I started wondering about that incident where a Cruise taxi ran over a person that had been thrown into its path, stopped, and then started up again to move to the curb.[1] Did the car stop, notify the mothership, and then have a person direct it to pull out of the flow of traffic? How would we know? If the car moved on its own, that's bad. If the car moved after being contacting a "remote assistant agent", that's bad, too. 1 "The Cruise car ran over her, briefly stopped and then dragged her some 20 feet before pulling to the curb, causing severe injuries." https://www.nytimes.com/2023/11/03/technology/cruise-general-motors-self-driving-cars.html https://www.nytimes.com/2023/11/03/technology/cruise-general...

monero-xmr 3y ago

This is the wider problem of AI in general. It is right some percent of the time. You can only use AI in situations where being wrong randomly, without anyone to sue or blame, is OK. Why people keep trying to use AI to drive cars, I have no idea. It’s like starting with the hardest of hard problems immediately. Fools errand, if you work in self driving cars get out ASAP while the rest of AI is still hot and you can get paid. Because the other stuff is on cusp of the trough of despair given the increasingly desperate emails from salesmen I get from XYZ.ai startups.

tux1968 3y ago
If it follows the trajectory chess AI took, it will seem pathetic for quite some time, right up until the moment it outperforms every human on the planet by a wide margin.
- monero-xmr 3y ago
  Nobody wants to play chess AI. We want to play against humans.
- steveBK123 3y ago
  Isn't it also like Go though where AI changes the game, and once humans adapt there are adversarial methods that AI trained on human players fail to understand? So AI entering a space changes a space, and humans respond faster to that change. I think this is an underconsidered part of autonomous cars. As soon as every vehicle in Manhattan is autonomous, people will bully them into sitting idle at green lights while walkers, bikers, scooters and everyone else flagrantly jaywalk in front of them. With a human on human interaction there is a chance the driver will try to kill you, or you feel guilt at inconveniencing them. With an AI trained to drive safely, the safest thing to do will accept being bullied. A similar thing will happen in a mixed human & autonomous environment where human drivers will abuse the AI drivers.
  - ghaff 3y ago
    >As soon as every vehicle in Manhattan is autonomous, people will bully them into sitting idle at green lights while walkers, bikers, scooters and everyone else flagrantly jaywalk in front of them. Sounds like pretty much how things work now so long as anything has paused the road traffic--which is all the time.

ajsnigrutin 3y ago

If a (usually lead) construction engineer notices (or is told about) a fault/design issue/problem, knows that it is dangerous, and does nothing about it, then something bad happens because of that, in many cases and many countries, the engineer ends up personally responsible for the outcome. This should be expended to other industries as well.

batiudrami 3y ago
It’s all engineering except “software engineering”. You sign it, you own it. The senior techcnical leads who knew about this should face legal repercussions.
- zardo 3y ago
  When has that happened with any discipline other then civil engineering?
  - batiudrami 3y ago
    It happens routinely in my industry (rail), which is primarily civil, electrical and systems engineering. In all engineering degrees it is impressed upon you that you are responsible for your work and if you behave negligently or outside your competency you can be held accountable for the consequences of your actions.
    - ajsnigrutin 3y ago
      Same with pure electrical... both at technician level (which is relatively simple, just a few standards to keep up with, choosing the right wires and devices) and at pure engineering level, where you have a lot of formulas instead of tables, and if you miscalculate something when connecting something huge, you can bring the whole grid down, stuff can burn, explode, people can get hurt, and you can be personally kept responsible... and that goes doubly so, if you've been warned in advance and knew that there was an issue.

charcircuit 3y ago

The title makes it sound like the car wasn't detecting children as opposed to misclassifying children as adult humans.

thegrim33 3y ago
True, but still, the way I drive is different when there's a little child on the side of the road vs. when there's an adult on the side of the road, as there's a higher expectation of one doing something dumb and running into the road than the other, so it not being able to differentiate between the two isn't the best.
- Veserv 3y ago
  I mean, Tesla gets away with straight up not even recognizing children [1] and running them down in crosswalks in high visibility vests as demonstrated publicly over a year ago. Over a year later and they still have not fixed it or pulled it [2]. Why bother with safety when a criminal disregard for safety makes you the world’s largest car company and the world’s richest person? [1] https://dawnproject.com/dawn-project-safety-test-tesla-full-self-driving-runs-down-child-mannequin-in-school-crosswalk/ https://dawnproject.com/dawn-project-safety-test-tesla-full-... [2] https://www.youtube.com/watch?v=CPMoLmQgxTw https://www.youtube.com/watch?v=CPMoLmQgxTw

SheinhardtWigCo 3y ago

> “Our driverless operations have always performed higher than a human benchmark, and we constantly evaluate and mitigate new risks to continuously improve,” said Erik Moser, Cruise’s director of communications. “We have the lowest risk tolerance for contact with children and treat them with the highest safety priority. No vehicle — human operated or autonomous — will have zero risk of collision.” Notice that this contains nothing of substance: no "human benchmark" is actually defined; nobody was asking if they constantly evaluate risks (do they also remember to breathe in and out every few seconds?); "the lowest risk tolerance" is meaningless, as is "the highest safety priority"; and obviously no vehicle will have "zero risk of collision", just as no vehicle has zero risk of being hit by a meteorite.

oldgradstudent 3y ago
Until the leaks started Cruise had zero transparency. Even when reporting was mandated, they twisted the definitions to make them meaningless. Cruise cars spend 2-4% of the time under remote assistance, but Cruise reported one disengagement per 95,000 miles in their most recent DMV report.

steveBK123 3y ago

It's also curious how quickly this gets voted off the front page because it doesn't fit the "AI eating the world" narrative sufficiently.

preciousoo 3y ago
I accidentally closed the post and it took me 5 mins to find it. Damn.