25 ms·

San Francisco raises Tesla 'self-driving' safety concerns as public test nears

141 points by camjohnson26 5y ago

happytiger 5y ago
I’m confused by sf. They did this for Uber in 2016, Amazon’s zoox, and a host of other companies, etc. Tesla has a safety record that’s not unimpressive compared to the 36k people who die every year even though their failures have been more public and newsworthy. Their stats have been relatively objective. We also have a framework for this: https://www.nhtsa.gov/technology-innovation/automated-vehicles-safety https://www.nhtsa.gov/technology-innovation/automated-vehicl... As there is a framework in place already, shouldn’t we push forward? As long as we are reducing the statistical risk of driving below the risk of human drivers, shouldn’t that be the litmus for whether these programs are allowed to progress? I know I am being simplistic, and I understand that no statistical model will ever make up for the loss of a family member, the generally accepted foundation for the morality of self-driving car technologies hinges on whether they can beat the human-driven statistical equivalent. If there is concern about that they shouldn’t continue. But if there isn’t, these experiments should progress as they could ultimately reduce mortality. There were 3,723 deaths in California in 2020. The lifetime odds of dying in a motor-vehicle crash for a person born in 2019 were 1 in 107. These odds suck. Is that really acceptable? Can we do better? We know we can. If we can reduce that number we should. If we can’t, we should restrict this technology’s rollout until it can. Am I overly objective? I’m not an expert. I mean ultimately I want legislation built on objective data and loving values with human compassion at the core. I feel like sometimes we see these decisions based on how fast the system can adapt the ocean of existing automotive and roadway regulatory compliance to the technology changes, rather than new realities. People have no idea how much the books and legal cases that many cal trans engineers have spent their lives developing stand in the way of self-driving vehicles. I honestly think the biggest issue is probably the legal a s regulatory overhead of responding to the technology rather than the technology itself, but since it’s so hard to quantify I don’t often bring this up. But the biggest passion I have seen if from those same engineering minds, who truly understand the complexity, but also want to see it happen. Their timeframes are just long. I’ve heard 2045 and 2050. Anyone more expert than I care to weigh in? I’d love to learn more. I am not trying to weigh in with any merit — I am a nubile entrant and simply sharing the output of conversation with people who will always be more qualified and I, but I’d love to learn more about this.
- lotophage 5y ago
  > I’m confused by sf. They did this for Uber in 2016, Amazon’s zoox, and a host of other companies, etc. My understanding is that those companies had trained test drivers, but Telsa is pushing this out as a beta feature update and allowing the consumers be their test drivers?
  - happytiger 5y ago
    That is a critical difference for sure.
  - svet_0 5y ago
    > trained test drivers Are normal people paid minimum wage who become complacent and occasionally fall asleep (e.g. Uber accident), I would argue non-paid Tesla customers are at least as good as "trained drivers". There are 2K drivers using FSD beta for almost a year, w/o any accidents.
    - ChemSpider 5y ago
      100 times this. Tesla Beta testers are people driving their own cars, and are - by selection of being able to afford a Tesla - in average more qualified then the minimum wage test driver. The word "trained test driver" is misleading. It sound like "test pilot", which are usually the best in their field. But paid test drivers are just people that applied for a low pay job. Heck, they maybe even be tired all the time because they have to have two jobs.
      - Xylakant 5y ago
        > by selection of being able to afford a Tesla - in average more qualified then the minimum wage test driver. How does “income received” translate into “qualified for driving”? A good lawyer, doctor, it consultant, … is probably on average a less safe driver than someone that has been through a training, despite earning more. If your worry is that the test drivers are tired because they need to work multiple jobs - mandate better pay. And from what I know from my doctor acquaintances, being tired from overwork is part of the job description.
yawaworht1978 5y ago
Tesla fsd has many parallels to Theranos Edison device. They don't share the data. They don't work as promised. They charge investors up front for what is essentially vaporware. They advertise the ware can or will be able to do much more than it currently does. They both put peoples health at risk. At least Tesla has a proper usable product, an electric car. Good on sf to raise concerns, too many other governments are just looking away.
- dataking 5y ago
  Instead of using the Edison device, as far as I understand, Theranos performed blood testing using traditional providers but hid this fact from investors. FSD, on the other hand, is something 3rd parties can evaluate on their own, the flaws are there for all to see. Hype, while not always okay, isn't fraud.
  - bumby 5y ago
    >Hype, while not always okay, isn't fraud. There is a legal concept of “theft by false promises” though. I don’t know where the line is drawn between that and “hype”.
    - clouddrover 5y ago
      In 2016 Tesla falsely claimed that all Tesla vehicles being produced had the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver: https://www.tesla.com/blog/all-tesla-cars-being-produced-now-have-full-self-driving-hardware https://www.tesla.com/blog/all-tesla-cars-being-produced-now... It doesn't seem to have hurt Tesla so far. People are still spending money on the undelivered promise of full self driving.
      - deleted 5y ago
        [deleted]
      - scoopertrooper 5y ago
        Not fraud if they give you a free upgrade to more capable hardware. https://www.tesla.com/support/full-self-driving-computer https://www.tesla.com/support/full-self-driving-computer
harshaw 5y ago
I am terrified at the concept of "self-driving" or anything marketed as "self-driving" even if it is really advanced driver assist. When I talk to my engineer friends they are also deeply skeptical. Maybe its our age (mid 40's) or that after dealing with software development for 20+ years you tend to be jaded. Maybe it's that I know that ML based systems have limits based on the training data and it's not real deep AI. Maybe its that all the training is done in CA and someone will try to use this software where I live in New England and do it when it is snowing at night on a drive to Vermont and the system will fall apart. I don't think we understand the human factors involved and the the hubris that most people will ignore their role in oversight as the software gets more advanced. If the software is 90% effective for most driving people will tune out. Holy crap that is scary.
- suifbwish 5y ago
  I know that there are probably versions of AI that are hidden from the public both private and military but something tells me if we were close to the advent of anything self operational it would be evident. There would be transitional technologies emerging on the markets. More than likely there are pieces of very complex intelligence resembling mechanisms that are held together by teams of people. The idea of even the most modern intelligence system navigating through an environment as elaborate and dynamic as New York City or downtown Chicago should give anyone pause. I can see them being more useful for long distance journeys such as hauling a trailer across rural desert areas where all the vehicle needs to do is stay in its own lane, maintain a set speed and brake if something obvious gets in front of it.
  - moffkalast 5y ago
    To be frank, the lane keeping ADAS has been out for years and most car manufacturers offer such a thing on their high end models. Usually radar supported and pretty reliable from what I hear. Perhaps it's not the best idea to use these more advanced Level 4 systems in the densest city yet, but most of driving does usually happen on highways and regional roads so it's not a major loss. I'm sure someone will eventually figure out how to detect "eyyy im walkin eer" and honk back automatically, and then it should be usable in NY as well. > versions of AI that are hidden from the public both private and military I do think you overestimate how much the military has its shit together. These systems need gigantic amounts of driving data to train and nobody else has that aside from car manufacturers with their fleets. And out of those Tesla is the one that's been doing it the longest.
ClumsyPilot 5y ago
I am appalled at the way we are attempting to introduce driverless cars. Cities install traffic lights with sensors and an internet connection so that it can be controlled from the city traffic management HQ. There are dozens of professionals in front of giant screens, and a supercomputer at their disposal. Could that information be sent directly to cars? No, instead we cram a computer and a camera into every car and hope it can find the right pixels on the picture in fog and rain. Smart approach would be to invest in infrastructure - standard location beacons that tell cars exactly where they are and what is ahead, wireless communication between cars so they know locations and bearing of every vehicle within a square mile, wireless tags could be installed on all bikes and cycles, etc. We have an entire department for traffic management in each city that could suggest a route to every car? We already installed speed cameras, number plate recognition, pedestrian counting, we have vehicle sensing sensors inside the road that count and weigh cars, and none of this information is being used. If cloud computing makes sence for crappy apps, then it definitely makes sence to assist cars.
- mmastrac 5y ago
  And if anything, governments are in a privileged position to create and enforce these standards across multiple large regions.
- CorrectHorseBat 5y ago
  What do you do when something falls on the road? Should everyone walk around with a wireless tag if they don't want to be run over by a car?
  - mepiethree 5y ago
    Obviously not. In such a case, a much better option would be to fall back to the camera system or, worst case, force the car to go into manual mode with a human driver if it detects that nearby systems are unresponsive
    - ctdonath 5y ago
      Then the fallback must be so good that you don’t need primary system in the first place.
fasteddie31003 5y ago
This is another chapter in the battle of ethical mindsets that I see playing over and over again in America, consequentialism vs deontology. From everything I've heard of Tesla self driving is that per-mile it is more safe than human control (consequentialism argument for self driving). However, there have been a few high profile accidents recently with emergency vehicles (deontological argument against self driving). You ultimately need to pick an ethical mindset to judge the viability of Tesla self driving. I think we have seen a trend in American culture recently to more of deontological ethics especially in places like San Francisco. I'm personally more towards consequentialism and therefore are pro Tesla self driving, if per-mile it is statistically safer than a human driver.
- api 5y ago
  People aren’t consistent though. The argument for vaccine mandates is 100% consequentialist and more or less identical to the argument for pushing forward with self drive: net fewer people will die. The antivaxxers are deontological, arguing that a few vaccine reactions and deaths should halt the whole thing. Most of the SF folks you are describing lean toward the pro-vaccine-mandate position. They should therefore also be for self driving cars.
- TigeriusKirk 5y ago
  There's also the factor that Musk is a high-profile 0.01%er who tweets a lot.
- ajross 5y ago
  > However, there have been a few high profile accidents recently with emergency vehicles Interestingly even that's a bit spun. There was one recent accident, in August. And the NTSB going back discovered that there's a cluster of (I think) 11 others where the car behaved similarly and appeared to strike an undetected emergency vehicle without trying to avoid. Now... that's interesting, and potentially represents a bug worth fixing. But it was also rapidly pointed out that (1) this is an extremely common kind of accident (human drivers hit these things all the time too) and (2) given the number of reported Tesla AP miles driven and assuming no other unreported collisions cases, Autopilot is actually about twice as safe as the average US driver vs. stopped emergency vehicles.
supperburg 5y ago
If you are worried about people’s safety then why don’t you stand in the back of a bar at closing and try to stop a few people in the mob of drunk people drunkenly stumbling into their cars? You would save more lives that way than protesting self driving. Do any of the Tesla detractors take any action about or even comment about the tens of thousands of deaths caused by people craning over their phones, doing their makeup or intoxication? No, because tesla alarmism is not a rational position and it is not based in objective assessment of public safety. I’m so tired of the media and the mob taking these select issues out of context. The media plasters FSD accidents across every screen in the world, even when later it turns out the driver was drunk, but it doesn’t mention the zillion miles of uneventful riding. You see clips of FSD doing something weird, “diving toward pedestrians,” shooting straight to the top of reddit/HN (is there any distinction nowadays?) but videos of FSD doing amazing things, negotiating the Berkeley hills without intervention, never reach the front page. If you showed a person from the 90s two hours of v10 FSD in various environments, mistakes and interventions and all, that person would say with astonishment “that car basically drives itself.” And anyone who denies that is being intellectually dishonest.
- IshKebab 5y ago
  > anyone who denies that is being intellectually dishonest. Nobody disputes that it can drive fine most of the time on easy roads.
  - supperburg 5y ago
    Berkeley hills are easy roads?
    - IshKebab 5y ago
      I haven't driven there but I had a look at a random spot (Grizzly Peak Blvd) and yeah they look very easy. I'll say "this car can drive itself" when it can drive over double roundabouts and down streets with on-street parking on both sides: https://i.dailymail.co.uk/i/pix/2016/01/15/16/3032888D00000578-3401389-image-a-28_1452875549965.jpg https://i.dailymail.co.uk/i/pix/2016/01/15/16/3032888D000005...
neonate 5y ago
https://archive.md/wawKb https://archive.md/wawKb
almost_usual 5y ago
Most Bay Area Tesla drivers aren’t great. I look forward to a computer driving instead of them one day.
echopurity 5y ago
People still believe in this unsustainable scam?
inglor_cz 5y ago
This is yet another instance of the "what is the acceptable risk for introducing a new technology" problem. New technologies tend to be rather unsafe when introduced: steam engines, cars, trains, airplanes, electricity, gas heating, all those things tended to kill the early adopters and some bystanders as well. Then they improve and the next generations get used to them and their inherent risks that diminish over time but rarely get down to pure 0. We are much more risk-aware than our ancestors, though, because we no longer live in a world of ubiquitous premature death. With an average life expectancy exceeding 80 in many countries, we tend to be much more careful than our ancestors in times when life expectancy was 47. (As in the USA in the year 1900.) I can understand that - I do not want to die under wheels of a robot gone wild any more than anyone else - but we should still find a balance, unless we want to stagnate indefinitely. The risks of self driving cars are nothing compared to the risks of biotech, which is just coming of age. But biotech holds a lot of promises, too. PS.: Frankly, my worst fear about self-driving cars is malware and ransomware, not honest mistakes in the code. Computer security is basically an oxymoron and some people are smart and evil at the same time.
- jdavis703 5y ago
  I have no problem driving or biking among the LIDAR-based systems from Waymo and others in the SF Bay Area. I’ve seen them do stupid stuff, but it was always on the side of safety. On the other hand I’ve seen hands-free Tesla drivers reading books, using their phone, etc, which is both stupid and dangerous.
  - theshrike79 5y ago
    Tesla definitely should have a camera watching the driver and detect if they're being an ass. if user.is_an_ass == true: disable_autopilot(days=7) That should teach them pretty quickly.
    - panick21_ 5y ago
      They literally do have that.
amelius 5y ago
I just hope for one thing: that they ban automatic updates.
- concordDance 5y ago
  So if a bug is found such as the fact a particular pattern that happens to be the logo of an up-and-coming boy band crashes the car's visual software we just have to tell everyone to stay home and not use their self driving cars until they've installed the fix and hope people actually receive the message before tragedy happens?
  - amelius 5y ago
    What would you suggest otherwise? Allow an update to be rushed? The cars should be grounded anyway (or self-driving should be disabled). This can be done remotely in a safe way. Any new update to the software should be tested by the DoT for at least X miles and Y hours before it can be installed. The installation of the update should be done by an agency of the DoT to prevent car manufacturers from cheating (and we know they will cheat when they can find a way, see VW scandal). Let's not allow the car industry to be run like the software industry (!!!)
deleted 5y ago
[deleted]
greggman3 5y ago
my feeling is I won't trust it until the company takes the liability. Tesla won't so they clearly do not believe it works. Waymo does so I'd consider riding a Wayno car. To me it's a useless feature for me to have to be ready to correct the car's mistakes. To actually do it is way way more stressful than just driving myself. And, if I ever get comfortable in what the car is doing then I'll stop giving it my full attention which is exactly what your told you're not allowed to do.
- LeoPanthera 5y ago
  > Tesla won't so they clearly do not believe it works. But Tesla offer their own insurance. Isn't that taking responsibility?
  - 0xdeadb00f 5y ago
    Is that a joke? You must be joking. Insurance? Responsibility??
carapace 5y ago
I've said this before, so apologies to those who've heard it already, but I believe testing big fast robots in public is irresponsible. These machines have already killed people. We know that human drivers kill and maim tens of thousands of people every year, so it's pretty obvious that robotic drivers will do so too, unless and until they become better drivers than humans. If somebody wanted to let their killer robots move around in public with limited supervision it would be obviously illegal, but somehow dressing up as cars falls into a weird blind spot. I've encountered at least two major objections: > Human drivers kill people so it's okay if robot drivers do too. But is it really okay for human drivers to kill people? Our complacency in the face of automotive mayhem is the result of a deliberate campaign of social engineering: "The Real Reason Jaywalking Is A Crime" (Adam Ruins Everything) https://www.youtube.com/watch?v=vxopfjXkArM https://www.youtube.com/watch?v=vxopfjXkArM We went from decrying "speed demons" (reckless drivers who hit pedestrians) to victim-blaming "jaywalkers". Now, for example, more Americans have been killed by cars than by all the wars we've fought. (And that's just counting collisions, not deaths from air pollution, etc.) > The machines will eventually be safer than human drivers (a belief I share) but they need to practice in real world conditions to do it. So build a model city, populate it with people willing to take the risks, and test them there. (This is setting aside the additional idea of doing extensive modeling in simulated cities.) It would be expensive but it would avoid killing people who haven't e.g. signed a release agreeing to be test dummies in the robot car experiments. The obvious way to make large, heavy, fast robot cars is to start with small, light, slow robot cars. Imagine a self-driving golf cart covered in nerf (soft foam) that doesn't go faster than about 5mph. There are applications for that and it would be much more difficult for such a robot to injure or kill people. Going straight for self-driving cars (and can we please call them "auto-autos"?) seems to me to be hubris.
- cto_of_antifa 5y ago
  This is a great and well reasoned post and in general, thank you. I like the test city idea; at first I thought it was still a little risky but then I realized that this is probably one of those situations where you can't actually test or prove its efficacy without a "live-fire" environment. In medicine you famously can't conduct an experiment that knowingly exposes participants to a greater risk of death than the alternative of not participating, and i wonder if that's a useful ethical framework for this
NoblePublius 5y ago
Why would anyone take seriously the opinion of SF government about “safety” when it’s official policy explicitly condones open IV drug use?
GhostVII 5y ago
The problem with level 2 "FSD" is that the better it gets, the more dangerous it is because the driver gets complacent. Autopilot is already being investigated because drivers don't monitor it enough and hit emergency vehicles, and all autopilot does is go in a straight line. FSD is making far more decisions, and gives the driver far less time to react to it behaving badly. Tesla FSD has several situations where the car regularly performs maneuvers that, if the driver didn't intervene, it would likely kill the driver and other people on the road. It also has many types of routes where it can operate without any disengagements for a long period of time. I think that is a very dangerous combination. If they really wanted to reduce deaths due to driving, they could use this FSD software to add better collision avoidance technology. Once you have that, introducing self-driving should be safer, even if it doesn't always make the right decisions it will be caught by the collision avoidance tech.
- amelius 5y ago
  Perhaps someone can build a Tesla-detector, so when it sees one the owner can brace for impact ... Sadly, only half-joking here.
  - judge2020 5y ago
    Valets driving manual seems to be a bigger risk, in my experience.
- LeoPanthera 5y ago
  > and all autopilot does is go in a straight line This isn't true, and there's any number of YouTube videos demonstrating that. At the very least, it knows how to navigate highway interchanges, how to change lanes, how to avoid other cars that would otherwise collide with it, and to avoid/slow down for pedestrians and cyclists. The last feature is perhaps overly sensitive in my personal experience. It used to slow down dramatically whenever it was passing a cyclist, that has improved in recent versions.
  - GhostVII 5y ago
    To clarify, I wasn't saying that it literally just goes in a straight line without any additional collision avoidance logic, just that from the drivers perspective all they have to do is make sure it continues in it's lane, as opposed to FSD where there is an order of magnitude more decisions the driver has to monitor. There is a huge difference between the focus required to make sure your car driving on the highway stays in it's lane and doesn't run into stopped vehicles, vs the focus required to make sure your car doesn't turn in front of other cars or take a turn to wide and hit something. I wasn't including Navigate on Autopilot as a part of that but even with Navigate on Autopilot, the driver has far less to monitor than with FSD, since it just changes lanes on the highway occasionally rather than having to turn across traffic, frequently switch lanes in the city, or manage stop signs and lights.
rsj_hn 5y ago
I'm pretty frustrated that Tesla has taken fantastic innovation in bringing electric cars to market and marred it with these stupid fixations on both self-driving tech and things like ridiculous steering wheels. Go ahead and add more tech to avoid collisions and improve safety. But quit it already with these features that seem like the manifestation of Elon Musk's personal hangups.
- canada_dry 5y ago
  > ridiculous steering wheels I really don't get how regulators are OK with the yoke steering in a car! To me it says that Tesla really doesn't prioritize safety.
cdepman 5y ago
How timely. I was almost hit by a Tesla while on a run yesterday in San Francisco. The driver was not paying attention at all, slowly rolling through a stop sign. While this is one data point and could happen with any type of automobile, this type of behavior in "autonomous" cars worries me a lot - the carelessness other posters have noted in which these drivers put down their guard and think the car is going to do everything for them. They put their full faith in this nascent technology and therefore put others' safety at risk. Sure, I am very excited for this technology to mature, but in the meantime, let's not fool ourselves.
deleted 5y ago
[deleted]
6gvONxR4sf7o 5y ago
How is it that we all rail against the naming of “unlimited data plans” and shit like that, and then let tesla get away calling their level 2 system “full self driving?” Or even worse, “FSD capable” without being capable of FSD today?

141 points by camjohnson26 5y ago

I’m confused by sf. They did this for Uber in 2016, Amazon’s zoox, and a host of other companies, etc. Tesla has a safety record that’s not unimpressive compared to the 36k people who die every year even though their failures have been more public and newsworthy. Their stats have been relatively objective. We also have a framework for this: https://www.nhtsa.gov/technology-innovation/automated-vehicles-safety https://www.nhtsa.gov/technology-innovation/automated-vehicl... As there is a framework in place already, shouldn’t we push forward? As long as we are reducing the statistical risk of driving below the risk of human drivers, shouldn’t that be the litmus for whether these programs are allowed to progress? I know I am being simplistic, and I understand that no statistical model will ever make up for the loss of a family member, the generally accepted foundation for the morality of self-driving car technologies hinges on whether they can beat the human-driven statistical equivalent. If there is concern about that they shouldn’t continue. But if there isn’t, these experiments should progress as they could ultimately reduce mortality. There were 3,723 deaths in California in 2020. The lifetime odds of dying in a motor-vehicle crash for a person born in 2019 were 1 in 107. These odds suck. Is that really acceptable? Can we do better? We know we can. If we can reduce that number we should. If we can’t, we should restrict this technology’s rollout until it can. Am I overly objective? I’m not an expert. I mean ultimately I want legislation built on objective data and loving values with human compassion at the core. I feel like sometimes we see these decisions based on how fast the system can adapt the ocean of existing automotive and roadway regulatory compliance to the technology changes, rather than new realities. People have no idea how much the books and legal cases that many cal trans engineers have spent their lives developing stand in the way of self-driving vehicles. I honestly think the biggest issue is probably the legal a s regulatory overhead of responding to the technology rather than the technology itself, but since it’s so hard to quantify I don’t often bring this up. But the biggest passion I have seen if from those same engineering minds, who truly understand the complexity, but also want to see it happen. Their timeframes are just long. I’ve heard 2045 and 2050. Anyone more expert than I care to weigh in? I’d love to learn more. I am not trying to weigh in with any merit — I am a nubile entrant and simply sharing the output of conversation with people who will always be more qualified and I, but I’d love to learn more about this.

lotophage 5y ago
> I’m confused by sf. They did this for Uber in 2016, Amazon’s zoox, and a host of other companies, etc. My understanding is that those companies had trained test drivers, but Telsa is pushing this out as a beta feature update and allowing the consumers be their test drivers?
- happytiger 5y ago
  That is a critical difference for sure.
- svet_0 5y ago
  > trained test drivers Are normal people paid minimum wage who become complacent and occasionally fall asleep (e.g. Uber accident), I would argue non-paid Tesla customers are at least as good as "trained drivers". There are 2K drivers using FSD beta for almost a year, w/o any accidents.
  - ChemSpider 5y ago
    100 times this. Tesla Beta testers are people driving their own cars, and are - by selection of being able to afford a Tesla - in average more qualified then the minimum wage test driver. The word "trained test driver" is misleading. It sound like "test pilot", which are usually the best in their field. But paid test drivers are just people that applied for a low pay job. Heck, they maybe even be tired all the time because they have to have two jobs.
    - Xylakant 5y ago
      > by selection of being able to afford a Tesla - in average more qualified then the minimum wage test driver. How does “income received” translate into “qualified for driving”? A good lawyer, doctor, it consultant, … is probably on average a less safe driver than someone that has been through a training, despite earning more. If your worry is that the test drivers are tired because they need to work multiple jobs - mandate better pay. And from what I know from my doctor acquaintances, being tired from overwork is part of the job description.

yawaworht1978 5y ago

Tesla fsd has many parallels to Theranos Edison device. They don't share the data. They don't work as promised. They charge investors up front for what is essentially vaporware. They advertise the ware can or will be able to do much more than it currently does. They both put peoples health at risk. At least Tesla has a proper usable product, an electric car. Good on sf to raise concerns, too many other governments are just looking away.

dataking 5y ago
Instead of using the Edison device, as far as I understand, Theranos performed blood testing using traditional providers but hid this fact from investors. FSD, on the other hand, is something 3rd parties can evaluate on their own, the flaws are there for all to see. Hype, while not always okay, isn't fraud.
- bumby 5y ago
  >Hype, while not always okay, isn't fraud. There is a legal concept of “theft by false promises” though. I don’t know where the line is drawn between that and “hype”.
  - clouddrover 5y ago
    In 2016 Tesla falsely claimed that all Tesla vehicles being produced had the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver: https://www.tesla.com/blog/all-tesla-cars-being-produced-now-have-full-self-driving-hardware https://www.tesla.com/blog/all-tesla-cars-being-produced-now... It doesn't seem to have hurt Tesla so far. People are still spending money on the undelivered promise of full self driving.
    - deleted 5y ago
      [deleted]
    - scoopertrooper 5y ago
      Not fraud if they give you a free upgrade to more capable hardware. https://www.tesla.com/support/full-self-driving-computer https://www.tesla.com/support/full-self-driving-computer

harshaw 5y ago

I am terrified at the concept of "self-driving" or anything marketed as "self-driving" even if it is really advanced driver assist. When I talk to my engineer friends they are also deeply skeptical. Maybe its our age (mid 40's) or that after dealing with software development for 20+ years you tend to be jaded. Maybe it's that I know that ML based systems have limits based on the training data and it's not real deep AI. Maybe its that all the training is done in CA and someone will try to use this software where I live in New England and do it when it is snowing at night on a drive to Vermont and the system will fall apart. I don't think we understand the human factors involved and the the hubris that most people will ignore their role in oversight as the software gets more advanced. If the software is 90% effective for most driving people will tune out. Holy crap that is scary.

suifbwish 5y ago
I know that there are probably versions of AI that are hidden from the public both private and military but something tells me if we were close to the advent of anything self operational it would be evident. There would be transitional technologies emerging on the markets. More than likely there are pieces of very complex intelligence resembling mechanisms that are held together by teams of people. The idea of even the most modern intelligence system navigating through an environment as elaborate and dynamic as New York City or downtown Chicago should give anyone pause. I can see them being more useful for long distance journeys such as hauling a trailer across rural desert areas where all the vehicle needs to do is stay in its own lane, maintain a set speed and brake if something obvious gets in front of it.
- moffkalast 5y ago
  To be frank, the lane keeping ADAS has been out for years and most car manufacturers offer such a thing on their high end models. Usually radar supported and pretty reliable from what I hear. Perhaps it's not the best idea to use these more advanced Level 4 systems in the densest city yet, but most of driving does usually happen on highways and regional roads so it's not a major loss. I'm sure someone will eventually figure out how to detect "eyyy im walkin eer" and honk back automatically, and then it should be usable in NY as well. > versions of AI that are hidden from the public both private and military I do think you overestimate how much the military has its shit together. These systems need gigantic amounts of driving data to train and nobody else has that aside from car manufacturers with their fleets. And out of those Tesla is the one that's been doing it the longest.

ClumsyPilot 5y ago

I am appalled at the way we are attempting to introduce driverless cars. Cities install traffic lights with sensors and an internet connection so that it can be controlled from the city traffic management HQ. There are dozens of professionals in front of giant screens, and a supercomputer at their disposal. Could that information be sent directly to cars? No, instead we cram a computer and a camera into every car and hope it can find the right pixels on the picture in fog and rain. Smart approach would be to invest in infrastructure - standard location beacons that tell cars exactly where they are and what is ahead, wireless communication between cars so they know locations and bearing of every vehicle within a square mile, wireless tags could be installed on all bikes and cycles, etc. We have an entire department for traffic management in each city that could suggest a route to every car? We already installed speed cameras, number plate recognition, pedestrian counting, we have vehicle sensing sensors inside the road that count and weigh cars, and none of this information is being used. If cloud computing makes sence for crappy apps, then it definitely makes sence to assist cars.

mmastrac 5y ago
And if anything, governments are in a privileged position to create and enforce these standards across multiple large regions.
CorrectHorseBat 5y ago
What do you do when something falls on the road? Should everyone walk around with a wireless tag if they don't want to be run over by a car?
- mepiethree 5y ago
  Obviously not. In such a case, a much better option would be to fall back to the camera system or, worst case, force the car to go into manual mode with a human driver if it detects that nearby systems are unresponsive
  - ctdonath 5y ago
    Then the fallback must be so good that you don’t need primary system in the first place.

fasteddie31003 5y ago

This is another chapter in the battle of ethical mindsets that I see playing over and over again in America, consequentialism vs deontology. From everything I've heard of Tesla self driving is that per-mile it is more safe than human control (consequentialism argument for self driving). However, there have been a few high profile accidents recently with emergency vehicles (deontological argument against self driving). You ultimately need to pick an ethical mindset to judge the viability of Tesla self driving. I think we have seen a trend in American culture recently to more of deontological ethics especially in places like San Francisco. I'm personally more towards consequentialism and therefore are pro Tesla self driving, if per-mile it is statistically safer than a human driver.

api 5y ago
People aren’t consistent though. The argument for vaccine mandates is 100% consequentialist and more or less identical to the argument for pushing forward with self drive: net fewer people will die. The antivaxxers are deontological, arguing that a few vaccine reactions and deaths should halt the whole thing. Most of the SF folks you are describing lean toward the pro-vaccine-mandate position. They should therefore also be for self driving cars.
TigeriusKirk 5y ago
There's also the factor that Musk is a high-profile 0.01%er who tweets a lot.
ajross 5y ago
> However, there have been a few high profile accidents recently with emergency vehicles Interestingly even that's a bit spun. There was one recent accident, in August. And the NTSB going back discovered that there's a cluster of (I think) 11 others where the car behaved similarly and appeared to strike an undetected emergency vehicle without trying to avoid. Now... that's interesting, and potentially represents a bug worth fixing. But it was also rapidly pointed out that (1) this is an extremely common kind of accident (human drivers hit these things all the time too) and (2) given the number of reported Tesla AP miles driven and assuming no other unreported collisions cases, Autopilot is actually about twice as safe as the average US driver vs. stopped emergency vehicles.

supperburg 5y ago

If you are worried about people’s safety then why don’t you stand in the back of a bar at closing and try to stop a few people in the mob of drunk people drunkenly stumbling into their cars? You would save more lives that way than protesting self driving. Do any of the Tesla detractors take any action about or even comment about the tens of thousands of deaths caused by people craning over their phones, doing their makeup or intoxication? No, because tesla alarmism is not a rational position and it is not based in objective assessment of public safety. I’m so tired of the media and the mob taking these select issues out of context. The media plasters FSD accidents across every screen in the world, even when later it turns out the driver was drunk, but it doesn’t mention the zillion miles of uneventful riding. You see clips of FSD doing something weird, “diving toward pedestrians,” shooting straight to the top of reddit/HN (is there any distinction nowadays?) but videos of FSD doing amazing things, negotiating the Berkeley hills without intervention, never reach the front page. If you showed a person from the 90s two hours of v10 FSD in various environments, mistakes and interventions and all, that person would say with astonishment “that car basically drives itself.” And anyone who denies that is being intellectually dishonest.

IshKebab 5y ago
> anyone who denies that is being intellectually dishonest. Nobody disputes that it can drive fine most of the time on easy roads.
- supperburg 5y ago
  Berkeley hills are easy roads?
  - IshKebab 5y ago
    I haven't driven there but I had a look at a random spot (Grizzly Peak Blvd) and yeah they look very easy. I'll say "this car can drive itself" when it can drive over double roundabouts and down streets with on-street parking on both sides: https://i.dailymail.co.uk/i/pix/2016/01/15/16/3032888D00000578-3401389-image-a-28_1452875549965.jpg https://i.dailymail.co.uk/i/pix/2016/01/15/16/3032888D000005...

neonate 5y ago

https://archive.md/wawKb https://archive.md/wawKb

almost_usual 5y ago

Most Bay Area Tesla drivers aren’t great. I look forward to a computer driving instead of them one day.

echopurity 5y ago

People still believe in this unsustainable scam?

inglor_cz 5y ago

This is yet another instance of the "what is the acceptable risk for introducing a new technology" problem. New technologies tend to be rather unsafe when introduced: steam engines, cars, trains, airplanes, electricity, gas heating, all those things tended to kill the early adopters and some bystanders as well. Then they improve and the next generations get used to them and their inherent risks that diminish over time but rarely get down to pure 0. We are much more risk-aware than our ancestors, though, because we no longer live in a world of ubiquitous premature death. With an average life expectancy exceeding 80 in many countries, we tend to be much more careful than our ancestors in times when life expectancy was 47. (As in the USA in the year 1900.) I can understand that - I do not want to die under wheels of a robot gone wild any more than anyone else - but we should still find a balance, unless we want to stagnate indefinitely. The risks of self driving cars are nothing compared to the risks of biotech, which is just coming of age. But biotech holds a lot of promises, too. PS.: Frankly, my worst fear about self-driving cars is malware and ransomware, not honest mistakes in the code. Computer security is basically an oxymoron and some people are smart and evil at the same time.

jdavis703 5y ago
I have no problem driving or biking among the LIDAR-based systems from Waymo and others in the SF Bay Area. I’ve seen them do stupid stuff, but it was always on the side of safety. On the other hand I’ve seen hands-free Tesla drivers reading books, using their phone, etc, which is both stupid and dangerous.
- theshrike79 5y ago
  Tesla definitely should have a camera watching the driver and detect if they're being an ass. if user.is_an_ass == true: disable_autopilot(days=7) That should teach them pretty quickly.
  - panick21_ 5y ago
    They literally do have that.

amelius 5y ago

I just hope for one thing: that they ban automatic updates.

concordDance 5y ago
So if a bug is found such as the fact a particular pattern that happens to be the logo of an up-and-coming boy band crashes the car's visual software we just have to tell everyone to stay home and not use their self driving cars until they've installed the fix and hope people actually receive the message before tragedy happens?
- amelius 5y ago
  What would you suggest otherwise? Allow an update to be rushed? The cars should be grounded anyway (or self-driving should be disabled). This can be done remotely in a safe way. Any new update to the software should be tested by the DoT for at least X miles and Y hours before it can be installed. The installation of the update should be done by an agency of the DoT to prevent car manufacturers from cheating (and we know they will cheat when they can find a way, see VW scandal). Let's not allow the car industry to be run like the software industry (!!!)

deleted 5y ago

[deleted]

greggman3 5y ago

my feeling is I won't trust it until the company takes the liability. Tesla won't so they clearly do not believe it works. Waymo does so I'd consider riding a Wayno car. To me it's a useless feature for me to have to be ready to correct the car's mistakes. To actually do it is way way more stressful than just driving myself. And, if I ever get comfortable in what the car is doing then I'll stop giving it my full attention which is exactly what your told you're not allowed to do.

LeoPanthera 5y ago
> Tesla won't so they clearly do not believe it works. But Tesla offer their own insurance. Isn't that taking responsibility?
- 0xdeadb00f 5y ago
  Is that a joke? You must be joking. Insurance? Responsibility??

carapace 5y ago

I've said this before, so apologies to those who've heard it already, but I believe testing big fast robots in public is irresponsible. These machines have already killed people. We know that human drivers kill and maim tens of thousands of people every year, so it's pretty obvious that robotic drivers will do so too, unless and until they become better drivers than humans. If somebody wanted to let their killer robots move around in public with limited supervision it would be obviously illegal, but somehow dressing up as cars falls into a weird blind spot. I've encountered at least two major objections: > Human drivers kill people so it's okay if robot drivers do too. But is it really okay for human drivers to kill people? Our complacency in the face of automotive mayhem is the result of a deliberate campaign of social engineering: "The Real Reason Jaywalking Is A Crime" (Adam Ruins Everything) https://www.youtube.com/watch?v=vxopfjXkArM https://www.youtube.com/watch?v=vxopfjXkArM We went from decrying "speed demons" (reckless drivers who hit pedestrians) to victim-blaming "jaywalkers". Now, for example, more Americans have been killed by cars than by all the wars we've fought. (And that's just counting collisions, not deaths from air pollution, etc.) > The machines will eventually be safer than human drivers (a belief I share) but they need to practice in real world conditions to do it. So build a model city, populate it with people willing to take the risks, and test them there. (This is setting aside the additional idea of doing extensive modeling in simulated cities.) It would be expensive but it would avoid killing people who haven't e.g. signed a release agreeing to be test dummies in the robot car experiments. The obvious way to make large, heavy, fast robot cars is to start with small, light, slow robot cars. Imagine a self-driving golf cart covered in nerf (soft foam) that doesn't go faster than about 5mph. There are applications for that and it would be much more difficult for such a robot to injure or kill people. Going straight for self-driving cars (and can we please call them "auto-autos"?) seems to me to be hubris.

cto_of_antifa 5y ago
This is a great and well reasoned post and in general, thank you. I like the test city idea; at first I thought it was still a little risky but then I realized that this is probably one of those situations where you can't actually test or prove its efficacy without a "live-fire" environment. In medicine you famously can't conduct an experiment that knowingly exposes participants to a greater risk of death than the alternative of not participating, and i wonder if that's a useful ethical framework for this

NoblePublius 5y ago

Why would anyone take seriously the opinion of SF government about “safety” when it’s official policy explicitly condones open IV drug use?

GhostVII 5y ago

The problem with level 2 "FSD" is that the better it gets, the more dangerous it is because the driver gets complacent. Autopilot is already being investigated because drivers don't monitor it enough and hit emergency vehicles, and all autopilot does is go in a straight line. FSD is making far more decisions, and gives the driver far less time to react to it behaving badly. Tesla FSD has several situations where the car regularly performs maneuvers that, if the driver didn't intervene, it would likely kill the driver and other people on the road. It also has many types of routes where it can operate without any disengagements for a long period of time. I think that is a very dangerous combination. If they really wanted to reduce deaths due to driving, they could use this FSD software to add better collision avoidance technology. Once you have that, introducing self-driving should be safer, even if it doesn't always make the right decisions it will be caught by the collision avoidance tech.

amelius 5y ago
Perhaps someone can build a Tesla-detector, so when it sees one the owner can brace for impact ... Sadly, only half-joking here.
- judge2020 5y ago
  Valets driving manual seems to be a bigger risk, in my experience.
LeoPanthera 5y ago
> and all autopilot does is go in a straight line This isn't true, and there's any number of YouTube videos demonstrating that. At the very least, it knows how to navigate highway interchanges, how to change lanes, how to avoid other cars that would otherwise collide with it, and to avoid/slow down for pedestrians and cyclists. The last feature is perhaps overly sensitive in my personal experience. It used to slow down dramatically whenever it was passing a cyclist, that has improved in recent versions.
- GhostVII 5y ago
  To clarify, I wasn't saying that it literally just goes in a straight line without any additional collision avoidance logic, just that from the drivers perspective all they have to do is make sure it continues in it's lane, as opposed to FSD where there is an order of magnitude more decisions the driver has to monitor. There is a huge difference between the focus required to make sure your car driving on the highway stays in it's lane and doesn't run into stopped vehicles, vs the focus required to make sure your car doesn't turn in front of other cars or take a turn to wide and hit something. I wasn't including Navigate on Autopilot as a part of that but even with Navigate on Autopilot, the driver has far less to monitor than with FSD, since it just changes lanes on the highway occasionally rather than having to turn across traffic, frequently switch lanes in the city, or manage stop signs and lights.

rsj_hn 5y ago

I'm pretty frustrated that Tesla has taken fantastic innovation in bringing electric cars to market and marred it with these stupid fixations on both self-driving tech and things like ridiculous steering wheels. Go ahead and add more tech to avoid collisions and improve safety. But quit it already with these features that seem like the manifestation of Elon Musk's personal hangups.

canada_dry 5y ago
> ridiculous steering wheels I really don't get how regulators are OK with the yoke steering in a car! To me it says that Tesla really doesn't prioritize safety.

cdepman 5y ago

How timely. I was almost hit by a Tesla while on a run yesterday in San Francisco. The driver was not paying attention at all, slowly rolling through a stop sign. While this is one data point and could happen with any type of automobile, this type of behavior in "autonomous" cars worries me a lot - the carelessness other posters have noted in which these drivers put down their guard and think the car is going to do everything for them. They put their full faith in this nascent technology and therefore put others' safety at risk. Sure, I am very excited for this technology to mature, but in the meantime, let's not fool ourselves.

deleted 5y ago

[deleted]

6gvONxR4sf7o 5y ago

How is it that we all rail against the naming of “unlimited data plans” and shit like that, and then let tesla get away calling their level 2 system “full self driving?” Or even worse, “FSD capable” without being capable of FSD today?