13 ms·

It should by now be pretty clear that Tesla, like Uber before it, are taking the Lord Farquaad approach to self-driving: "Some of you may die, but it's a sacrif

It should by now be pretty clear that Tesla, like Uber before it, are taking the Lord Farquaad approach to self-driving: "Some of you may die, but it's a sacrifice I am willing to make." It does not have to be this way, and these companies and their leadership deserve every bit of reputational damage they will get out of this.

sho_hn 5y ago
It's quite literally a public beta, just that the other traffic participants didn't get a say in the opt-in. Given the stakes for them I agree that it begs the question. In the end it's a regulation problem, not a Tesla problem (even though yes, the specific decision to exercise their options is theirs) - do you allow unlicensed drivers to perform this on-road testing of unfinshed vehicles or not? It's not quite a deliberate information hiding campaign as with MCAS (as far as we know). But it is pre-production testing of experimental engineering on safety-critical equipment. Experiments not guaranteed to pan out, and engineering lacking a mature certification process.
- Barrin92 5y ago
  the fact that there is such a thing as a 'public beta' for moving two tons of steel through pedestrian areas at lethal speeds seriously makes me question our collective sanity. Imagine if gun vendors did public betas where random police officers just get a beta testing gun. "It might fire in random directions at arbitrary intervals 5% of the time, but that's just how innovation goes folks. You don't like that, do you hate progress?"
  - throwaway0a5e 5y ago
    >Imagine if gun vendors did public betas where random police officers just get a beta testing gun. This is a thing. They give marketing samples to organizations they want to sell to on a "try 'em out and let us know if you want to buy more" basis.
    - b112 5y ago
      Marketing samples are not public betas. They are marketing samples of tested product.
      - throwaway0a5e 5y ago
        If your complaint is that a very conservative manufacturing industry doesn't have 1:1 analogues with software development then I guess it's valid. Giving organizations stuff you haven't 100% nailed down yet expecting them to come back with stuff they want changed in order to buy is pretty par for the course in the firearms industry. That's as close to a public beta as you're gonna get.
        everforward 5y ago
        The big difference is that marketing firearms are still safe, or as safe as firearms tend to get. It's a fully functional firearm; they want feedback on whether the ergonomics are good, whether the weight is good, how it handles recoil, etc. It would be more akin to handing out prototype weapons to police departments, which doesn't happen specifically because they might fire themselves or blow up or fail in some annoying but not spectacular way like failing to feed new rounds.
        throwaway0a5e 5y ago
        The key mechanical features always "work" but little stuff like "gun fails to cycle 500% more often with the particular spec of ammo a particular force uses". A rifle spec'd out for sale to Cambodia probably is going to have a lot of teething issues if you try and sell it to the Swedes unchanged. Are these things "safe"? IDK. But no force is going to put in an order for tens of thousands of guns unless correctable issues like those are proven fixed. Infantry arms are a battle of margins. Everything "works" but a marginally more finicky gun to use is going to mean the people using it are marginally less effective which extrapolates pretty directly to "safety". At the end of the day the analogy isn't great and we're building a castle in a swamp here.
  - dkjaudyeqooe 5y ago
    That reminds me of the ED-209 prototype in Robocop: "You have 20 seconds to comply!"
    - pmoriarty 5y ago
      Here's the "20 seconds to comply" scene from Robocop: https://www.youtube.com/watch?v=Hzlt7IbTp6M https://www.youtube.com/watch?v=Hzlt7IbTp6M
  - renewiltord 5y ago
    This is a fair point. It's pretty bad that we have learner drivers on the road. Can't think of anything more public beta-y than that. Maybe trainee doctors?
    - gvb 5y ago
      I've had three "learner drivers" in my family and I spent quite a few hours as the "safety person" during their apprenticeship. During all those hours I never touched the wheel, brake, or gas. I gave advice occasionally, but never felt that the "learner driver" was driving unsafely. I assure you that 15 year old young adults drive much better than a Tesla "self driving" computer with millions of miles and millions of hours of training.
      - renewiltord 5y ago
        Oh I’m sure you were convinced but you never asked me if I consented to you randomly putting underaged kids behind the wheel of a two ton vehicle on a road which I share with you. You’re a regular Yaroslav Kudrinsky, just with better luck. But I suppose one can go around swinging swords in a crowd, hit no one, and convince oneself of one’s good judgment.
    - rhizome 5y ago
      Category error: people have brains.
      - HarryHirsch 5y ago
        That's the important one - a Tesla has no concept of the real world. Remember that clip where a Tesla was slowing down because it mistook the full moon in the sky for a traffic light? Likewise, no one would aim at a cyclist, except when they are a homicidal criminal.
        noizejoy 5y ago
        … or under the influence of some substance, or a health event, or voluntary or involuntary distraction, or …
  - FireBeyond 5y ago
    Someone said in another thread earlier today that there is a worry that we are making the path towards FSD too erratic, and that if we screw it up, the next shot might be in generations. People are arguing, with a straight face, that we are ruining (aspects of) humanity by not letting Tesla be laisez-faire with such things (and lets be real, it's predominantly Tesla who is just in full Cowboy/YOLO mode... other players in this field certainly aren't perfect, but they are at least trying to be conservative and cautious).
    - jimmydorry 5y ago
      It's still a regulatory failure in the end. If the government does not want to lay down exactly what path should be taken in getting to FSD, then we should not complain when we don't agree with the path that the manufacturers choose. As long as Tesla can demonstrate a similar or better rate of serious harm than human drivers, why should we stop them? It's up to government to do their job and outline exactly what price society should bear in the pursuit of this paradigm shift (because let's not kid ourselves, FSD is a complete shift in paradigm for society).
      - FireBeyond 5y ago
        > As long as Tesla can demonstrate a similar or better rate of serious harm than human drivers, why should we stop them? Because it can't, or won't. Instead they release statistics that _imply_ that it is better, that they have to know are absolutely misleading, because anyone who did high school statistics would be aware thereof. Everything else, they refuse to release.
        jimmydorry 5y ago
        I'm not aware of _anyone_ releasing such data in a way. Not that I have been looking that closely for it (so I certainly could have missed this!). This is another major regulatory failure. Government should lay out the minimum data that the companies must publish, so we can get a better idea of their performance. (It's still a chicken and egg scenario though. You can't get that kind of data anywhere but on public roads in uncontrolled environments.)
- Retric 5y ago
  I was expecting autopilot to be seriously dangerous, but the data doesn’t back that up. Out of 234 deaths from accidents involving Tesla’s 12 people have died while Tesla autopilot was known to be in use or had recently been in use. http://www.tesladeaths.com http://www.tesladeaths.com Autopilot might be slightly worse than human drivers depending on how you slice the data but that’s about it. I don’t think it’s a regulation problem as much as it is an acceptance by regulators that driving is inherently dangerous and autopilot isn’t dramatically worse. It’s not even obvious if on net more people would have died if Tesla had never released autopilot.
  - andbberger 5y ago
    the data is extremely questionable
    - Retric 5y ago
      In what way?
      - andbberger 5y ago
        Sampling bias, confirmation bias. Did you delete your link to tesladeaths.com because you realized the fine print about sketchy autopilot data contradicted your point? Oops!
        Retric 5y ago
        No, I was trying to keep the link without a http:// http://, https://www.tesladeaths.com/ https://www.tesladeaths.com/ or https://www.tesladeaths.com/ https://www.tesladeaths.com/ works www.tesladeaths.com doesn’t nore does www.tesladeaths.com/ I am sure I have seen comment links without the http:// http:// but might have just gotten so used to it than I stopped seeing it.
        andbberger 5y ago
        uh huh. why do you think it's reasonable to conclude that this system is safe given that the reported data is based on faulty analysis [0] and inexplicably exempt from reporting requirements for testing self-driving cars? [0] https://jalopnik.com/feds-tesla-autosteer-safety-investigation-was-bullshit-1832542003 https://jalopnik.com/feds-tesla-autosteer-safety-investigati...
- paulcole 5y ago
  > In the end it's a regulation problem, not a Tesla problem Are you joking? If you’re the one causing the problem, you’re the problem. No regulation can prevent me from kicking someone in the shins. This isn’t a regulation problem, it’s a me problem.
  - nrb 5y ago
    The <Insert State Here> Code certainly applies penalties for assault and battery, and you can rest assured knowing shin-kicking would happen regularly without it. The car problem presented here is quite a bit more shades of gray and we absolutely need to set acceptable parameters for the technology.
    - paulcole 5y ago
      Yes, but my point is that if you’re being a jerk, then you’re the problem, not the person who is not saying, “please don’t be a jerk.” You don’t need a regulation telling you not to be a jerk — unless of course you are a jerk. And even then, you’d probably still be a jerk, regulation or not.
  - XorNot 5y ago
    Except it is: literally none of this is illegal. And it's not clear how it could be illegal: what's the distinction between cruise control and lane keeping and these systems? Very little - nothing you can codify. It's also very likely that this system can pass a standard driving test: AIs are really good at handling fixed test cases. So any regulation you do pass has a complicated assurance problem: how does it prove the system is safe? Pretend Tesla just knocked "beta" off the name and said "this is a limited self driving system, Tesla exclusive" - then you wind up in the same bucket with the same "it's the drivers responsibility problem". People see this footage and assume Tesla self drive does this all the time - but it wouldn't. It probably passes this scenario in testing 100% of the time: but the issue of course is the test is not quite whatever this circumstance is, and the system has a dramatic failure. There is absolutely a regulation problem.
    - 5y ago
- salawat 5y ago
  ...I am aware of zero other manufacturers brazen enough in terms of blatant disregard for Engineering Ethics to have embarked on this type of testing before Silicon Valley did. In fact, there were licensure processes for this sort of thing already in place in California iirc, so this is less gray area, more "lets outsource testing to the end user". Frankly, it's downright disturbing.
- cwkoss 5y ago
  What does MCAS mean here?
  - ofcrpls 5y ago
    https://en.wikipedia.org/wiki/Maneuvering_Characteristics_Augmentation_System https://en.wikipedia.org/wiki/Maneuvering_Characteristics_Au...
  - sho_hn 5y ago
    It's a software system overriding pilot input on Boeing 737 MAX airplanes, designed to make up for cost reductions in adequate hardware design. Pilots were deliberately not well-informed and the system caused two crashes. It was brought up elsewhere in the discussion. In general cars are headed into an aviation-like situation with automation, so case comparisons are becoming more frequent. It's very apples to oranges here, but loosely relevant wrt/ information/documentation and in the cameras-vs-LIDAR/sensor fusion debate.
    - rhizome 5y ago
      I'm sure the policy and regulatory systems will be able to fend off the lobbyists of the richest person in the world working to get the word "liability" redefined in the law. \s
- wavefunction 5y ago
  lol @ public beta
- oogabooga13 5y ago
  I've wondered if it would be a good idea to have some sort of external indicator on cars that indicates to others that are driving, cycling, on foot that the vehicle is in autonomous mode. At least then maybe one can understand the potential risk driving around them. (Noted that an indicator doesn't really help if the vehicle is behind you in a lot of instances!)
  - heavyset_go 5y ago
    One thing I've noticed the past couple of years is that it isn't that hard to determine whether or not a car is autonomously driving. If they're driving like they're drunk and they're in a Tesla, 9 out of 10 times, you'll see that that the driver isn't paying attention to the road at all because they're using Autopilot or FSD. I just treat the Tesla drivers around me on the road as if they're drunk and keep my distance.
    - sho_hn 5y ago
      Elsewhere in the thread a FSD Beta participant is imploring other drivers to keep their distance to Teslas because the software "gets easily spooked". This makes it even harder to assess the safety of the system - other drivers are helping it along as well.
  - sho_hn 5y ago
    Several manufacturers have experimented with such systems in concept cars, e.g. animated grilles that signal in green when an oncoming car has recognized a pedestrian and is slowing in response.
- bcrl 5y ago
  A public "beta" of a self driving car should lean towards failing safe. I would expect it to have sudden braking issues and be overly conservative. The kind of bugs we're seeing where obstacles are blatantly ignored make it feel more like an "alpha" release. But hey, we're not designing airplanes here, just thousand pound chunks of metal flying around roads capable of killing people.
- jrib 5y ago
  MCAS: https://en.wikipedia.org/wiki/Maneuvering_Characteristics_Augmentation_System https://en.wikipedia.org/wiki/Maneuvering_Characteristics_Au...
lvs 5y ago
This is normally when regulators step in.
ModernMech 5y ago
The thing that really grinds my gears about all of this as a roboticist, is that as a community we had a pretty good culture of taking safety seriously. In 2014, the idea of unleashing onto the public admittedly beta-quality autonomous car software, on a platform that eschews industry best practices was unfathomable. Then came Autopilot and later FSD, and Tesla threw the concept of safety was thrown out the window. An autonomous Tesla decapitated a man in 2016, and Tesla continued unabated! What actionable steps did Tesla take as a result of that incident to make sure it never happens again. Even today in 2022 you can find videos of Teslas on public roads just completely failing to detect and colliding with stationary objects. The culture at Tesla seems to not care this is happening, as it continues to happen. It used to be that when your robot did a single thing wrong or caused the most minor of injuries, let alone it killed someone, then you engineered the hell out of it and implemented safety protocols to make sure that failure mode would never happen again. Not at Tesla, apparently. I get the utilitarian argument that maybe self driving cars could reduce traffic accidents in the future by being better than humans on average. But safety doesn’t happen unless you really work at it, with intention. Using your customers to beta test your 2 ton robots on public roads is not a safety-focused mindset. Tesla is not a company that values safety first when it comes to their autonomous car project.
- belter 5y ago
  Why all the euphemisms and diplomatic language? There is one word for it: Criminal.
- kodah 5y ago
  I haven't worked in robotics, but I've worked in automation for quite some time (digital and real-world). Two examples I can think of: - SCADA always had hot, hardware backups and failure detection. To the extent that I thought this was dogma. - Physical automation (like car construction equipment and robotics, pick and place machines, etc) have safety zones that humans aren't allowed in or near. Same thing for Amazon warehouse robotics. I'm not sure how we leapt to, "beta test cars on the street" from there.
  - TOMDM 5y ago
belter 5y ago
https://videos.dailymail.co.uk/video/mol/2022/02/08/2352711287929719429/640x360_MP4_2352711287929719429.mp4 https://videos.dailymail.co.uk/video/mol/2022/02/08/23527112...
- ape4 5y ago
  Going out of its way to hit the bollard - wow
  - belter 5y ago
    Did you notice the most likely reason? Slightly to the right, of the line of bollards, there is a small patch of road and another strong white line. Kind of almost 1 meter, ( 3 feet) wide. So it thinks the tiny patch between the bollard, and the right most white line, is the right driving side of the road.
colinmhayes 5y ago
40,000 people die in car crashes every year in the US alone. If the lord farquad approach brings widespread autonomous car adoption which decreases that number substantially even a month sooner it's the right thing to do. Many of us are already dying.
- 46756e 5y ago
  Many people die of HIV, it’s a plague on society. If we just killed everyone who has it, it would be gone. Many of “us” are already dying, and it saves lives in the future. /s
- ModernMech 5y ago
  Sorry, but I don't see how a company that... - Beta tests critical software in the public which has lead to actual customer deaths; - Refuses to follow industry best practices when it comes to safety sensors; - Refuses to seriously investigate safety incidents and implement technological and cultural fixes to prevent them from happening in the future; ...will lead us into a utopia of automotive safety. At best what they're liable to do is create a climate where mega corporations feel free to use public spaces as their own personal laboratories, because the ends justify the means after all (don't pay attention to the decapitations, we're doing it for the safety of us all). We will all pay the price for that.
  - colinmhayes 5y ago
    I agree it would require an incredible amount of evidence to convince me that this actually would speed up the development of self driving. Just saying that I'm not opposed to the lord farquad method if it helps.
  - randoglando 5y ago
    Don't forget to add - willfully misrepresents data to imply a better safety record than they actually do have https://twitter.com/Tweetermeyer/status/1488673180403191808 https://twitter.com/Tweetermeyer/status/1488673180403191808 (I've posted this elsewhere on this thread too)
- 5y ago
lhorie 5y ago
For Uber at least, self driving was effectively a money-losing pipe dream, so it kinda did itself a favor by divesting via the sale of the division to Aurora. For Tesla, though, FSD is still such a front and center thing that I'm having trouble seeing how it ever plans on saving face as it gets more and more bad press from incidents like this.
- rhizome 5y ago
  It's not money-losing if you take the program's effect on stock prices into account, and maybe you'll even accept that it's always been a stock manipulation tool.
- twblalock 5y ago
  The feature sells cars. Remember that guy in southern California who got arrested for sleeping in the back of his Tesla while it drove him home, and he said he did it many times and would do it again? The system works well enough that he did it many times before he got caught. That guy is an idiot, but think about that story from another angle: You can fall asleep while driving any car. The difference is, if that car is a Tesla, you might not die. As a driver aid for non-crazy people FSD is a good system and Tesla owners like it. It's far from being vaporware.
  - andbberger 5y ago
    dang is going to ban me if I share how this take makes me feel. so let me just say that the proliferation of evidence-resistant memes such as this one concerns me greatly, and I feel even more unsafe as a pedestrian and cyclist on the street than I did before knowing that there are people who think that this is reasonable.
  - jquery 5y ago
    > he said he did it many times and would do it again? Based on the videos I’m seeing of FSD in action, I have my doubts. Wouldn’t be surprised if this was a publicity stunt. Elon Musk has a very loose relationship with ethics and the truth. They’re just obstacles for him.
  - 5y ago
throwawaylinux 5y ago
I also think there is a certain fear or outrage around this stuff which is fueled by "technology" or the unknown. People are some of the worst drivers around, they cause all kinds of problems with cluelessness and inattention and they crash and die and kill and injure in all sorts of stupid ways. They get drunk, high, check their phone, fall asleep, speed, show off I think it's great that there are still companies out there throwing research into moonshot types of research projects like this. Automobile deaths are perhaps the biggest and most persistent pandemic we've faced, we've been facing for a hundred years, and it kills and maims young and old alike. Millions of people dead over the years. Many were willing to relax some of the protocols around vaccine testing to bring the covid vaccines in to use earlier, put mandates and coercion around them, and yes some suffered adverse reactions and even died from them. Still, it was thought to be a reasonable compromise by many. I hope we don't go the other way on other technology that can advance society and potentially save lives just because not everything goes perfectly from the start, or we get stuck in the pharmaceutical corporations = bad mindset. Not to say there perhaps shouldn't be more regulation around it, but a serious and careful study of risks and benefits I think is appropriate rather than handwringing and outrage.
- couchand 5y ago
  We don't need moonshots to solve the car pandemic. Strong Towns and others have shown that we already have plenty of common-sense, effective tools that we've simply chosen not to roll out on a wider basis. We don't need new technology, we need new leaders.
  - throwawaylinux 5y ago
    Of course we do. If you can get "Strong Towns and others", whatever exactly that is, to end the car pandemic, that would be a a moonshot wouldn't it? Maybe not technologically but politically and socially. You're not the first one to have thought it's a good idea or that we should try to do something about it. It's not like it's easy and people have just decided not to.
- doodpants 5y ago
  > People are some of the worst drivers around Compared to which other species?
twblalock 5y ago
Self-driving cars will never be a reality if the standard for success is zero deaths during the early stages of development. It just won't happen. The technology won't ever be good enough if it's not tested in the real world. Furthermore, no self-driving system will be perfect even in a mature stage of development. People will still die, and they may die because of mistakes a human driver would not have made. If society isn't willing to accept that sacrifice, then let's just ban self-driving cars. But let's not try to have it both ways -- let's not allow self-driving while also holding the view that risk of death is unacceptable.
- admax88qqq 5y ago
  We don't have to ban self driving cars. We just set the success criteria at zero deaths. If companies want to spend money pursuing it believing they can hit that goal that's on them.
  - twblalock 5y ago
    > We don't have to ban self driving cars. We just set the success criteria at zero deaths. That's equivalent to a ban.
    - rhizome 5y ago
      Because the technology is a non-starter and will never work unless accepted terms of public safety are redefined in its favor. "Just draw a bigger bullseye!"
    - Karrot_Kream 5y ago
      There's multiple alternatives to driver-driven cars that result in many fewer excess deaths. They've been trialed and deployed in many countries very successfully. You may know of them by common names such as "buses", "streetcars", "trains", "metros", or even "people movers". There's nothing wrong with innovation but we do not need to put self-driving technology on a pedestal when there are alternatives that exist today that move people around that have none of these externalities. If entities want to innovate, let them innovate within safety parameters.
davej 5y ago
> "Some of you may die, but it's a sacrifice I am willing to make." Honest question: If beta testing speeds up the transition to FSD by x months/years and causes y deaths during the intervening period but FSD ultimately reduces the number of road deaths by z per day. For what values of x, y and z is it an acceptable trade off? Zeroism is often overly simplistic. y being non-zero might be hugely net-positive in the long run.
- tptacek 5y ago
  No, of course it isn't; think, for instance, of applying that same logic to other circumstances. You can't knowingly or recklessly endanger people simply because your long-term intentions are benign.
  - davej 5y ago
    There needs to be some testing of FSD solutions in real world scenarios and there will inevitably be deaths. There are thousands of deaths on the road every day right now. Even if FSD is mature and 1000x safer than human driving then there will still be deaths. Are you suggesting that if FSD is 1000x safer then it still shouldn't be allowed on the road because there is the possibility of deaths? I'm not claiming that FSD should or shouldn't be allowed on the road right now. My point is that we shouldn't wait for the chances of death to be zero.
    - heavyset_go 5y ago
      > There needs to be some testing of FSD solutions in real world scenarios and there will inevitably be deaths. There are thousands of deaths on the road every day right now. This is a solved problem with experimental design. Experiments are designed and reviewed by ethics boards, and participants are gathered after getting their informed consent and giving them just compensation for the risks involved. "Testing of FSD solutions in the real world" can follow the same path in their experimental designs, but they don't, because executives and shareholders don't want to pay for expensive ethical experimentation. They'd rather move fast and break bodies. Most modern life saving devices and treatments have gone through exactly this type of ethical experimentation. There's no reason Tesla's products couldn't, as well.
AtlasBarfed 5y ago
Not that it justified overhyping or undercaution by Tesla, but lets be real: everyday driving comes down to that same acceptance of risk Let's not be perfect be the enemy of the good: a self-driving highway system can probably be achieved with current technology that beats human drivers for long distances. I wish that was the priority rather than the robotaxi obsession. Self-driving highway systems would produce immediate profit and economic savings that could fund the drive-grandma-to-Taco-Bell systems, as well as the convergent infrastructure, common signaling systems, embedded sensors, better signage and standards, improved and predictable construction signals and routing, etc. Following the usual hype cycle, it will be highway logistics that smooths over the "despair" portion and does the practical incremental improvement.

sho_hn 5y ago

It's quite literally a public beta, just that the other traffic participants didn't get a say in the opt-in. Given the stakes for them I agree that it begs the question. In the end it's a regulation problem, not a Tesla problem (even though yes, the specific decision to exercise their options is theirs) - do you allow unlicensed drivers to perform this on-road testing of unfinshed vehicles or not? It's not quite a deliberate information hiding campaign as with MCAS (as far as we know). But it is pre-production testing of experimental engineering on safety-critical equipment. Experiments not guaranteed to pan out, and engineering lacking a mature certification process.

Barrin92 5y ago
the fact that there is such a thing as a 'public beta' for moving two tons of steel through pedestrian areas at lethal speeds seriously makes me question our collective sanity. Imagine if gun vendors did public betas where random police officers just get a beta testing gun. "It might fire in random directions at arbitrary intervals 5% of the time, but that's just how innovation goes folks. You don't like that, do you hate progress?"
- throwaway0a5e 5y ago
  >Imagine if gun vendors did public betas where random police officers just get a beta testing gun. This is a thing. They give marketing samples to organizations they want to sell to on a "try 'em out and let us know if you want to buy more" basis.
  - b112 5y ago
    Marketing samples are not public betas. They are marketing samples of tested product.
    - throwaway0a5e 5y ago
      If your complaint is that a very conservative manufacturing industry doesn't have 1:1 analogues with software development then I guess it's valid. Giving organizations stuff you haven't 100% nailed down yet expecting them to come back with stuff they want changed in order to buy is pretty par for the course in the firearms industry. That's as close to a public beta as you're gonna get.
      - everforward 5y ago
        The big difference is that marketing firearms are still safe, or as safe as firearms tend to get. It's a fully functional firearm; they want feedback on whether the ergonomics are good, whether the weight is good, how it handles recoil, etc. It would be more akin to handing out prototype weapons to police departments, which doesn't happen specifically because they might fire themselves or blow up or fail in some annoying but not spectacular way like failing to feed new rounds.
        throwaway0a5e 5y ago
        The key mechanical features always "work" but little stuff like "gun fails to cycle 500% more often with the particular spec of ammo a particular force uses". A rifle spec'd out for sale to Cambodia probably is going to have a lot of teething issues if you try and sell it to the Swedes unchanged. Are these things "safe"? IDK. But no force is going to put in an order for tens of thousands of guns unless correctable issues like those are proven fixed. Infantry arms are a battle of margins. Everything "works" but a marginally more finicky gun to use is going to mean the people using it are marginally less effective which extrapolates pretty directly to "safety". At the end of the day the analogy isn't great and we're building a castle in a swamp here.
- dkjaudyeqooe 5y ago
  That reminds me of the ED-209 prototype in Robocop: "You have 20 seconds to comply!"
  - pmoriarty 5y ago
    Here's the "20 seconds to comply" scene from Robocop: https://www.youtube.com/watch?v=Hzlt7IbTp6M https://www.youtube.com/watch?v=Hzlt7IbTp6M
- renewiltord 5y ago
  This is a fair point. It's pretty bad that we have learner drivers on the road. Can't think of anything more public beta-y than that. Maybe trainee doctors?
  - gvb 5y ago
    I've had three "learner drivers" in my family and I spent quite a few hours as the "safety person" during their apprenticeship. During all those hours I never touched the wheel, brake, or gas. I gave advice occasionally, but never felt that the "learner driver" was driving unsafely. I assure you that 15 year old young adults drive much better than a Tesla "self driving" computer with millions of miles and millions of hours of training.
    - renewiltord 5y ago
      Oh I’m sure you were convinced but you never asked me if I consented to you randomly putting underaged kids behind the wheel of a two ton vehicle on a road which I share with you. You’re a regular Yaroslav Kudrinsky, just with better luck. But I suppose one can go around swinging swords in a crowd, hit no one, and convince oneself of one’s good judgment.
  - rhizome 5y ago
    Category error: people have brains.
    - HarryHirsch 5y ago
      That's the important one - a Tesla has no concept of the real world. Remember that clip where a Tesla was slowing down because it mistook the full moon in the sky for a traffic light? Likewise, no one would aim at a cyclist, except when they are a homicidal criminal.
      - noizejoy 5y ago
        … or under the influence of some substance, or a health event, or voluntary or involuntary distraction, or …
- FireBeyond 5y ago
  Someone said in another thread earlier today that there is a worry that we are making the path towards FSD too erratic, and that if we screw it up, the next shot might be in generations. People are arguing, with a straight face, that we are ruining (aspects of) humanity by not letting Tesla be laisez-faire with such things (and lets be real, it's predominantly Tesla who is just in full Cowboy/YOLO mode... other players in this field certainly aren't perfect, but they are at least trying to be conservative and cautious).
  - jimmydorry 5y ago
    It's still a regulatory failure in the end. If the government does not want to lay down exactly what path should be taken in getting to FSD, then we should not complain when we don't agree with the path that the manufacturers choose. As long as Tesla can demonstrate a similar or better rate of serious harm than human drivers, why should we stop them? It's up to government to do their job and outline exactly what price society should bear in the pursuit of this paradigm shift (because let's not kid ourselves, FSD is a complete shift in paradigm for society).
    - FireBeyond 5y ago
      > As long as Tesla can demonstrate a similar or better rate of serious harm than human drivers, why should we stop them? Because it can't, or won't. Instead they release statistics that _imply_ that it is better, that they have to know are absolutely misleading, because anyone who did high school statistics would be aware thereof. Everything else, they refuse to release.
      - jimmydorry 5y ago
        I'm not aware of _anyone_ releasing such data in a way. Not that I have been looking that closely for it (so I certainly could have missed this!). This is another major regulatory failure. Government should lay out the minimum data that the companies must publish, so we can get a better idea of their performance. (It's still a chicken and egg scenario though. You can't get that kind of data anywhere but on public roads in uncontrolled environments.)
Retric 5y ago
I was expecting autopilot to be seriously dangerous, but the data doesn’t back that up. Out of 234 deaths from accidents involving Tesla’s 12 people have died while Tesla autopilot was known to be in use or had recently been in use. http://www.tesladeaths.com http://www.tesladeaths.com Autopilot might be slightly worse than human drivers depending on how you slice the data but that’s about it. I don’t think it’s a regulation problem as much as it is an acceptance by regulators that driving is inherently dangerous and autopilot isn’t dramatically worse. It’s not even obvious if on net more people would have died if Tesla had never released autopilot.
- andbberger 5y ago
  the data is extremely questionable
  - Retric 5y ago
    In what way?
    - andbberger 5y ago
      Sampling bias, confirmation bias. Did you delete your link to tesladeaths.com because you realized the fine print about sketchy autopilot data contradicted your point? Oops!
      - Retric 5y ago
        No, I was trying to keep the link without a http:// http://, https://www.tesladeaths.com/ https://www.tesladeaths.com/ or https://www.tesladeaths.com/ https://www.tesladeaths.com/ works www.tesladeaths.com doesn’t nore does www.tesladeaths.com/ I am sure I have seen comment links without the http:// http:// but might have just gotten so used to it than I stopped seeing it.
        andbberger 5y ago
        uh huh. why do you think it's reasonable to conclude that this system is safe given that the reported data is based on faulty analysis [0] and inexplicably exempt from reporting requirements for testing self-driving cars? [0] https://jalopnik.com/feds-tesla-autosteer-safety-investigation-was-bullshit-1832542003 https://jalopnik.com/feds-tesla-autosteer-safety-investigati...
paulcole 5y ago
> In the end it's a regulation problem, not a Tesla problem Are you joking? If you’re the one causing the problem, you’re the problem. No regulation can prevent me from kicking someone in the shins. This isn’t a regulation problem, it’s a me problem.
- nrb 5y ago
  The <Insert State Here> Code certainly applies penalties for assault and battery, and you can rest assured knowing shin-kicking would happen regularly without it. The car problem presented here is quite a bit more shades of gray and we absolutely need to set acceptable parameters for the technology.
  - paulcole 5y ago
    Yes, but my point is that if you’re being a jerk, then you’re the problem, not the person who is not saying, “please don’t be a jerk.” You don’t need a regulation telling you not to be a jerk — unless of course you are a jerk. And even then, you’d probably still be a jerk, regulation or not.
- XorNot 5y ago
  Except it is: literally none of this is illegal. And it's not clear how it could be illegal: what's the distinction between cruise control and lane keeping and these systems? Very little - nothing you can codify. It's also very likely that this system can pass a standard driving test: AIs are really good at handling fixed test cases. So any regulation you do pass has a complicated assurance problem: how does it prove the system is safe? Pretend Tesla just knocked "beta" off the name and said "this is a limited self driving system, Tesla exclusive" - then you wind up in the same bucket with the same "it's the drivers responsibility problem". People see this footage and assume Tesla self drive does this all the time - but it wouldn't. It probably passes this scenario in testing 100% of the time: but the issue of course is the test is not quite whatever this circumstance is, and the system has a dramatic failure. There is absolutely a regulation problem.
  - 5y ago
salawat 5y ago
...I am aware of zero other manufacturers brazen enough in terms of blatant disregard for Engineering Ethics to have embarked on this type of testing before Silicon Valley did. In fact, there were licensure processes for this sort of thing already in place in California iirc, so this is less gray area, more "lets outsource testing to the end user". Frankly, it's downright disturbing.
cwkoss 5y ago
What does MCAS mean here?
- ofcrpls 5y ago
  https://en.wikipedia.org/wiki/Maneuvering_Characteristics_Augmentation_System https://en.wikipedia.org/wiki/Maneuvering_Characteristics_Au...
- sho_hn 5y ago
  It's a software system overriding pilot input on Boeing 737 MAX airplanes, designed to make up for cost reductions in adequate hardware design. Pilots were deliberately not well-informed and the system caused two crashes. It was brought up elsewhere in the discussion. In general cars are headed into an aviation-like situation with automation, so case comparisons are becoming more frequent. It's very apples to oranges here, but loosely relevant wrt/ information/documentation and in the cameras-vs-LIDAR/sensor fusion debate.
  - rhizome 5y ago
    I'm sure the policy and regulatory systems will be able to fend off the lobbyists of the richest person in the world working to get the word "liability" redefined in the law. \s
wavefunction 5y ago
lol @ public beta
oogabooga13 5y ago
I've wondered if it would be a good idea to have some sort of external indicator on cars that indicates to others that are driving, cycling, on foot that the vehicle is in autonomous mode. At least then maybe one can understand the potential risk driving around them. (Noted that an indicator doesn't really help if the vehicle is behind you in a lot of instances!)
- heavyset_go 5y ago
  One thing I've noticed the past couple of years is that it isn't that hard to determine whether or not a car is autonomously driving. If they're driving like they're drunk and they're in a Tesla, 9 out of 10 times, you'll see that that the driver isn't paying attention to the road at all because they're using Autopilot or FSD. I just treat the Tesla drivers around me on the road as if they're drunk and keep my distance.
  - sho_hn 5y ago
    Elsewhere in the thread a FSD Beta participant is imploring other drivers to keep their distance to Teslas because the software "gets easily spooked". This makes it even harder to assess the safety of the system - other drivers are helping it along as well.
- sho_hn 5y ago
  Several manufacturers have experimented with such systems in concept cars, e.g. animated grilles that signal in green when an oncoming car has recognized a pedestrian and is slowing in response.
bcrl 5y ago
A public "beta" of a self driving car should lean towards failing safe. I would expect it to have sudden braking issues and be overly conservative. The kind of bugs we're seeing where obstacles are blatantly ignored make it feel more like an "alpha" release. But hey, we're not designing airplanes here, just thousand pound chunks of metal flying around roads capable of killing people.
jrib 5y ago
MCAS: https://en.wikipedia.org/wiki/Maneuvering_Characteristics_Augmentation_System https://en.wikipedia.org/wiki/Maneuvering_Characteristics_Au...

lvs 5y ago

This is normally when regulators step in.

ModernMech 5y ago

The thing that really grinds my gears about all of this as a roboticist, is that as a community we had a pretty good culture of taking safety seriously. In 2014, the idea of unleashing onto the public admittedly beta-quality autonomous car software, on a platform that eschews industry best practices was unfathomable. Then came Autopilot and later FSD, and Tesla threw the concept of safety was thrown out the window. An autonomous Tesla decapitated a man in 2016, and Tesla continued unabated! What actionable steps did Tesla take as a result of that incident to make sure it never happens again. Even today in 2022 you can find videos of Teslas on public roads just completely failing to detect and colliding with stationary objects. The culture at Tesla seems to not care this is happening, as it continues to happen. It used to be that when your robot did a single thing wrong or caused the most minor of injuries, let alone it killed someone, then you engineered the hell out of it and implemented safety protocols to make sure that failure mode would never happen again. Not at Tesla, apparently. I get the utilitarian argument that maybe self driving cars could reduce traffic accidents in the future by being better than humans on average. But safety doesn’t happen unless you really work at it, with intention. Using your customers to beta test your 2 ton robots on public roads is not a safety-focused mindset. Tesla is not a company that values safety first when it comes to their autonomous car project.

belter 5y ago
Why all the euphemisms and diplomatic language? There is one word for it: Criminal.
kodah 5y ago
I haven't worked in robotics, but I've worked in automation for quite some time (digital and real-world). Two examples I can think of: - SCADA always had hot, hardware backups and failure detection. To the extent that I thought this was dogma. - Physical automation (like car construction equipment and robotics, pick and place machines, etc) have safety zones that humans aren't allowed in or near. Same thing for Amazon warehouse robotics. I'm not sure how we leapt to, "beta test cars on the street" from there.
- TOMDM 5y ago

belter 5y ago

https://videos.dailymail.co.uk/video/mol/2022/02/08/2352711287929719429/640x360_MP4_2352711287929719429.mp4 https://videos.dailymail.co.uk/video/mol/2022/02/08/23527112...

ape4 5y ago
Going out of its way to hit the bollard - wow
- belter 5y ago
  Did you notice the most likely reason? Slightly to the right, of the line of bollards, there is a small patch of road and another strong white line. Kind of almost 1 meter, ( 3 feet) wide. So it thinks the tiny patch between the bollard, and the right most white line, is the right driving side of the road.

colinmhayes 5y ago

40,000 people die in car crashes every year in the US alone. If the lord farquad approach brings widespread autonomous car adoption which decreases that number substantially even a month sooner it's the right thing to do. Many of us are already dying.

46756e 5y ago
Many people die of HIV, it’s a plague on society. If we just killed everyone who has it, it would be gone. Many of “us” are already dying, and it saves lives in the future. /s
ModernMech 5y ago
Sorry, but I don't see how a company that... - Beta tests critical software in the public which has lead to actual customer deaths; - Refuses to follow industry best practices when it comes to safety sensors; - Refuses to seriously investigate safety incidents and implement technological and cultural fixes to prevent them from happening in the future; ...will lead us into a utopia of automotive safety. At best what they're liable to do is create a climate where mega corporations feel free to use public spaces as their own personal laboratories, because the ends justify the means after all (don't pay attention to the decapitations, we're doing it for the safety of us all). We will all pay the price for that.
- colinmhayes 5y ago
  I agree it would require an incredible amount of evidence to convince me that this actually would speed up the development of self driving. Just saying that I'm not opposed to the lord farquad method if it helps.
- randoglando 5y ago
  Don't forget to add - willfully misrepresents data to imply a better safety record than they actually do have https://twitter.com/Tweetermeyer/status/1488673180403191808 https://twitter.com/Tweetermeyer/status/1488673180403191808 (I've posted this elsewhere on this thread too)
5y ago

lhorie 5y ago

For Uber at least, self driving was effectively a money-losing pipe dream, so it kinda did itself a favor by divesting via the sale of the division to Aurora. For Tesla, though, FSD is still such a front and center thing that I'm having trouble seeing how it ever plans on saving face as it gets more and more bad press from incidents like this.

rhizome 5y ago
It's not money-losing if you take the program's effect on stock prices into account, and maybe you'll even accept that it's always been a stock manipulation tool.
twblalock 5y ago
The feature sells cars. Remember that guy in southern California who got arrested for sleeping in the back of his Tesla while it drove him home, and he said he did it many times and would do it again? The system works well enough that he did it many times before he got caught. That guy is an idiot, but think about that story from another angle: You can fall asleep while driving any car. The difference is, if that car is a Tesla, you might not die. As a driver aid for non-crazy people FSD is a good system and Tesla owners like it. It's far from being vaporware.
- andbberger 5y ago
  dang is going to ban me if I share how this take makes me feel. so let me just say that the proliferation of evidence-resistant memes such as this one concerns me greatly, and I feel even more unsafe as a pedestrian and cyclist on the street than I did before knowing that there are people who think that this is reasonable.
- jquery 5y ago
  > he said he did it many times and would do it again? Based on the videos I’m seeing of FSD in action, I have my doubts. Wouldn’t be surprised if this was a publicity stunt. Elon Musk has a very loose relationship with ethics and the truth. They’re just obstacles for him.
- 5y ago

throwawaylinux 5y ago

I also think there is a certain fear or outrage around this stuff which is fueled by "technology" or the unknown. People are some of the worst drivers around, they cause all kinds of problems with cluelessness and inattention and they crash and die and kill and injure in all sorts of stupid ways. They get drunk, high, check their phone, fall asleep, speed, show off I think it's great that there are still companies out there throwing research into moonshot types of research projects like this. Automobile deaths are perhaps the biggest and most persistent pandemic we've faced, we've been facing for a hundred years, and it kills and maims young and old alike. Millions of people dead over the years. Many were willing to relax some of the protocols around vaccine testing to bring the covid vaccines in to use earlier, put mandates and coercion around them, and yes some suffered adverse reactions and even died from them. Still, it was thought to be a reasonable compromise by many. I hope we don't go the other way on other technology that can advance society and potentially save lives just because not everything goes perfectly from the start, or we get stuck in the pharmaceutical corporations = bad mindset. Not to say there perhaps shouldn't be more regulation around it, but a serious and careful study of risks and benefits I think is appropriate rather than handwringing and outrage.

couchand 5y ago
We don't need moonshots to solve the car pandemic. Strong Towns and others have shown that we already have plenty of common-sense, effective tools that we've simply chosen not to roll out on a wider basis. We don't need new technology, we need new leaders.
- throwawaylinux 5y ago
  Of course we do. If you can get "Strong Towns and others", whatever exactly that is, to end the car pandemic, that would be a a moonshot wouldn't it? Maybe not technologically but politically and socially. You're not the first one to have thought it's a good idea or that we should try to do something about it. It's not like it's easy and people have just decided not to.
doodpants 5y ago
> People are some of the worst drivers around Compared to which other species?

twblalock 5y ago

Self-driving cars will never be a reality if the standard for success is zero deaths during the early stages of development. It just won't happen. The technology won't ever be good enough if it's not tested in the real world. Furthermore, no self-driving system will be perfect even in a mature stage of development. People will still die, and they may die because of mistakes a human driver would not have made. If society isn't willing to accept that sacrifice, then let's just ban self-driving cars. But let's not try to have it both ways -- let's not allow self-driving while also holding the view that risk of death is unacceptable.

admax88qqq 5y ago
We don't have to ban self driving cars. We just set the success criteria at zero deaths. If companies want to spend money pursuing it believing they can hit that goal that's on them.
- twblalock 5y ago
  > We don't have to ban self driving cars. We just set the success criteria at zero deaths. That's equivalent to a ban.
  - rhizome 5y ago
    Because the technology is a non-starter and will never work unless accepted terms of public safety are redefined in its favor. "Just draw a bigger bullseye!"
  - Karrot_Kream 5y ago
    There's multiple alternatives to driver-driven cars that result in many fewer excess deaths. They've been trialed and deployed in many countries very successfully. You may know of them by common names such as "buses", "streetcars", "trains", "metros", or even "people movers". There's nothing wrong with innovation but we do not need to put self-driving technology on a pedestal when there are alternatives that exist today that move people around that have none of these externalities. If entities want to innovate, let them innovate within safety parameters.

davej 5y ago

> "Some of you may die, but it's a sacrifice I am willing to make." Honest question: If beta testing speeds up the transition to FSD by x months/years and causes y deaths during the intervening period but FSD ultimately reduces the number of road deaths by z per day. For what values of x, y and z is it an acceptable trade off? Zeroism is often overly simplistic. y being non-zero might be hugely net-positive in the long run.

tptacek 5y ago
No, of course it isn't; think, for instance, of applying that same logic to other circumstances. You can't knowingly or recklessly endanger people simply because your long-term intentions are benign.
- davej 5y ago
  There needs to be some testing of FSD solutions in real world scenarios and there will inevitably be deaths. There are thousands of deaths on the road every day right now. Even if FSD is mature and 1000x safer than human driving then there will still be deaths. Are you suggesting that if FSD is 1000x safer then it still shouldn't be allowed on the road because there is the possibility of deaths? I'm not claiming that FSD should or shouldn't be allowed on the road right now. My point is that we shouldn't wait for the chances of death to be zero.
  - heavyset_go 5y ago
    > There needs to be some testing of FSD solutions in real world scenarios and there will inevitably be deaths. There are thousands of deaths on the road every day right now. This is a solved problem with experimental design. Experiments are designed and reviewed by ethics boards, and participants are gathered after getting their informed consent and giving them just compensation for the risks involved. "Testing of FSD solutions in the real world" can follow the same path in their experimental designs, but they don't, because executives and shareholders don't want to pay for expensive ethical experimentation. They'd rather move fast and break bodies. Most modern life saving devices and treatments have gone through exactly this type of ethical experimentation. There's no reason Tesla's products couldn't, as well.

AtlasBarfed 5y ago

Not that it justified overhyping or undercaution by Tesla, but lets be real: everyday driving comes down to that same acceptance of risk Let's not be perfect be the enemy of the good: a self-driving highway system can probably be achieved with current technology that beats human drivers for long distances. I wish that was the priority rather than the robotaxi obsession. Self-driving highway systems would produce immediate profit and economic savings that could fund the drive-grandma-to-Taco-Bell systems, as well as the convergent infrastructure, common signaling systems, embedded sensors, better signage and standards, improved and predictable construction signals and routing, etc. Following the usual hype cycle, it will be highway logistics that smooths over the "despair" portion and does the practical incremental improvement.