20 ms·

Drivers react to Tesla’s full self-driving beta release

129 points by elsewhen 6y ago

candiddevmike 6y ago
What will it take to stop them? I believe it's a settled matter that their self driving is fundamentally flawed due to relying on ML and cameras instead of lidar. Will it take people dying for the government to step in? Lawsuits? I don't think our politicians are well versed enough in how this works and the level of risk letting this kind of thing happen.
- sokoloff 6y ago
  Is that a permanently settled matter? When humans drive, they are relying almost exclusively on visual inputs. I don’t see a reason to believe that humans are so far superior at visual processing that no self-driving based on video could best them and many reasons to think that we’re inferior in certain aspects of the task.
  - mdorazio 6y ago
    1) Human eyes are superior to any camera system in a Tesla. both in resolution and dynamic range. 2) The human visual system is backed by the equivalent of strong, general AI that intuits and understands the entire world around the vehicle and all objects interacting with it. It is in theory possible for a video-only system to work, but ML is nowhere near the level of development needed for that to be a reality.
    - mlindner 6y ago
      Sure they're superior, but you don't decide whether something is flawed by whether it's superior or not. Our road systems are not designed for the full dynamic range of the human eye, which is why we drive with headlights and we have street lights on highways rather than relying on night vision for all driving. > The human visual system is backed by the equivalent of strong, general AI that intuits and understands the entire world around the vehicle and all objects interacting with it. I think the Human general AI vastly overestimates it's ability to drive and respond to conditions that suddenly appear when driving. > It is in theory possible for a video-only system to work, but ML is nowhere near the level of development needed for that to be a reality. [citation needed]
  - candiddevmike 6y ago
    ML shows no signs of achieving intuition, which allows us to predict various scenarios in real time and adjust accordingly.
    - sokoloff 6y ago
      Does “instead of LIDAR” mean that LIDAR solves that in some significant way?
      - MontyCarloHall 6y ago
        Nope. I don’t think we’ll have Level 5 autonomy with safety on par with human drivers until we have general AI. At that point, whether or not that AGI had LIDAR supplementing its visual inputs becomes irrelevant.
        ggreer 6y ago
        Doesn’t Wayno already have level 5 autonomous vehicles operating in the Phoenix area? So far they’ve had about 65,000 driverless miles[1] (and 6 million miles with safety drivers). That number should increase pretty quickly now that they’re scaling up their service. So far it seems like the main problem is that they get rear-ended by inattentive human drivers. 1. https://www.engadget.com/waymo-indepth-details-selfdriving-activities-phoenix-163000481.html https://www.engadget.com/waymo-indepth-details-selfdriving-a...
      - tsimionescu 6y ago
        With camera-based sensing, you need AI to interpret the inputs in addition to AI for deciding how to react to those inputs. With LIDAR, the job of detecting objects seems to be much simpler. Taking a relatively simple to work around example, you need AI to distinguish lens flare from an object approaching/departing.
        mlindner 6y ago
        With LIDAR the job of detecting some objects becomes much simpler but for other objects becomes impossible.
  - FartyMcFarter 6y ago
    Look at the car's display in this video: https://youtu.be/RN5Qoei7v1k?t=2281 https://youtu.be/RN5Qoei7v1k?t=2281 The "lane" in front of it is wildly moving around, so it's definitely misinterpreting the visual inputs.
    - xeromal 6y ago
      As an owner of a tesla, the rendering on the screen and what the car actually does with the inputs don't always correlate. The rendering is purely for your own viewing pleasure. Hell, there's a disconnect where if you have the new autopilot computer but the old entertainment screen computer, it didn't even have the capability to render anything worthwhile but you still had self driving
      - gamblor956 6y ago
        Given that the rendering on the screen is supposed to be showing the inputs the car thinks it is receiving, it is even more concerning to learn that what the car actually does isn't correlated with the inputs it is receiving.
        sokoloff 6y ago
        That seems intentionally tortured logic. Why would it be a given for the rendering to show the inputs as received rather than being an entertaining caricature loosely related? Surely you don’t think the animations of the car’s battery being charged are realistic in any engineering or physical sense.
        gamblor956 6y ago
        Because Tesla says that it is a rendering of what the car thinks it is seeing, meaning that the rendering is supposed to roughly correlate with the information upon which the FSD systems are acting. In contrast, the animations of a car battery being charged don't claim to be realistic in any sense, and they deliberately attempt not to be realistic: oversized images of batteries or other standardized iconography, stylized arrows or other indicators of charging "flow", etc.
  - rainyMammoth 6y ago
    >> Is that a permanently settled matter? When humans drive, they are relying almost exclusively on visual inputs. I really don't understand how so many smart people are repeating this argument over and over. it's one the most flawed arguments that was used by Elon Musk to sell his lack of ability to retrofit lidars in every single car. We also got a single brain, so why can a computer not do what we do?
  - perryizgr8 6y ago
    I'm fact I'd argue the opposite is true. It's settled that the endgame of self driving will rely only on visual input, since that is what humans need.
- 205guy 6y ago
  You could short Tesla stock. It is well proven that the free market will correct mistakes like this. /s
- SheinhardtWigCo 6y ago
  It will take a few serious accidents that most humans would have avoided.
rkochman 6y ago
It’s not as simple as “the driver can take over if they need to.” In many cases it takes a human too long to recognize the system doing something dangerous and to intervene. This is going to end badly.
- t0mas88 6y ago
  Indeed, recognizing the problem takes vital time and taking over to execute a maneuver is way harder than executing the same maneuver while you're already in control.
- crabcakes010 6y ago
  But think of the upside -- they could create serious value for their shareholders!
Sebb767 6y ago
> “Oh Jeeeesus” > In another video, Brandon's Tesla was making a left turn but wasn't turning sharply enough to avoid hitting a car parked on the opposite side of the cross street. "Oh Jeeeesus," Brandon said as he grabbed the steering wheel and jerked it to the left. "Oh my God," Brandon's passenger added. Maybe it's just me, but this screams clickbait and low effort. The overall article is fine, but I nearly stopped reading after this. Is this kind of writing really necessary nowadays?
- Waterluvian 6y ago
  I think given the high stakes of killing yourself or other people, we should be brutally unforgiving in highlighting major failures of killer technology.
  - rubyron 6y ago
    “Killer Technology” - the Tesla marketing team is going to use that now.
  - Sebb767 6y ago
    Fair enough. But on the other hand, if our headlines we're like this: > OOOOH MY GOOOOSH! - Data scientist REACTS to possible DATA BREACH by HAXORS ... I doubt it would help highlight the need for security. On a more general level, I personally don't think it's worth to trade reputation for attention via clickbait. But it might be the better option, I honestly don't know.
    - deft 6y ago
      In that case a computer isn't nearly killing someone. Very obviously not a fair comparison.
      - grecy 6y ago
        And in the videos shown the vehicles are not going fast enough to kill any occupants either.
arkitaip 6y ago
Terrifying scenarios are described in the article. This is obviously nothing more than (very) advanced lane assistance and absolutely requires fully observant drivers who might have to take over the wheel in the fraction of a second.
- t0mas88 6y ago
  Humans aren't designed to monitor something that requires no action for a long time and then suddenly requires attention, nor are we good at taking over control of something at the last moment. You're not in the "flow" at that point. The article is correct that expecting average drivers to do this without training is a high risk move. I've given flight instruction, taking over a landing 20ft off the ground is way harder than landing a plane yourself. And that's with a lot of training, not just an average driver being put in a place to supervise Tesla autopilot with no training at all.
  - 1337shadow 6y ago
    > Humans aren't designed to monitor something that requires no action for a long time and then suddenly requires attention Isn't that exactly the definition of cruise control ? I'm sorry but I won't be driving a car without cruise control anymore: cruise control saves oil and is much more comfortable.
    - sibartlett 6y ago
      With cruise control you’re still very much the one who is driving and paying attention.
      - 1337shadow 6y ago
        Ok, I can't imagine myself dropping attention from driving. As a member of a national roadkill prevention awareness org and having lost close friends on the road: I would never had imagined that it would require you to drop attention. I imagined it was something you could enable and then disable by just breaking like cruise control or auto parking with toyotas. But now I see how complex it becomes because of the wheel, even tesla owners in the comments of the article don't like to let their hands off the wheel.
nostrademons 6y ago
"Move fast and break things." I really don't understand how a company can get away with using its customers as QA when they sell products that can kill people. It's one thing if it's a website for throwing virtual sheep at another; nobody really cares if your Facebook feed omits an update or two. It's another thing if a software bug means that you die. Or at least it should be - I'm amazed at the number of people who seemingly don't care. I also wonder what this is doing to Tesla's brand reputation. I was just in the market for solar panels and Tesla's offering (both the panels and the solar roof) was very, very attractive on price & aesthetics, but their reliability record with their cars made me think twice about buying a 25+ year product that keeps my home powered and dry.
- ericmay 6y ago
  Hate to say it but I think you’re in the minority with that opinion. Reaction from most people has been “woah”. Also, there is no moving fast and breaking things here in real terms. Drivers are and have to pay attention while driving, and Tesla is strictly monitoring this with beta. You also specifically have to opt-in to beta software. 99% of Tesla owners do not have this functionality.
  - dontcarethrow2 6y ago
    I think we are mainly silenced by the hype folks who can't not tell the rest of the internet about their excitement. I'd like to think OP's opinion is the majority, hard to tell on a startup forum where most folks are hodling TSLA.
    - Smaug123 6y ago
      I would be astonished if a majority of people on HN have Tesla stock, for what it's worth.
      - OJFord 6y ago
        Would you only be astonished if they directly hold it, or also if it's via a fund or three? (I think you're probably right they don't, HN international and diverse enough, but I wouldn't be astonished.)
cblconfederate 6y ago
Those are going to be good arguments for the gun lobby people.
- speedgoose 6y ago
  I don't get it, why?
FpUser 6y ago
>"I really don't understand how a company can get away with using its customers as QA when they sell products that can kill people." Since when did this ever stop companies when there are money to be made. Driving cars in general kills tens of thousands of people per year and so do many other activities. For governments it is just a question of how much money vs number of people killed.
NewEntryHN 6y ago
Is the trick here that the driver is responsible in case of accident? That would be evil genius.
- deleted 6y ago
  [deleted]
cmckn 6y ago
I think I get less interested in self-driving cars the closer we seem to get to them. It honestly just stresses me out. Driving is something I enjoy doing. I’m not sure I will ever feel comfortable taking my hands off the wheel. This has little to do with how well the tech works in tests or anecdotally; I think I just don’t want this.
- agumonkey 6y ago
  For dense urban areas (low speed, lots of stopping) I could see waymo tiny wheelhubs working. A tiny taxi bubble.
  - cmckn 6y ago
    I agree. I think I’d feel differently if the vehicles were more like trains on a track, if the roads had some added features that guided the vehicle. I just don’t think I’m down with the autonomous approach where the car is trying to make all the decisions I do.
- toast0 6y ago
  I'm a staunch grumpy old man, but our minivan has lane keeping, automated emergency braking and radar cruise control. The lane keeping and emergency braking are just there and mostly help out (although, apparently I like to drive on the line when exiting a freeway, so it's kind of annoying there). Radar cruise would be nice, except I don't do much long distance driving, and I don't like the following distance options, a different car might give different options that would be better. I wouldn't like to be in a car that drives like either the Tesla or the Waymo cars though, the Tesla's are too aggresive, and the Waymo cars are not aggressive enough (which is generally good and safety aware, but would still be frustrating to ride in).
- rsynnott 6y ago
  > I think I get less interested in self-driving cars the closer we seem to get to them. It honestly just stresses me out. That's probably a reasonable response. These things have the nasty feature that they both require less input from the driver the more sophisticated they get, but also that they require more _attention_ from the driver the more sophisticated they get (a boring old cruise control system won't abruptly swerve into a concrete road divider, but a high Level 2 system will). And humans just don't work like this. "Don't do anything, but be ready to take control within a second or so when it does something insane" is not something we're good at.
kryptiskt 6y ago
Bet it's only named "Full Self-Driving" to get out of their obligations to customers who bought that package.
- scollet 6y ago
  You can drive yourself anywhere!
helsinkiandrew 6y ago
When Elon Musk said real world testing was needed, I assumed it would be Tesla test cars with trained ‘drivers’ not beta testing guineapigs. Given that there’s regular stories of people playing phone games and crashing whilst using the current lane assist tech, are we safe? ‘Real-world testing was needed to uncover what would be a "long tail" of problems, he added.’ https://www.bbc.com/news/technology-53349313 https://www.bbc.com/news/technology-53349313
- rjkennedy98 6y ago
  Its definitely going to be real world drivers who opt in at their own risk. My understanding is that Tesla views its large number of cars as a test ground to get to level 5 autonomy. It will slowly roll out more and more autonomous functionality and incrementally get there. You simply cannot solve the tail end of issues with test driving alone.
  - rrss 6y ago
    > drivers who opt in at their own risk If it was actually just at their own risk, it would be substantially less bad. They opt in at their own risk, and at the risk of those around them.
    - nullc 6y ago
      I learned that Teslas had some self driving functionality by seeing obviously inattentive drivers in them. While on the road I try to stay as far away from any tesla as I can.
- justapassenger 6y ago
  > When Elon Musk said real world testing was needed, I assumed it would be Tesla test cars with trained ‘drivers’ not beta testing guineapigs. Cynical view - it's being tested by twitter users who are on good terms with a company, with an explicit goal of generating more hype for Tesla.
Strilanc 6y ago
Tesla should follow Waymo's lead and release statistics on collisions while self-driving, and also the number of simulated counterfactual collisions avoided by the human taking over [1]. Based on the multiple incidents from the individual drivers cited in the article, it sounds like the numbers wouldn't be very good. 1: https://storage.googleapis.com/sdc-prod/v1/safety-report/Waymo-Public-Road-Safety-Performance-Data.pdf#page=5 https://storage.googleapis.com/sdc-prod/v1/safety-report/Way... Disclaimer: I work for Google but am not involved with Waymo. Opinions my own.
- mlindner 6y ago
  > Tesla should follow Waymo's lead and release statistics on collisions while self-driving Tesla does rely statistics on collisions while self-driving. https://www.tesla.com/VehicleSafetyReport https://www.tesla.com/VehicleSafetyReport > In the 3rd quarter [of 2020], we registered one accident for every 4.59 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot but with our active safety features, we registered one accident for every 2.42 million miles driven. For those driving without Autopilot and without our active safety features, we registered one accident for every 1.79 million miles driven.
  - Strilanc 6y ago
    This is good, I had forgotten they released that data, but I was still asking for a lot more. For example, Waymo bucketed accidents by severity and went down to very low severity (like a pedestrian bumping into the side of a stopped car) whereas on that site Tesla is essentially only reporting airbag deployment counts. (To Tesla's credit, they have more time granularity than Waymo.) I do think the counterfactual "how many times did the human driver taking over prevent a crash" data is really important to have. In the article, the scary near misses they talked about fell into that category rather than into the "actual airbag deployment" category.
    - mlindner 6y ago
      > I do think the counterfactual "how many times did the human driver taking over prevent a crash" data is really important to have. In the article, the scary near misses they talked about fell into that category rather than into the "actual airbag deployment" category. It would be nice to have however I think it misrepresents. If the end result of having Autopilot on is that the driver, on average, crashes less then it's an improvement, even if the driver has to regularly intervene. What's important is the end effect, not how it happens.
kthejoker2 6y ago
Without commenting on this particular technological advance, it did make me think of the early days of the automobile https://amp.detroitnews.com/amp/26312107 https://amp.detroitnews.com/amp/26312107 My key takeaway is regulation of advanced technology lags waaaaaaay behind the disruption caused by the technology.
- 1337shadow 6y ago
  Is it really possible to regulate without any experience at all ? I thought regulation was something that you could define based on a sufficiently widespread practice.
  - timmytokyo 6y ago
    I can guarantee you that if Tesla's auto-driving starts killing people, there will be near-instant regulation. For that reason alone, supporters of auto-driving technology should be aghast at what Tesla is doing.
    - Espressosaurus 6y ago
      It has already killed three people. The question is how many more will it kill before real regulation starts hitting? Waymo has been conservative, which is the right way to go about this. The number of outright faulty actions I'm seeing on these videos is terrifying. Pedestrians and other cars on the road didn't ask to be part of this live fire experiment.
  - Pfhreak 6y ago
    >Is it really possible to regulate without any experience at all ? Yes. You can absolutely set safety metrics and boundaries. They might not be perfect, but as you get experience you refine those boundaries. Just like settings alarms in web services -- you have a sense of roughly where the performance should be, and you set some alarms. You iterate on those alarms as you get data, but you still absolutely have a sense of what's roughly appropriate from the start.
scared2 6y ago
The main problem is that Tesla autopilot uses pytorch which is a prototype and toying library designed by an intern as compared to tensor flow which was designed from ground up targeting realistic applications such as this one.
- aeternum 6y ago
  Comma.ai recently switched their entire codebase from tensor flow to pytorch, how do you explain that?
- orbifold 6y ago
  Completely irrelevant since the model is run on a custom chip. So no part of the pytorch code actually ever makes it into production.
- MontyCarloHall 6y ago
  I'd be very curious to hear some elaboration on this claim; it's the first time I've ever seen it. From my point-of-view, the fact that TensorFlow completely scrapped its entire ≤v1.0 graph-based API for a PyTorch-style eager execution API for v2.0 onwards does not suggest that it was “designed from the ground up” in any way, shape, or form. We're talking a total rewrite of the entire frontend codebase going from v1.0 to v2.0, and a total abandonment of all v1.0 style code. TF 2.0 is basically a brand new language. Under the hood it's all the same BLAS/LAPACK operations anyway. I've found TF superior to PyTorch for stats methods and probabilistic programming in general (far richer API), but for standard linear algebra the two are equivalent performancewise, with PyTorch having the superior API IMHO.
  - scared2 6y ago
    Just being easy to use doesn't mean it has a solid back end. I agree pytorch is easier to comprehend UI for average person.
1337shadow 6y ago
I guess that's why it's still beta ! Kudos to the driver ++
WhompingWindows 6y ago
Tesla is very "thrifty" with their FSD software. Other companies need to hire safety drivers, but Tesla just foists that labor onto their paying customers, and any liability to boot.
- mlindner 6y ago
  Who very willingly enable the option after several warnings that the software is beta. In fact tons of people are begging to get access.
num3ric 6y ago
The line selection for turns seems to be quite poor still.
deleted 6y ago
[deleted]
Zigurd 6y ago
If you don't qualify what you mean by "AVs will kill people" it's not useful. People can, and do, accidentally and intentionally, step in front of vehicles. They also intentionally crash vehicles. They won't stop because of AVs. Even the fact of suicide by vehicle is complex: AVs can prevent some suicide by driver but possibly raise the risk of pedestrians committing suicide by vehicle. One reason suicide by vehicle happens is because it can conceal the intent to commit suicide. This is compounded by suicide being a taboo subject. This alone will complicate and possibly delay adoption of AVs. Millions of innocent people are injured every year and about a million die every year in road accidents. Cutting that number significantly would have roughly the same impact on global well-being as curing a dread disease. There are potentially hundreds of millions of new customers for vehicles as developing nations grow their middle class. If humans drive those vehicles, hundreds of thousands more will die every year.
pmcollins 6y ago
You don't have to read Tim Lee's writing to know how the latest FSD performs. There are lots of videos on YouTube, and new ones posted every day. My personal opinion is that this is a quantum leap over the previous implementation. Much respect to the engineers at Tesla for this amazing accomplishment. https://www.youtube.com/results?search_query=tesla+fsd+beta https://www.youtube.com/results?search_query=tesla+fsd+beta
FartyMcFarter 6y ago
This thing is obviously not ready to drive properly: https://youtu.be/RN5Qoei7v1k?t=2218 https://youtu.be/RN5Qoei7v1k?t=2218 It can't even figure out which traffic lights apply to it.
- cma 6y ago
  Wow, that's like 10 disengagement's in just a couple minutes and it almost drives through a train boom barrier and does some scary stuff with a pedestrian...
  - FartyMcFarter 6y ago
    I missed that on the first watch but you're right, the driver engages the self-driving mode in front of the train barrier and it immediately accelerates towards it. This system is really bad.
- tsss 6y ago
  Unbelievable that the guy has to swipe and type on the center console while driving. Touch screens in cars should be banned and Tesla held liable for all accidents that happen while interacting with the console in existing vehicles.
- mlindner 6y ago
  The guy disengaged it just as it started accelerating. It's sometimes slow to respond to green lights right now. Give it a few more weeks and it'll be fine. Look at how it responds to pedestrians here: https://youtu.be/RN5Qoei7v1k?t=587 https://youtu.be/RN5Qoei7v1k?t=587 And his quote: > Improvements across the board, turned into Target and took me to the front! Drove around East Sac, through Sacramento State, Midtown, and Downtown. This release definitely was far less interventions for me. Also this guy in general gives a lot more vocal reaction than I would personally give in similar situation so it makes it look and feel worse than it actually is.
_greim_ 6y ago
By now it should be obvious—and this is purely descriptive of what I see playing out, not prescriptive—that self-driving tech will have to clear a higher bar than just statistically better in order to become mainstream. Not only because anecdotes carry more weight than statistics in the media—as this story illustrates—but because people are much less afraid of bad things happening, as long as they're in control when it happens. Also, I suspect everything associated with fuel-burning cars is by now irrevocably linked with cultural ideals of individualism and freedom. Resistance to EVs and self-driving tech will continually materialize, as if out of thin air, regardless of how well documented the benefits are. It's going to be an uphill battle for the foreseeable future.
- sidibe 6y ago
  But also there is no way Tesla FSD is anywhere close to being statistically better than normal drivers considering what we see in the videos. The reason I make this obvious statement is you are making it seem like the article is using anecdotes while there are statistics somewhere that show safety
- gamblor956 6y ago
  Anecdotes are data. They're a different kind of data, that usually leads to investigations resulting in more rigorously gathered data. Multiple drivers are reporting multiple incidents of varying severity in the short (week?) that the "full" FSD beta has been available. That's a huge concern, and it absolutely needs to be followed up on.
  - mlindner 6y ago
    The software isn't widely available. It's available to a tiny set of Tesla's "closed beta" customers. Tesla picks and chooses who it sends it to right now. Basically it's only being sent to Tesla large stock owners or people who Tesla has worked with in the past who are non-employees right now.
- gfodor 6y ago
  I'm not so sure. It seems equally if not more likely to me that people will eventually take a ride in a self driving car, be impressed since that's the statistically likely outcome, and slowly get used to it. Accidents from robocars will become things that happen to "other people" like all things that ultimately boil down to random chance.
Animats 6y ago
From the article: "These YouTube videos underscored how important it is for drivers to actively supervise Tesla's new software. Over the course of three hours, the drivers took control more than a dozen times, including at least two cases when the car seemed to be on the verge of crashing into another vehicle." In a way, it's good that the Tesla system sucks so badly. If they had a disconnect rate of one per month, drivers would trust the thing. The other guys, disengagements per 1,000 self-driven miles in California: Waymo, 0.076, or one per 13,000 miles. Cruise, 0.082, or one per 12,000 miles. Not clear how many of those would have resulted in a crash, as opposed to just stopping. Probably not many, the California autonomous DMV reports indicate. US humans have a crash rate of one per 508,000 miles driven. Anyway, Tesla does not have "full self driving". They have slightly better Level 2. Not even Level 3. You'd think that by now they'd have hands-off freeway driving totally automated to a better than human level of safety. But no.
- banana_giraffe 6y ago
  Makes me wonder: I have two driving modes, not autopilot, but not as aware as I could be, and then tracking everything I can because something odd is going on and I'm trying to avoid a bad scenario. Perhaps I'm a bad driver, but I suspect this is normal, kind of in a low-attention mode most of the time, but if the car a lane over starts to change lanes too close, you go into high-alert mode. I wonder if there are studies on how often humans need to do this sort of "disengagement" for lack of a better term? Not sure how you'd measure it, perhaps with more eye movement than normal? Dunno.
- supernova87a 6y ago
  You really shouldn't trust those disengagement statistics much at all. They are totally up to the individual companies what counts as "disengagement", how to define it, and how they report them.
- modeless 6y ago
  > Not clear how many of those would have resulted in a crash Waymo just published an extremely detailed paper explaining exactly how many of their disengagements would have resulted in a crash, among other things. It's very interesting reading and shows how far Tesla has to go. Summary: https://www.theverge.com/2020/10/30/21538999/waymo-self-driving-car-data-miles-crashes-phoenix-google https://www.theverge.com/2020/10/30/21538999/waymo-self-driv... The paper: https://storage.googleapis.com/sdc-prod/v1/safety-report/Waymo-Public-Road-Safety-Performance-Data.pdf https://storage.googleapis.com/sdc-prod/v1/safety-report/Way...
6gvONxR4sf7o 6y ago
"Full self-driving" > Tesla ... says it's not intended for fully autonomous operation. Drivers are expected to keep their eyes on the road and hands on the wheel at all times. If tesla rolls out a product where you have to pay close attention and be ready to take over and calls it full self driving, that seems like a massive case of fraud. I'd be pissed if I'd bought the FSD package.
- giancarlostoro 6y ago
  On the other hand I am okay with that requirement but it becomes as diluted as the term smartphone / smarttv / smart cars.... Dont call it self driving call it smart cruise control or something?
- p1esk 6y ago
  It’s a beta release
- mlindner 6y ago
  1. It's not even available to 99% of people right now. 2. It's a beta. 3. People bought into it early when the price was lower. It's a subscription to the feature set as it develops, not a guarantee that it works perfectly the day you pay for it.
gfodor 6y ago
Without releasing data, Tesla is forcing analysts to focus on anecdotes, which suffer from all the usual problems like selection bias. What we need to know: - How often do drivers intervene? - How often do drivers fail to intervene and the system causes an accident? - How often does the system take an action which has a high likelihood of having prevented an accident? - How often does the system do so when it seems likely a human driver would have failed to do so? - Integrated together, what's the expected dynamics of these probabilities and their net impact insofar as the system being releases more widely creates training data to help improve them more quickly over time? It could very well turn out to be the case that this system is purely positive, strongly net positive, or neutral in harm reduction. The question then is, given that, what ethics should inform its release: is putting the stress on the driver sufficient if it say, saves 1000 lives in the next six months and will harm noone in exchange, other than some drivers having to intervene and have moments of stress? What if it will save 1000 and 10 people will be harmed by failing to intervene? What if it's an even swap, harming people who fail to intervene in exchange for saving people who would have inevitably been lost to accidents they couldn't have prevented? Saying "this is wrong to do" based upon anecdotes is dumb analysis. Saying "this is wrong to do" based upon an absolutist form of ethics is fair, but it also means you reject that we should be trying to solve self driving. If you think it's wrong to do and are not doing either, you ought to articulate what scenario in terms of data would justify the action vs not, even if that data is unknown right now.
- kanox 6y ago
  Tesla released data for autopilot, FSD beta has been out for ~1 week so there's not much real data.
  - gfodor 6y ago
    Yeah, I'm not arguing that Tesla should have done something differently so much that I'm encouraging them to make sure they get data out ASAP.
- tsimionescu 6y ago
  > It could very well turn out to be the case that this system is purely positive, strongly net positive, or neutral in harm reduction It could also turn out to be negative, right? Certainly the anecdotes shown here would point much more to that - for example, its inability to turn sharply enough is indicative of some deep flaws. > If you think it's wrong to do and are not doing either, you ought to articulate what scenario in terms of data would justify the action vs not, even if that data is unknown right now. It's wrong to do as long as you rely on CUSTOMERS to ensure safety. There is no further argument to be had: as long as the system is not PROVEN to be safer than a human driver even in the absence of human attention, it can't be put into the hands of customers. That is not to say that they shouldn't be developing it, and paying and training people to operate it safely. That is what all other companies are in fact doing - taking a safe and responsible route, which is apparently too much to ask of the most highly valued car company in the world.
mensetmanusman 6y ago
Each year ~1.3 million people are killed on the road (worldwide). (interesting to compare to covid deaths). Giving people the option to sacrifice their lives to contribute to solving this problem is an amazing thing when you think about how many million people the capability could save over the next century. These are the risk taking explorers of the modern era.
- ryukafalz 6y ago
  It’s not just the Tesla owners who can die when things go wrong.
  - mensetmanusman 6y ago
    That’s true of most all car crashes. Also, alcohol.
    - ryukafalz 6y ago
      Yes, but we don’t call drunk drivers “the risk taking explorers of the modern era.”
  - mlindner 6y ago
    Has a Tesla on autopilot ever killed a non-involved person?
- 6gvONxR4sf7o 6y ago
  > Giving people the option to sacrifice their lives to contribute to solving this problem... Given that that's not how this is being marketed, I don't think the people "sacrificing their lives" would necessarily agree with that.
- decker 6y ago
  You're assuming those sacrifices will not have been in vain. What if Tesla fails to create a self driving system that's superior to human driving? What then?
nutanc 6y ago
Would we all want a world where every car drives itself? Yes. Are we anywhere close yet? No. Then why is Tesla calling it as Full Self Driving? Because it's marketing and more money can be made in updates. Is it wrong to call something as Full self driving when it obviously is not full self driving? Well, duh!
slaw 6y ago
Tesla shows us high quality mockup[1]. The system has all the features, but nothing really works. [1] https://en.wikipedia.org/wiki/Mockup https://en.wikipedia.org/wiki/Mockup
olefoo 6y ago
With this release Tesla has become a very attractive platform for assassins; if your target drives a Tesla it can be subverted and their death will look like an accident.
jiofih 6y ago
> An experienced human driver can drive for thousands of miles without making a serious mistake Absolute bullshit. 99% of drivers will frequently commit a series of small mistakes on an everyday <5km drive.
- Def_Os 6y ago
  Try to be more civil, please. And he said "a series mistake", presumably meaning something resulting in a crash. The statistics back him, in that case (1 in 500k miles, someone else mentioned).
  - jiofih 6y ago
    In which case Teslas incident rate is one every 5 million miles.
- dragonwriter 6y ago
  > > An experienced human driver can drive for thousands of miles without making a serious mistake > Absolute bullshit. It's not, though. > 99% of drivers will frequently commit a series of small mistakes on an everyday <5km drive. Perhaps, but the vast majority of such series of small mistakes don't add up to a serious mistake, so that's not relevant to the claim it was offered to rebut.
  - mlindner 6y ago
    Tesla statistics say there's 1 accident every ~4.5 million miles driven on autopilot and they release those statistics. https://www.tesla.com/VehicleSafetyReport https://www.tesla.com/VehicleSafetyReport
alkonaut 6y ago
Either the human should never have to intervene (can sleep) or the human should never be allowed to take their eyes of the road. Any level between that is dangerous, irresponsible, and should be banned from use on public roads. Which leads to an interesting question or two: a) why isn’t it? b) what will happen to Tesla and Uber once it’s clear that “Unsupervised driving” will remain a decade away for at least another decade?
- rsynnott 6y ago
  > why isn’t it? Reactionary regulation. Once a few high Level 2 driver assist systems drive into schoolbuses or something, expect to see that change.
deleted 6y ago
[deleted]
deleted 6y ago
[deleted]
breakingcups 6y ago
"Beta" has no place on the public road. I'm gonna be so fucking pissed if I die to Tesla's flawed risk/reward assessment, or worse yet, my kid. I bet Musk wont lose a moment's sleep over it though.
crocal 6y ago
Ok, so. I will say it again to all the people out there who think they can accumulate millions of miles of trials and use statistics to eventually declare a driverless transportation system is safe: stop. What you are doing is morally and technically wrong. You have to prove safety by design, and this design must be fail-safe. It’s not something you can refactor in software or test on users.
- StevenWaterman 6y ago
  That's a great ideal to strive for, but it's unrealistic. Driving is unsafe. People die all the time on the roads. The benefits of driverless cars mean that they will never be held to a higher standard than human drivers.
  - crocal 6y ago
    Driving is unsafe if humans drive because humans make mistakes. It should not be when machines do. Otherwise, what is the benefits of using them? Driverless cars can, must and will be held to a higher standard. But it is not a lofty ideal, it has been done before. And in the field of transportation. Trains used to crash all the time. Driverless trains don't. Why? Because fail safe systems have been added on railroads that nearly totally prevent this. Don't take my word for it, the bar for rail safety is defined in well documented standards (namely, EN 50129, EN 50126 and EN 50128). It is 10-9 hourly probability of fatal failure. You can also look up "CBTC" on the Internet. This is the reference bar for driverless. The real reason Tesla and friends don't want to go there is because fail-safe systems like a CBTC requires some form of collaboration with devices installed on the ground infrastructure, meaning Tesla should accept to become part of a system, and not "the" system. Obviously a paradigm change for the car industry. How long will the public authority require to understand that they must step in to force a systemic collaboration and standards between infrastructure and cars to ensure driverless safety, I don't know, but I think it's time to call bullshit on the concept that you can have an acceptably safe driverless car system without it.
ilaksh 6y ago
I agree that this whole experiment had been crazy, but the thing that people don't realize is that there is a vast difference in performance and safety between the original FSD beta version and the latest one from a few days later. Up to 1/3 less disengagements due to fixing low-hanging bugs.
deleted 6y ago
[deleted]
nullc 6y ago
Can we get a law that cars with the beta self driving need a big "DANGER STUDENT ROBOT DRIVER" sign on them?
decker 6y ago
Given that our legal system tends to be retroactive, and Tesla seems to have pushed all legal liability onto the drivers, our only hope is that the first victim of the FSD beta manages to sue the driver into oblivion and puts up a barrier to drivers using software that the manufacturer won't take responsibility for.

129 points by elsewhen 6y ago

What will it take to stop them? I believe it's a settled matter that their self driving is fundamentally flawed due to relying on ML and cameras instead of lidar. Will it take people dying for the government to step in? Lawsuits? I don't think our politicians are well versed enough in how this works and the level of risk letting this kind of thing happen.

sokoloff 6y ago
Is that a permanently settled matter? When humans drive, they are relying almost exclusively on visual inputs. I don’t see a reason to believe that humans are so far superior at visual processing that no self-driving based on video could best them and many reasons to think that we’re inferior in certain aspects of the task.
- mdorazio 6y ago
  1) Human eyes are superior to any camera system in a Tesla. both in resolution and dynamic range. 2) The human visual system is backed by the equivalent of strong, general AI that intuits and understands the entire world around the vehicle and all objects interacting with it. It is in theory possible for a video-only system to work, but ML is nowhere near the level of development needed for that to be a reality.
  - mlindner 6y ago
    Sure they're superior, but you don't decide whether something is flawed by whether it's superior or not. Our road systems are not designed for the full dynamic range of the human eye, which is why we drive with headlights and we have street lights on highways rather than relying on night vision for all driving. > The human visual system is backed by the equivalent of strong, general AI that intuits and understands the entire world around the vehicle and all objects interacting with it. I think the Human general AI vastly overestimates it's ability to drive and respond to conditions that suddenly appear when driving. > It is in theory possible for a video-only system to work, but ML is nowhere near the level of development needed for that to be a reality. [citation needed]
- candiddevmike 6y ago
  ML shows no signs of achieving intuition, which allows us to predict various scenarios in real time and adjust accordingly.
  - sokoloff 6y ago
    Does “instead of LIDAR” mean that LIDAR solves that in some significant way?
    - MontyCarloHall 6y ago
      Nope. I don’t think we’ll have Level 5 autonomy with safety on par with human drivers until we have general AI. At that point, whether or not that AGI had LIDAR supplementing its visual inputs becomes irrelevant.
      - ggreer 6y ago
        Doesn’t Wayno already have level 5 autonomous vehicles operating in the Phoenix area? So far they’ve had about 65,000 driverless miles[1] (and 6 million miles with safety drivers). That number should increase pretty quickly now that they’re scaling up their service. So far it seems like the main problem is that they get rear-ended by inattentive human drivers. 1. https://www.engadget.com/waymo-indepth-details-selfdriving-activities-phoenix-163000481.html https://www.engadget.com/waymo-indepth-details-selfdriving-a...
    - tsimionescu 6y ago
      With camera-based sensing, you need AI to interpret the inputs in addition to AI for deciding how to react to those inputs. With LIDAR, the job of detecting objects seems to be much simpler. Taking a relatively simple to work around example, you need AI to distinguish lens flare from an object approaching/departing.
      - mlindner 6y ago
        With LIDAR the job of detecting some objects becomes much simpler but for other objects becomes impossible.
- FartyMcFarter 6y ago
  Look at the car's display in this video: https://youtu.be/RN5Qoei7v1k?t=2281 https://youtu.be/RN5Qoei7v1k?t=2281 The "lane" in front of it is wildly moving around, so it's definitely misinterpreting the visual inputs.
  - xeromal 6y ago
    As an owner of a tesla, the rendering on the screen and what the car actually does with the inputs don't always correlate. The rendering is purely for your own viewing pleasure. Hell, there's a disconnect where if you have the new autopilot computer but the old entertainment screen computer, it didn't even have the capability to render anything worthwhile but you still had self driving
    - gamblor956 6y ago
      Given that the rendering on the screen is supposed to be showing the inputs the car thinks it is receiving, it is even more concerning to learn that what the car actually does isn't correlated with the inputs it is receiving.
      - sokoloff 6y ago
        That seems intentionally tortured logic. Why would it be a given for the rendering to show the inputs as received rather than being an entertaining caricature loosely related? Surely you don’t think the animations of the car’s battery being charged are realistic in any engineering or physical sense.
        gamblor956 6y ago
        Because Tesla says that it is a rendering of what the car thinks it is seeing, meaning that the rendering is supposed to roughly correlate with the information upon which the FSD systems are acting. In contrast, the animations of a car battery being charged don't claim to be realistic in any sense, and they deliberately attempt not to be realistic: oversized images of batteries or other standardized iconography, stylized arrows or other indicators of charging "flow", etc.
- rainyMammoth 6y ago
  >> Is that a permanently settled matter? When humans drive, they are relying almost exclusively on visual inputs. I really don't understand how so many smart people are repeating this argument over and over. it's one the most flawed arguments that was used by Elon Musk to sell his lack of ability to retrofit lidars in every single car. We also got a single brain, so why can a computer not do what we do?
- perryizgr8 6y ago
  I'm fact I'd argue the opposite is true. It's settled that the endgame of self driving will rely only on visual input, since that is what humans need.
205guy 6y ago
You could short Tesla stock. It is well proven that the free market will correct mistakes like this. /s
SheinhardtWigCo 6y ago
It will take a few serious accidents that most humans would have avoided.

rkochman 6y ago

It’s not as simple as “the driver can take over if they need to.” In many cases it takes a human too long to recognize the system doing something dangerous and to intervene. This is going to end badly.

t0mas88 6y ago
Indeed, recognizing the problem takes vital time and taking over to execute a maneuver is way harder than executing the same maneuver while you're already in control.
crabcakes010 6y ago
But think of the upside -- they could create serious value for their shareholders!

Sebb767 6y ago

> “Oh Jeeeesus” > In another video, Brandon's Tesla was making a left turn but wasn't turning sharply enough to avoid hitting a car parked on the opposite side of the cross street. "Oh Jeeeesus," Brandon said as he grabbed the steering wheel and jerked it to the left. "Oh my God," Brandon's passenger added. Maybe it's just me, but this screams clickbait and low effort. The overall article is fine, but I nearly stopped reading after this. Is this kind of writing really necessary nowadays?

Waterluvian 6y ago
I think given the high stakes of killing yourself or other people, we should be brutally unforgiving in highlighting major failures of killer technology.
- rubyron 6y ago
  “Killer Technology” - the Tesla marketing team is going to use that now.
- Sebb767 6y ago
  Fair enough. But on the other hand, if our headlines we're like this: > OOOOH MY GOOOOSH! - Data scientist REACTS to possible DATA BREACH by HAXORS ... I doubt it would help highlight the need for security. On a more general level, I personally don't think it's worth to trade reputation for attention via clickbait. But it might be the better option, I honestly don't know.
  - deft 6y ago
    In that case a computer isn't nearly killing someone. Very obviously not a fair comparison.
    - grecy 6y ago
      And in the videos shown the vehicles are not going fast enough to kill any occupants either.

arkitaip 6y ago

Terrifying scenarios are described in the article. This is obviously nothing more than (very) advanced lane assistance and absolutely requires fully observant drivers who might have to take over the wheel in the fraction of a second.

t0mas88 6y ago
Humans aren't designed to monitor something that requires no action for a long time and then suddenly requires attention, nor are we good at taking over control of something at the last moment. You're not in the "flow" at that point. The article is correct that expecting average drivers to do this without training is a high risk move. I've given flight instruction, taking over a landing 20ft off the ground is way harder than landing a plane yourself. And that's with a lot of training, not just an average driver being put in a place to supervise Tesla autopilot with no training at all.
- 1337shadow 6y ago
  > Humans aren't designed to monitor something that requires no action for a long time and then suddenly requires attention Isn't that exactly the definition of cruise control ? I'm sorry but I won't be driving a car without cruise control anymore: cruise control saves oil and is much more comfortable.
  - sibartlett 6y ago
    With cruise control you’re still very much the one who is driving and paying attention.
    - 1337shadow 6y ago
      Ok, I can't imagine myself dropping attention from driving. As a member of a national roadkill prevention awareness org and having lost close friends on the road: I would never had imagined that it would require you to drop attention. I imagined it was something you could enable and then disable by just breaking like cruise control or auto parking with toyotas. But now I see how complex it becomes because of the wheel, even tesla owners in the comments of the article don't like to let their hands off the wheel.

nostrademons 6y ago

"Move fast and break things." I really don't understand how a company can get away with using its customers as QA when they sell products that can kill people. It's one thing if it's a website for throwing virtual sheep at another; nobody really cares if your Facebook feed omits an update or two. It's another thing if a software bug means that you die. Or at least it should be - I'm amazed at the number of people who seemingly don't care. I also wonder what this is doing to Tesla's brand reputation. I was just in the market for solar panels and Tesla's offering (both the panels and the solar roof) was very, very attractive on price & aesthetics, but their reliability record with their cars made me think twice about buying a 25+ year product that keeps my home powered and dry.

ericmay 6y ago
Hate to say it but I think you’re in the minority with that opinion. Reaction from most people has been “woah”. Also, there is no moving fast and breaking things here in real terms. Drivers are and have to pay attention while driving, and Tesla is strictly monitoring this with beta. You also specifically have to opt-in to beta software. 99% of Tesla owners do not have this functionality.
- dontcarethrow2 6y ago
  I think we are mainly silenced by the hype folks who can't not tell the rest of the internet about their excitement. I'd like to think OP's opinion is the majority, hard to tell on a startup forum where most folks are hodling TSLA.
  - Smaug123 6y ago
    I would be astonished if a majority of people on HN have Tesla stock, for what it's worth.
    - OJFord 6y ago
      Would you only be astonished if they directly hold it, or also if it's via a fund or three? (I think you're probably right they don't, HN international and diverse enough, but I wouldn't be astonished.)

cblconfederate 6y ago

Those are going to be good arguments for the gun lobby people.

speedgoose 6y ago
I don't get it, why?

FpUser 6y ago

>"I really don't understand how a company can get away with using its customers as QA when they sell products that can kill people." Since when did this ever stop companies when there are money to be made. Driving cars in general kills tens of thousands of people per year and so do many other activities. For governments it is just a question of how much money vs number of people killed.

NewEntryHN 6y ago

Is the trick here that the driver is responsible in case of accident? That would be evil genius.

deleted 6y ago
[deleted]

cmckn 6y ago

I think I get less interested in self-driving cars the closer we seem to get to them. It honestly just stresses me out. Driving is something I enjoy doing. I’m not sure I will ever feel comfortable taking my hands off the wheel. This has little to do with how well the tech works in tests or anecdotally; I think I just don’t want this.

agumonkey 6y ago
For dense urban areas (low speed, lots of stopping) I could see waymo tiny wheelhubs working. A tiny taxi bubble.
- cmckn 6y ago
  I agree. I think I’d feel differently if the vehicles were more like trains on a track, if the roads had some added features that guided the vehicle. I just don’t think I’m down with the autonomous approach where the car is trying to make all the decisions I do.
toast0 6y ago
I'm a staunch grumpy old man, but our minivan has lane keeping, automated emergency braking and radar cruise control. The lane keeping and emergency braking are just there and mostly help out (although, apparently I like to drive on the line when exiting a freeway, so it's kind of annoying there). Radar cruise would be nice, except I don't do much long distance driving, and I don't like the following distance options, a different car might give different options that would be better. I wouldn't like to be in a car that drives like either the Tesla or the Waymo cars though, the Tesla's are too aggresive, and the Waymo cars are not aggressive enough (which is generally good and safety aware, but would still be frustrating to ride in).
rsynnott 6y ago
> I think I get less interested in self-driving cars the closer we seem to get to them. It honestly just stresses me out. That's probably a reasonable response. These things have the nasty feature that they both require less input from the driver the more sophisticated they get, but also that they require more _attention_ from the driver the more sophisticated they get (a boring old cruise control system won't abruptly swerve into a concrete road divider, but a high Level 2 system will). And humans just don't work like this. "Don't do anything, but be ready to take control within a second or so when it does something insane" is not something we're good at.

kryptiskt 6y ago

Bet it's only named "Full Self-Driving" to get out of their obligations to customers who bought that package.

scollet 6y ago
You can drive yourself anywhere!

helsinkiandrew 6y ago

When Elon Musk said real world testing was needed, I assumed it would be Tesla test cars with trained ‘drivers’ not beta testing guineapigs. Given that there’s regular stories of people playing phone games and crashing whilst using the current lane assist tech, are we safe? ‘Real-world testing was needed to uncover what would be a "long tail" of problems, he added.’ https://www.bbc.com/news/technology-53349313 https://www.bbc.com/news/technology-53349313

rjkennedy98 6y ago
Its definitely going to be real world drivers who opt in at their own risk. My understanding is that Tesla views its large number of cars as a test ground to get to level 5 autonomy. It will slowly roll out more and more autonomous functionality and incrementally get there. You simply cannot solve the tail end of issues with test driving alone.
- rrss 6y ago
  > drivers who opt in at their own risk If it was actually just at their own risk, it would be substantially less bad. They opt in at their own risk, and at the risk of those around them.
  - nullc 6y ago
    I learned that Teslas had some self driving functionality by seeing obviously inattentive drivers in them. While on the road I try to stay as far away from any tesla as I can.
justapassenger 6y ago
> When Elon Musk said real world testing was needed, I assumed it would be Tesla test cars with trained ‘drivers’ not beta testing guineapigs. Cynical view - it's being tested by twitter users who are on good terms with a company, with an explicit goal of generating more hype for Tesla.

Strilanc 6y ago

Tesla should follow Waymo's lead and release statistics on collisions while self-driving, and also the number of simulated counterfactual collisions avoided by the human taking over [1]. Based on the multiple incidents from the individual drivers cited in the article, it sounds like the numbers wouldn't be very good. 1: https://storage.googleapis.com/sdc-prod/v1/safety-report/Waymo-Public-Road-Safety-Performance-Data.pdf#page=5 https://storage.googleapis.com/sdc-prod/v1/safety-report/Way... Disclaimer: I work for Google but am not involved with Waymo. Opinions my own.

mlindner 6y ago
> Tesla should follow Waymo's lead and release statistics on collisions while self-driving Tesla does rely statistics on collisions while self-driving. https://www.tesla.com/VehicleSafetyReport https://www.tesla.com/VehicleSafetyReport > In the 3rd quarter [of 2020], we registered one accident for every 4.59 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot but with our active safety features, we registered one accident for every 2.42 million miles driven. For those driving without Autopilot and without our active safety features, we registered one accident for every 1.79 million miles driven.
- Strilanc 6y ago
  This is good, I had forgotten they released that data, but I was still asking for a lot more. For example, Waymo bucketed accidents by severity and went down to very low severity (like a pedestrian bumping into the side of a stopped car) whereas on that site Tesla is essentially only reporting airbag deployment counts. (To Tesla's credit, they have more time granularity than Waymo.) I do think the counterfactual "how many times did the human driver taking over prevent a crash" data is really important to have. In the article, the scary near misses they talked about fell into that category rather than into the "actual airbag deployment" category.
  - mlindner 6y ago
    > I do think the counterfactual "how many times did the human driver taking over prevent a crash" data is really important to have. In the article, the scary near misses they talked about fell into that category rather than into the "actual airbag deployment" category. It would be nice to have however I think it misrepresents. If the end result of having Autopilot on is that the driver, on average, crashes less then it's an improvement, even if the driver has to regularly intervene. What's important is the end effect, not how it happens.

kthejoker2 6y ago

Without commenting on this particular technological advance, it did make me think of the early days of the automobile https://amp.detroitnews.com/amp/26312107 https://amp.detroitnews.com/amp/26312107 My key takeaway is regulation of advanced technology lags waaaaaaay behind the disruption caused by the technology.

1337shadow 6y ago
Is it really possible to regulate without any experience at all ? I thought regulation was something that you could define based on a sufficiently widespread practice.
- timmytokyo 6y ago
  I can guarantee you that if Tesla's auto-driving starts killing people, there will be near-instant regulation. For that reason alone, supporters of auto-driving technology should be aghast at what Tesla is doing.
  - Espressosaurus 6y ago
    It has already killed three people. The question is how many more will it kill before real regulation starts hitting? Waymo has been conservative, which is the right way to go about this. The number of outright faulty actions I'm seeing on these videos is terrifying. Pedestrians and other cars on the road didn't ask to be part of this live fire experiment.
- Pfhreak 6y ago
  >Is it really possible to regulate without any experience at all ? Yes. You can absolutely set safety metrics and boundaries. They might not be perfect, but as you get experience you refine those boundaries. Just like settings alarms in web services -- you have a sense of roughly where the performance should be, and you set some alarms. You iterate on those alarms as you get data, but you still absolutely have a sense of what's roughly appropriate from the start.

scared2 6y ago

The main problem is that Tesla autopilot uses pytorch which is a prototype and toying library designed by an intern as compared to tensor flow which was designed from ground up targeting realistic applications such as this one.

aeternum 6y ago
Comma.ai recently switched their entire codebase from tensor flow to pytorch, how do you explain that?
orbifold 6y ago
Completely irrelevant since the model is run on a custom chip. So no part of the pytorch code actually ever makes it into production.
MontyCarloHall 6y ago
I'd be very curious to hear some elaboration on this claim; it's the first time I've ever seen it. From my point-of-view, the fact that TensorFlow completely scrapped its entire ≤v1.0 graph-based API for a PyTorch-style eager execution API for v2.0 onwards does not suggest that it was “designed from the ground up” in any way, shape, or form. We're talking a total rewrite of the entire frontend codebase going from v1.0 to v2.0, and a total abandonment of all v1.0 style code. TF 2.0 is basically a brand new language. Under the hood it's all the same BLAS/LAPACK operations anyway. I've found TF superior to PyTorch for stats methods and probabilistic programming in general (far richer API), but for standard linear algebra the two are equivalent performancewise, with PyTorch having the superior API IMHO.
- scared2 6y ago
  Just being easy to use doesn't mean it has a solid back end. I agree pytorch is easier to comprehend UI for average person.

1337shadow 6y ago

I guess that's why it's still beta ! Kudos to the driver ++

WhompingWindows 6y ago

Tesla is very "thrifty" with their FSD software. Other companies need to hire safety drivers, but Tesla just foists that labor onto their paying customers, and any liability to boot.

mlindner 6y ago
Who very willingly enable the option after several warnings that the software is beta. In fact tons of people are begging to get access.

num3ric 6y ago

The line selection for turns seems to be quite poor still.

deleted 6y ago

[deleted]

Zigurd 6y ago

If you don't qualify what you mean by "AVs will kill people" it's not useful. People can, and do, accidentally and intentionally, step in front of vehicles. They also intentionally crash vehicles. They won't stop because of AVs. Even the fact of suicide by vehicle is complex: AVs can prevent some suicide by driver but possibly raise the risk of pedestrians committing suicide by vehicle. One reason suicide by vehicle happens is because it can conceal the intent to commit suicide. This is compounded by suicide being a taboo subject. This alone will complicate and possibly delay adoption of AVs. Millions of innocent people are injured every year and about a million die every year in road accidents. Cutting that number significantly would have roughly the same impact on global well-being as curing a dread disease. There are potentially hundreds of millions of new customers for vehicles as developing nations grow their middle class. If humans drive those vehicles, hundreds of thousands more will die every year.

pmcollins 6y ago

You don't have to read Tim Lee's writing to know how the latest FSD performs. There are lots of videos on YouTube, and new ones posted every day. My personal opinion is that this is a quantum leap over the previous implementation. Much respect to the engineers at Tesla for this amazing accomplishment. https://www.youtube.com/results?search_query=tesla+fsd+beta https://www.youtube.com/results?search_query=tesla+fsd+beta

FartyMcFarter 6y ago

This thing is obviously not ready to drive properly: https://youtu.be/RN5Qoei7v1k?t=2218 https://youtu.be/RN5Qoei7v1k?t=2218 It can't even figure out which traffic lights apply to it.

cma 6y ago
Wow, that's like 10 disengagement's in just a couple minutes and it almost drives through a train boom barrier and does some scary stuff with a pedestrian...
- FartyMcFarter 6y ago
  I missed that on the first watch but you're right, the driver engages the self-driving mode in front of the train barrier and it immediately accelerates towards it. This system is really bad.
tsss 6y ago
Unbelievable that the guy has to swipe and type on the center console while driving. Touch screens in cars should be banned and Tesla held liable for all accidents that happen while interacting with the console in existing vehicles.
mlindner 6y ago
The guy disengaged it just as it started accelerating. It's sometimes slow to respond to green lights right now. Give it a few more weeks and it'll be fine. Look at how it responds to pedestrians here: https://youtu.be/RN5Qoei7v1k?t=587 https://youtu.be/RN5Qoei7v1k?t=587 And his quote: > Improvements across the board, turned into Target and took me to the front! Drove around East Sac, through Sacramento State, Midtown, and Downtown. This release definitely was far less interventions for me. Also this guy in general gives a lot more vocal reaction than I would personally give in similar situation so it makes it look and feel worse than it actually is.

_greim_ 6y ago

By now it should be obvious—and this is purely descriptive of what I see playing out, not prescriptive—that self-driving tech will have to clear a higher bar than just statistically better in order to become mainstream. Not only because anecdotes carry more weight than statistics in the media—as this story illustrates—but because people are much less afraid of bad things happening, as long as they're in control when it happens. Also, I suspect everything associated with fuel-burning cars is by now irrevocably linked with cultural ideals of individualism and freedom. Resistance to EVs and self-driving tech will continually materialize, as if out of thin air, regardless of how well documented the benefits are. It's going to be an uphill battle for the foreseeable future.

sidibe 6y ago
But also there is no way Tesla FSD is anywhere close to being statistically better than normal drivers considering what we see in the videos. The reason I make this obvious statement is you are making it seem like the article is using anecdotes while there are statistics somewhere that show safety
gamblor956 6y ago
Anecdotes are data. They're a different kind of data, that usually leads to investigations resulting in more rigorously gathered data. Multiple drivers are reporting multiple incidents of varying severity in the short (week?) that the "full" FSD beta has been available. That's a huge concern, and it absolutely needs to be followed up on.
- mlindner 6y ago
  The software isn't widely available. It's available to a tiny set of Tesla's "closed beta" customers. Tesla picks and chooses who it sends it to right now. Basically it's only being sent to Tesla large stock owners or people who Tesla has worked with in the past who are non-employees right now.
gfodor 6y ago
I'm not so sure. It seems equally if not more likely to me that people will eventually take a ride in a self driving car, be impressed since that's the statistically likely outcome, and slowly get used to it. Accidents from robocars will become things that happen to "other people" like all things that ultimately boil down to random chance.

Animats 6y ago

From the article: "These YouTube videos underscored how important it is for drivers to actively supervise Tesla's new software. Over the course of three hours, the drivers took control more than a dozen times, including at least two cases when the car seemed to be on the verge of crashing into another vehicle." In a way, it's good that the Tesla system sucks so badly. If they had a disconnect rate of one per month, drivers would trust the thing. The other guys, disengagements per 1,000 self-driven miles in California: Waymo, 0.076, or one per 13,000 miles. Cruise, 0.082, or one per 12,000 miles. Not clear how many of those would have resulted in a crash, as opposed to just stopping. Probably not many, the California autonomous DMV reports indicate. US humans have a crash rate of one per 508,000 miles driven. Anyway, Tesla does not have "full self driving". They have slightly better Level 2. Not even Level 3. You'd think that by now they'd have hands-off freeway driving totally automated to a better than human level of safety. But no.

banana_giraffe 6y ago
Makes me wonder: I have two driving modes, not autopilot, but not as aware as I could be, and then tracking everything I can because something odd is going on and I'm trying to avoid a bad scenario. Perhaps I'm a bad driver, but I suspect this is normal, kind of in a low-attention mode most of the time, but if the car a lane over starts to change lanes too close, you go into high-alert mode. I wonder if there are studies on how often humans need to do this sort of "disengagement" for lack of a better term? Not sure how you'd measure it, perhaps with more eye movement than normal? Dunno.
supernova87a 6y ago
You really shouldn't trust those disengagement statistics much at all. They are totally up to the individual companies what counts as "disengagement", how to define it, and how they report them.
modeless 6y ago
> Not clear how many of those would have resulted in a crash Waymo just published an extremely detailed paper explaining exactly how many of their disengagements would have resulted in a crash, among other things. It's very interesting reading and shows how far Tesla has to go. Summary: https://www.theverge.com/2020/10/30/21538999/waymo-self-driving-car-data-miles-crashes-phoenix-google https://www.theverge.com/2020/10/30/21538999/waymo-self-driv... The paper: https://storage.googleapis.com/sdc-prod/v1/safety-report/Waymo-Public-Road-Safety-Performance-Data.pdf https://storage.googleapis.com/sdc-prod/v1/safety-report/Way...

6gvONxR4sf7o 6y ago

"Full self-driving" > Tesla ... says it's not intended for fully autonomous operation. Drivers are expected to keep their eyes on the road and hands on the wheel at all times. If tesla rolls out a product where you have to pay close attention and be ready to take over and calls it full self driving, that seems like a massive case of fraud. I'd be pissed if I'd bought the FSD package.

giancarlostoro 6y ago
On the other hand I am okay with that requirement but it becomes as diluted as the term smartphone / smarttv / smart cars.... Dont call it self driving call it smart cruise control or something?
p1esk 6y ago
It’s a beta release
mlindner 6y ago
1. It's not even available to 99% of people right now. 2. It's a beta. 3. People bought into it early when the price was lower. It's a subscription to the feature set as it develops, not a guarantee that it works perfectly the day you pay for it.

gfodor 6y ago

Without releasing data, Tesla is forcing analysts to focus on anecdotes, which suffer from all the usual problems like selection bias. What we need to know: - How often do drivers intervene? - How often do drivers fail to intervene and the system causes an accident? - How often does the system take an action which has a high likelihood of having prevented an accident? - How often does the system do so when it seems likely a human driver would have failed to do so? - Integrated together, what's the expected dynamics of these probabilities and their net impact insofar as the system being releases more widely creates training data to help improve them more quickly over time? It could very well turn out to be the case that this system is purely positive, strongly net positive, or neutral in harm reduction. The question then is, given that, what ethics should inform its release: is putting the stress on the driver sufficient if it say, saves 1000 lives in the next six months and will harm noone in exchange, other than some drivers having to intervene and have moments of stress? What if it will save 1000 and 10 people will be harmed by failing to intervene? What if it's an even swap, harming people who fail to intervene in exchange for saving people who would have inevitably been lost to accidents they couldn't have prevented? Saying "this is wrong to do" based upon anecdotes is dumb analysis. Saying "this is wrong to do" based upon an absolutist form of ethics is fair, but it also means you reject that we should be trying to solve self driving. If you think it's wrong to do and are not doing either, you ought to articulate what scenario in terms of data would justify the action vs not, even if that data is unknown right now.

kanox 6y ago
Tesla released data for autopilot, FSD beta has been out for ~1 week so there's not much real data.
- gfodor 6y ago
  Yeah, I'm not arguing that Tesla should have done something differently so much that I'm encouraging them to make sure they get data out ASAP.
tsimionescu 6y ago
> It could very well turn out to be the case that this system is purely positive, strongly net positive, or neutral in harm reduction It could also turn out to be negative, right? Certainly the anecdotes shown here would point much more to that - for example, its inability to turn sharply enough is indicative of some deep flaws. > If you think it's wrong to do and are not doing either, you ought to articulate what scenario in terms of data would justify the action vs not, even if that data is unknown right now. It's wrong to do as long as you rely on CUSTOMERS to ensure safety. There is no further argument to be had: as long as the system is not PROVEN to be safer than a human driver even in the absence of human attention, it can't be put into the hands of customers. That is not to say that they shouldn't be developing it, and paying and training people to operate it safely. That is what all other companies are in fact doing - taking a safe and responsible route, which is apparently too much to ask of the most highly valued car company in the world.

mensetmanusman 6y ago

Each year ~1.3 million people are killed on the road (worldwide). (interesting to compare to covid deaths). Giving people the option to sacrifice their lives to contribute to solving this problem is an amazing thing when you think about how many million people the capability could save over the next century. These are the risk taking explorers of the modern era.

ryukafalz 6y ago
It’s not just the Tesla owners who can die when things go wrong.
- mensetmanusman 6y ago
  That’s true of most all car crashes. Also, alcohol.
  - ryukafalz 6y ago
    Yes, but we don’t call drunk drivers “the risk taking explorers of the modern era.”
- mlindner 6y ago
  Has a Tesla on autopilot ever killed a non-involved person?
6gvONxR4sf7o 6y ago
> Giving people the option to sacrifice their lives to contribute to solving this problem... Given that that's not how this is being marketed, I don't think the people "sacrificing their lives" would necessarily agree with that.
decker 6y ago
You're assuming those sacrifices will not have been in vain. What if Tesla fails to create a self driving system that's superior to human driving? What then?

nutanc 6y ago

Would we all want a world where every car drives itself? Yes. Are we anywhere close yet? No. Then why is Tesla calling it as Full Self Driving? Because it's marketing and more money can be made in updates. Is it wrong to call something as Full self driving when it obviously is not full self driving? Well, duh!

slaw 6y ago

Tesla shows us high quality mockup[1]. The system has all the features, but nothing really works. [1] https://en.wikipedia.org/wiki/Mockup https://en.wikipedia.org/wiki/Mockup

olefoo 6y ago

With this release Tesla has become a very attractive platform for assassins; if your target drives a Tesla it can be subverted and their death will look like an accident.

jiofih 6y ago

> An experienced human driver can drive for thousands of miles without making a serious mistake Absolute bullshit. 99% of drivers will frequently commit a series of small mistakes on an everyday <5km drive.

Def_Os 6y ago
Try to be more civil, please. And he said "a series mistake", presumably meaning something resulting in a crash. The statistics back him, in that case (1 in 500k miles, someone else mentioned).
- jiofih 6y ago
  In which case Teslas incident rate is one every 5 million miles.
dragonwriter 6y ago
> > An experienced human driver can drive for thousands of miles without making a serious mistake > Absolute bullshit. It's not, though. > 99% of drivers will frequently commit a series of small mistakes on an everyday <5km drive. Perhaps, but the vast majority of such series of small mistakes don't add up to a serious mistake, so that's not relevant to the claim it was offered to rebut.
- mlindner 6y ago
  Tesla statistics say there's 1 accident every ~4.5 million miles driven on autopilot and they release those statistics. https://www.tesla.com/VehicleSafetyReport https://www.tesla.com/VehicleSafetyReport

alkonaut 6y ago

Either the human should never have to intervene (can sleep) or the human should never be allowed to take their eyes of the road. Any level between that is dangerous, irresponsible, and should be banned from use on public roads. Which leads to an interesting question or two: a) why isn’t it? b) what will happen to Tesla and Uber once it’s clear that “Unsupervised driving” will remain a decade away for at least another decade?

rsynnott 6y ago
> why isn’t it? Reactionary regulation. Once a few high Level 2 driver assist systems drive into schoolbuses or something, expect to see that change.

deleted 6y ago

[deleted]

deleted 6y ago

[deleted]

breakingcups 6y ago

"Beta" has no place on the public road. I'm gonna be so fucking pissed if I die to Tesla's flawed risk/reward assessment, or worse yet, my kid. I bet Musk wont lose a moment's sleep over it though.

crocal 6y ago

Ok, so. I will say it again to all the people out there who think they can accumulate millions of miles of trials and use statistics to eventually declare a driverless transportation system is safe: stop. What you are doing is morally and technically wrong. You have to prove safety by design, and this design must be fail-safe. It’s not something you can refactor in software or test on users.

StevenWaterman 6y ago
That's a great ideal to strive for, but it's unrealistic. Driving is unsafe. People die all the time on the roads. The benefits of driverless cars mean that they will never be held to a higher standard than human drivers.
- crocal 6y ago
  Driving is unsafe if humans drive because humans make mistakes. It should not be when machines do. Otherwise, what is the benefits of using them? Driverless cars can, must and will be held to a higher standard. But it is not a lofty ideal, it has been done before. And in the field of transportation. Trains used to crash all the time. Driverless trains don't. Why? Because fail safe systems have been added on railroads that nearly totally prevent this. Don't take my word for it, the bar for rail safety is defined in well documented standards (namely, EN 50129, EN 50126 and EN 50128). It is 10-9 hourly probability of fatal failure. You can also look up "CBTC" on the Internet. This is the reference bar for driverless. The real reason Tesla and friends don't want to go there is because fail-safe systems like a CBTC requires some form of collaboration with devices installed on the ground infrastructure, meaning Tesla should accept to become part of a system, and not "the" system. Obviously a paradigm change for the car industry. How long will the public authority require to understand that they must step in to force a systemic collaboration and standards between infrastructure and cars to ensure driverless safety, I don't know, but I think it's time to call bullshit on the concept that you can have an acceptably safe driverless car system without it.

ilaksh 6y ago

I agree that this whole experiment had been crazy, but the thing that people don't realize is that there is a vast difference in performance and safety between the original FSD beta version and the latest one from a few days later. Up to 1/3 less disengagements due to fixing low-hanging bugs.

deleted 6y ago

[deleted]

nullc 6y ago

Can we get a law that cars with the beta self driving need a big "DANGER STUDENT ROBOT DRIVER" sign on them?

decker 6y ago

Given that our legal system tends to be retroactive, and Tesla seems to have pushed all legal liability onto the drivers, our only hope is that the first victim of the FSD beta manages to sue the driver into oblivion and puts up a barrier to drivers using software that the manufacturer won't take responsibility for.