17 ms·
I thought it was telling that Andrej immediately "reframed" the question because Lex asked the "wrong question". This is a classic evasion technique one learns
by CharlesW 4y ago
I thought it was telling that Andrej immediately "reframed" the question because Lex asked the "wrong question". This is a classic evasion technique one learns from experience and/or media training. Lex's comment immediately after was a clever and gentle dig at Andrej's response.
It seemed like all the "full cost" negatives Andrej mentioned were related to Tesla's ability to execute, and not what would actually produce better results. Tesla would have to be able to reliably procure parts, write reliable firmware, create designs and processes that won't increase unexpected assembly line stops, etc.
Regarding results, the best Andrej can do is, "In this case, we looked at using it and not using it, and the delta was not massive." In other words, the results are better, but not enough to make up for the fact that Telsa can't support additional sensors without incurring a prohibitive amount of additional risk to Tesla. Risk to passengers doesn't appear to be a consideration.
- latchkey 4y agoNot just risk to passengers, risk to any thing in proximity to the vehicle while it is in motion.
- 3apo 4y agoI feel the other reason is that Tesla has not figured out a way to put Radar into their ML pipeline. If you take the Range-Doppler Map from the radar as the 'pixel' map, that data is inherently very dependent on the scenario and the radar sensor intrinsic parameters. This variability in what the radar sees in the RD space is what makes this a challenge for ML/AI pipelines. If Tesla were to 'fuse' information from these sensors in the object track level - I believe they will be less susceptible to this variability.
- dreamcompiler 4y agoExactly. Radar gives you direct range data; camera pixels need to be processed by ML to infer range data, and the latter is never going to be as close to ground truth as the former, so the former should be prioritized.
- Nomentatus 4y agoNot quite. Light waves are so short you'll get some return from almost any surface, because the surface is rough at the scale of such a small wavelength. This isn't true of radar, and it's not just what substance the outbound radar hits but how flat it is, too. You may get no return. Or almost none. Even smooth, round steel posts give very little return IIRC. There's also an echo problem with long waves such as sound and radar, particularly in urban areas. In which case what you think is a firm direct return may be a very indirect return that happens to be in synch with the signal you were expecting.
- touch_abs 4y agoIts interesting, that kind of object level fusion is a fairly different problem to training visual perception, following some of the less in fashion robotics techniques. I wonder if its a case of the Tesla engineers focusing on the fad technologies (or just their strengths) more than its a hardware cost thing.
- threeseed 4y agoIt was an ominous answer. We really should be focusing on what is the best solution and trying to solve price issues through existing techniques e.g. economies of scale, competition, miniaturisation. Instead they are trying to build whatever solution they can that fits in a pre-defined cost window. Except this isn't a new phone or sneakers we are trying to take to market it's something that will directly impact people's lives.
- Nomentatus 4y agoYou can get to this conclusion if you're sure Andrej is lying, and that the risks cited are smoke; but only then. BTW, I've upgraded my sneakers after a couple falls on a rough beach with tangled driftwood (drift trees, really) proved their cheap too-slick surface had real world consequences. I was lucky not to break a bone. I'm going to bet he isn't lying, but I can understand someone making the opposite bet, market competition being market competition.
- cbsmith 4y agoI don't think he's lying. I think he's looking at things from a particular perspective, and that perspective is being critiqued, as it should be.
- Nomentatus 4y agoTrue, he could be wildly misled but he's been around doing this for a while, so that seems unlikely. He could be truly delusional in either case but it's still kinda necessary to knock down his arguments or logic; and that's the critique or analysis I'm not seeing. Just assertions again and again that the real reason is economics triumphing over safety. I'm beginning to think that the idea that there are genuine trade-offs in life is just ungraspable or offensive to many.
- cbsmith 4y agoThere's a lack of evidence either way, which really should tell you all you need to know. I don't think they're delusional, but they are constrained by their context. Yes, with ML models, you often can be better off trimming down your sensor data. Usually though, you don't remove entire categories of sensor data. Even when you do, to be confident in such a move, you need to first achieve a working model with whatever data you have, and then through refinement you can prove that whole categories of data are more hinderance than help. They haven't done that. It seems quite clear that the reality is a full sensor array is just economically non-viable with their business model, and that's framing their whole thought process. Because they promised FSD back in 2017, they can't acknowledge that going without those sensors means it's going to take them much longer to achieve FSD. Because of safety/regulatory oversight, they also can't acknowledge that going without those sensors means there will be additional safety risk. So they're stuck making these rationalizations that everyone in the industry knows are at best half-truths. No doubt, at some point we'll figure out how to do self driving with a much more limited sensor package, and when we do, we'll achieve a significant improvement in the cost effectiveness of self-driving. In the meantime, there's a lot of "rationalization" going on.
- Nomentatus 4y agoAndrej did get around to answering the original question, he just wanted to say more, to put it into a bigger frame with more context. I had the same "weasling" concern at first; but his answer was more or less "You lose more than you gain, but yes there was a small delta; in exchange for which any organization would take on not just an economic hit but a lot of additional opportunities for process and maintenance errors; plus distracting the team." So he'll agree that in an ideal world you'd want 'em, just not want 'em that much; but in the real world, more geegaws that aren't really pulling their weight are a terrible idea. Although he didn't explicitly say so, neither his answer nor Elon's "take it out 'cause you can always put it back in if it turns out you really need it" philosophy absolutely rule out lidar coming back in the future if some remaining edge case just requires it. Clearly he thinks this is quite unlikely, however.
- flashgordon 4y agoThat was hilarious. Basically (unless this needs a reframing/realignment/repositioning/reorienting): Q: "are less sensors less safe/effective?" A: "well more sensors are costly to the organization and add more tech debt so safety is orthogonal and not worth answering".
- Retric 4y agoThe video has a more reasonable answer. The sensors are unreliable and expensive in terms of R&D. Having marginal parts which takes money from a finite R&D budget can easily result in a worse product. “They contribute noise and entropy into everything.” … “you’re investing fully into that [vision] and you can make that extremely good. You only have a finite amount of spend of focus across different facets of the system.” His standpoint can be summed up as “I think some of the other companies are going to drop it.” Which would be really interesting if true.
- a1369209993 4y ago> Having marginal parts which takes money from a finite R&D budget can easily result in a worse product. "Less sensors can be more safe/effective if that allows us to focus on making effective use of the sensor information we do have, which is the result we're aiming for with this descision." would be a reasonable answer (if true), but that doesn't seem like a fair interpretation of what he actually said.
- Retric 4y agoIt’s fairly rambling but he touches on that exact point several times most specifically here at the 2 minute mark: “Organizationally it can be very distracting. If all you want to get to work is vision resources are on it and you’re actually making forward progress. That is the sensor with the most bandwidth the most constraints and you’re investing fully into that and you can make that extremely good. You only have a finite amount of spend of focus across different facets of the system.” Which was from this section: Q: “Is it more bloat in the data engine?” “100%” (Q:“is it a distraction?”) “These sensors can change over time.” “Suddenly you need to worry about it. And they will have different distributions. They contribute noise and entropy into everything and they bloat stuff”. Even earlier he says: “These sensors aren’t free…” list of reasons including “you have to fuse them into the system in some way. So that like bloats the organization” “The cost is high and you’re not particularly seeing it if your just a computer vision engineer and I am just trying to improve my network.”
- quonn 4y agoI think the key point he‘s trying to make is that the size of the fleet is more important than the quality of the sensor. The risk would be reduced by a better system and he seems to be convinced that rolling out vision to more and cheaper cars would get you there.
- FreakLegion 4y ago> In other words, the results are better, but not enough to make up for the fact that Telsa can't support additional sensors without incurring a prohibitive amount of additional risk to Tesla. Risk to passengers doesn't appear to be a consideration. You may be right about the actual decision process Tesla went through, but Karpathy is right in principle. One of the first things he says is "there can be problems with [the sensors]", and a lot of what he mentions increases the risk of run-time failure, not just cost. It's easy to cast this as an optimization problem where you're trading off asymptotically improved sensing for linearly or superlinearly increased failure rates. There's certainly a point where the complexity of more sensors or certain types of sensors outweighs any marginal benefit they provide.
- bumby 4y agoThere are other ways of optimizing for reliability, though, like redundancy in parallel or higher spec’d sensors. But that still gets back to the same issue where they are going to be concerned about cost.
- YZF 4y agoTaking his point to the extreme why use 8 cameras? just use 4? 1? One photo-diode? Cameras can also fail at run-time there can (and is) be variability in how they're mounted, in the lenses, in the sensors. They can get blinded or not get enough light. Their cabling can fail random components can fail. Tesla has claimed that vision outperforms vision+radar but anecdotal reports don't seem to support that conclusion. IMHO these technologies are not directly replaceable, but are complementary. It's like you can't replace your ears with your eyes (yeah, you can read lips, if they're visible). But sure, there is a sweet spot. Is Tesla really optimizing for best performance at any cost or are they optimizing making more money and selling that to us as an improvement? That's really the question and I don't think we got a frank answer there.
- dmix 4y agoYou could take any one single point in a complex multifaceted argument to the extreme and basically strawman it to death. But that’s not helpful. I believe his point was to provide a new perspective on the problem, not to reduce the problem to a single reason. I highly doubt the only reason Tesla chose to use vision only in the short term was motivated by a single datapoint. Even if it was the most important point... in this one person (on a large team’s) mind... it doesn’t necessarily mean it was the most important in the sum of the complex process it took to get to the decision. So I don’t really see the value in taking it to the logical maximum because it’s not only illogical that they would be evaluating this one idea in isolation but even on its own they would still be balancing the optimal performance they got from x vs the optimized value they got from y, then compare it to the teams ability to work with both x+y(+z) at the same time. For ex: You’d probably need 8 cameras pointing different directions vs one highly capable rapidly spinning LiDAR to even compete with it, so why even ask? These problems a) always have context and b) can't be so easily simplified and broken down. Although you might make a good point that Tesla used this same poor logical-maximum reasoning to determine why not get rid of ALL sensors besides vision.
- patrec 4y ago> I thought it was telling that Andrej immediately "reframed" the question because Lex asked the "wrong question". This is a classic evasion technique I agree with this assessment. However: > Telsa can't support additional sensors without incurring a prohibitive amount of additional risk to Tesla. Risk to passengers doesn't appear to be a consideration. This is a stupidifying take. Of course when you work in a line of business producing gadgets that, as an unintended side-effect, kill a lot of people (napkin math suggests above 2 milli-kills per car in the US), you will need to pick a point at which you say further fatality reduction is no longer justified given the economic cost of achieving it. Even if you are a pure altruist (if you go out of business, less safe cars will replace yours). Conversely, even if you are the embodiment of capitalist evil, risks to passengers will absolutely affect your bottom line and if are rational you will take them into consideration. Any meaningful criticism needs to be about the trade-offs they make, not that they make them or are loath to explicitly say so on camera.
- deleted 4y ago[deleted]
- CharlesW 4y ago> …you will need to pick a point at which you say further fatality reduction is no longer justified given the economic cost of achieving it. You're right — the sad truth is that corporations put costs on human lives every day. Where I think we disagree is that you believe they made the decision based primarily on costs. After watching this video, I believe they made the decision because they didn't think they could reliably implement and support a sensor fusion approach. (BTW, I enjoyed "stupidifying"! I'm sorry I made people stupider.)
- ClumsyPilot 4y ago> risks to passengers will absolutely affect your bottom line and if are rational you will take them into consideration percieved, not real risks to customers. PR matters more than reality
- patrec 4y agoIt is kind of hard to control perceived risks to passengers without taking actual risks to them into account.
- ion_fury 4y agolex's comment did not strike me as a dig. i am actually concerned by your comment because it makes me wonder if i am missing other things too? it just doesnt seem like a dig. it seems like he thought of something funny and wanted to share it. am i alone in this? and also i dont understand your assertion that it was some kind of cynical maneuver to re-frame the question. he could have also said "yes, more sensors are always better but you can add an arbitrary number of sensors and so we had to decide where to draw the line. the cameras we use are capable of meeting our goal of full self driving that is significantly safer than a human driver. and this also streamlines the production and software which has a material impact on our ability to actually produce the cars which is of course necessary to meet the goal of making self driving cars. bloat could actually kill tesla." this is logically the same thing that he said in the interview, so whats cynical about it? how is it underhanded? also is there some intrinsic limitation of the dynamic range of cameras? people are talking about problems with dynamic range being intrinsic to cameras but im pretty sure that cameras and especially camera suites that do not have more problems with dynamic range than a human eye are possible to make and probably already on the market.
- JumpCrisscross 4y ago> did not strike me as a dig It wasn't a dig. It was calling out a bullshit move that, in my opinion, Andrej deployed out of panic more than strategically. (My evidence for this being Andrej eventually gave a good answer.)
- ion_fury 4y agoi just dont get it. lex says "lets re-frame the question: can a language model drive a car?" this doesnt have any insinuations about andrej's intentions or motives. if it were calling out bullshit it would be "lets re-frame the question: why is elon musk never wrong?" lex kisses elon musks ass, last time i checked hes on musks side in the lidar debate and also lex has a record of listening patiently no matter what and i have never known him to check people or "call out their bullshit." lastly, what andrej did wasnt a bullshit move. re-framing a question has never been known as something people do only when they are being deceptive and it is very common in intellectually honest answers/explanations in my experience. im still shocked because usually i see what other people see. but people calling this a dig/bullshit came out of left field for me. i hope i dont miss stuff like that more than i realize... edit: i read your comment again and i didnt read it right. he panics, gives himself some more room, lex acknowledges this but in a prickly way. it makes sense but im still upset because when i watch it all i see is lex making a joke. i guess i have severe autism.
- pmarreck 4y agoTwo thoughts: 1) He's not touching on the software cost of integrating different sensor data into the same trained machine learning model; it is likely far simpler to just stick to stereoscopic vision data (the same thing the human genome decided!) 2) That said, it seems at least theoretically advantageous to have a sensory system that exceeds that which humans are limited to; things like LIDAR can work in complete darkness and potentially spot, for example, pedestrians crossing a dark road without any reflective clothing on, where a vision-based system would fail (perhaps add infrared sensation?) Anyway, doesn't AEB (automatic emergency braking) have to be installed in every car, by law, in the US, around now? And wouldn't that be less reliable if done via vision?
- sfifs 4y ago> likely far simpler to just stick to stereoscopic vision data (the same thing the human genome decided!) Yeah and till we had reliable and powerful artificial lighting, it was highly unsafe to journey in low visibility/ darkness. We used to finish journeys when darkness fell. Animals that do require precise movement in low visibility (bats, dolphins) conditions often evolved ultrasound solutions. So should we license Tesla vehicles to only operate when visibility and weather forecast is good and not drive in the dark at all?
- jquery 4y agoExcellent points, I didn't think about the fact that even evolution couldn't come up with a vision system that works as well in the dark as it works in the daytime.
- pmarreck 4y agoWell actually, the human vision system at night, while not as good as cats and perhaps dogs, is still much greater than any camera we've come up with thus far, I read something that claimed we can actually detect individual photons hitting our retina, once we are adapted to the darkness https://www.science.org/content/article/human-eye-can-detect-single-photon https://www.science.org/content/article/human-eye-can-detect...
- snotrockets 4y agoTesla hasn't proven itself to be a capable major car manufacturer (they probably lead the minor category, at least in deliveries) in all but one: their de-prioritizing of human life.
- kotlin2 4y agoIs there hard data on how deadly they are vs. other auto manufacturers? There is definitely a narrative that the cars are dangerous, but I'd like to see that quantified.
- xodjmk 4y agoThe majority of comments surrounding stereo-camera/lidar questions have a ridiculously simplified idea of the problem. It's obviously the case that 'more sensor good, less sensor bad'. This is frankenstein's monster level technical analysis. Why don't the majority of large-brained animals have many eyes, and many different antannea appendages processing an array of diverse sensory input? You don't just automatically gain by adding lots of sensors. The signals have to be fused together and reliably cooperate and come to agreement in real time for any decision. Any sensor is only providing raw crude data. The majority of the work involved is done by processing this crude data and inferring a much more sophisticated approximation of the real environment from prior knowledge, hence using neural nets with pre-trained data. It is a good debate whether the approximation can be done better by adding more sensor input and diverting R&D & processing resources towards fusion as opposed to improving the results that can be obtained from stereo image sensor. It's not obvious to anyone. And nature seems to inform us that most large brain animals evolve to rely heavily on two eyes instead of 16 eyes + lasers. This is an interesting discussion, but the issue isn't 'tesla could just bolt a Lidar box to the roof and magic, but they want to scam you out of a few extra bucks'. That is a moronic idea.
- wonnage 4y agoThe issue here is trying to infer distance based on complex image processing or just… measuring the damn distances.
- xodjmk 4y agoNo, that is over simplification. It's not simply distances, you have to detect objects, motion, shadows, corners and all sorts of phenomena.
- JumpCrisscross 4y ago> Why don't the majority of large-brained animals have many eyes Because of the cost of additional eyes. If Tesla is optimizing for cost against safety, that's sort of the point. I don't believe that's totally the case. Andrej later makes a better argument regarding limited R&D bandwidth, noise and entropy. But the "I would almost reframe the question" evasion was disconcerting. It's a textbook media trained tactic for avoiding a question to which you have no good answer. That it was deployed here badly against a skilled interviewer such that it backfired is a valid observation.
- strangescript 4y agoYou are making a lot of potentially faulty assumptions. 1) The "delta" was wide enough to save/harm people, you have no idea. 2) The extra information provided would always be valuable and/or not be overcome with better AI models using the visual sensors in the future. 3) The amount of technical overhead generated by the extra sensors were not prohibitive long term. When working with AI there are often times where it would seem logical that extra relevant data will always improve a model, but that turns out not to always be the case, or provide so little value that managing another dataset is just not worth it.
- asah 4y agoRisk to PEDESTRIANS even less.
- steve_adams_86 4y agoThat’s one way to read it, but in my own experience the “do one thing really well” approach can yield far better results. Meaning, if vision is truly sufficient and you do it really well rather than a bunch of sensors “okay”, that may actually be safer overall. You might get far more focused and practical results from your efforts. I’m not saying this is definitely true, and at the moment we probably can’t verify it either. I’m just “steel manning” his case, as Lex loves to say. I think you’re probably correct that the business aspect was a significant factor, but perhaps it wasn’t everything.
- aaron695 4y ago
- L0stLink 4y agoThere is a great argument for having ultrasonic sensors and radar in a recent video by Rayan from FortNine discussing two fatal accidents involving tesla autopilot https://www.youtube.com/watch?v=yRdzIs4FJJg https://www.youtube.com/watch?v=yRdzIs4FJJg
- hammock 4y agoDevils advocate, if the cost of working to improve cameras to the point where they eliminate that delta is lower than the cost of using the sensors instead, then it is a net benefit
- tophi 4y agoYes, the current delta was not massive and will shrink over time. By getting rid of the extra sensors they eliminate a temporary crutch and focus resources on the simple solution. Not a new concept by the way. Henry Ford was obsessed with simplifying and eliminating every part that wasn’t necessary on the model T for virtually all the same reasons.
- 7e 4y agoWhat crutch? What simplification? These sensors are widely deployed and have already been perfected. Systems which use only one modality are the crutch. Sensor fused systems will always be safer, and are the future. This move is purely about screwing passenger safety for cost and sales.
- GuB-42 4y agoThe difference is that Ford started with something that worked. The Ford T is noteworthy because of the way it was made, not for its abilities as an automobile. Tesla is starting with something that doesn't work. No one has been able to achieve full autonomy yet, not even Waymo on its own turf, despite Waymo being well ahead of Tesla. I trust Tesla will be able to close the gap and be able to perform to the its current standards without radar and ultrasound, and it would be fine if the current standards weren't terrible in the first place. What I mean is that Tesla is currently at the awkward spot where it is good enough for cruise control, but not good enough to safely take a nap in the driver seat. As for the "simple solution", you may know the saying "For every complex problem there is an answer that is clear, simple, and wrong". I think it applies here.
- itsoktocry 4y ago>f the cost of working to improve cameras to the point where they eliminate that delta is lower than the cost of using the sensors instead In a vacuum, how can cameras ever be better than cameras + other sensors?
- adolph 4y ago> Regarding results, the best Andrej can do is, "In this case, we looked at using it and not using it, and the delta was not massive." In other words, the results are better, but not enough to make up for the fact that Telsa can't support additional sensors without incurring a prohibitive amount of additional risk to Tesla. Risk to passengers doesn't appear to be a consideration. I think this mischaracterizes Andrej's response. If anything he is referring to a wholistic view of the vehicle, which includes but doesn't entirely consist of Tesla. For example, 5-10 years down the road, when sensors start going bad, consumers will appreciate fewer things to go wrong with a vehicle--that is one of the advantages of electric over ICE after all. If anything this is an acknowledgement that George Hotz was right in focusing on optical sensors with Comma.ai.