29 ms·

Why Tesla removed radar and ultrasonic sensors [video]

241 points by shekhar101 4y ago

yreg 4y ago
They are pretending as if the USS were there only for self driving. I use them as well!
- georgeg23 4y ago
  Indeed the ultrasonic sensors are pretty critical for (human) parking and backing up.
Haga 4y ago
TheLoafOfBread 4y ago
This whole question about the vision boils down to "humans don't need it so cars should not need it too" the problem with this statement is that humans does not have wheels to move around, they have legs, but wheels are ridiculously simple compared to 4 legs tapping 160km/h on a highway. Same for birds - they also does not need jet engines to fly around, but imagine Airbus A380 flapping its wings and what kind of complexity would you need to flap 800km/h through air.
- ajross 4y ago
  But the question at hand is system control, not locomotion! You're not asking the automation to walk (well, I mean someday we will, but Teslas have wheels), nor the aircraft to flap. We want the automation to do what a human pilot would do. And that works with eyes. No, I think this argument is largely correct. And frankly settled: anyone who's driven recent FSD beta versions knows very well that the cars "see just fine". They don't hit anything, they see and avoid obstacles. Frankly they're much more observant than humans are, my car will twitch when pedestrians turn as if they're going to enter the road (where human drivers mostly don't notice, and if they do they ignore it). What problems still exist are in planning: things like sign reading, lane selection, etc... still need some work. But collision avoidance just isn't an issue. It isn't. The LIDAR folks were wrong, basically. (I will admit though that I'm a little sad about the removal of the ultrasound sensors though. It's true the autonomy probably doesn't need them, but I really like having the chimes to guide parking and garage maneuvering.)
  - elteto 4y ago
    > The LIDAR folks were wrong, basically. This is far, far from settled at this point.
    - ajross 4y ago
      No, it's over. Look, the LIDAR value proposition was necessarily "Yes, we're outrageously expensive and involve major tradeoffs in physical design of the vehicle, but vision can simply never do what we do". And... vision does. It does, every day. On hundreds of thousands of cars. In point of fact FSD beta vehicles are out there every day in environments where LIDAR has never been deployed, nor likely ever tested. And we're not seeing clear "it can't do this" failures. Anywhere. The closest you're going to get to evidence against vision are things like that "It Hit A Kid!" stunt from a few months back that turned out to be basically faked[1]. [1] At least the perps went silent and no one was ever able to reproduce. I mean the whole idea was ridiculous: my car twitches at pedestrians, including kids, including my kids, every day. It literally draws them on the screen.
      - deleted 4y ago
        [deleted]
      - threeseed 4y ago
        > And we're not seeing clear "it can't do this" failures. Anywhere Do you have evidence of the number of accidents, disengagements etc by region ? Because you're making awfully definitive statements about FSD safety.
        ra7 4y ago
        Tesla doesn’t release any data. But there are community trackers [1] that puts the disengagement rate in the single digit miles. In comparison, Waymo and Cruise had a rate of roughly 30,000 miles/disengagement according to CA DMV data. That’s how much worse Tesla FSD is. [1] https://www.teslafsdtracker.com/ https://www.teslafsdtracker.com/
      - 4y ago
- elteto 4y ago
  More importantly, we have a tremendous data "engine" processing input from our senses. So assuming for a second that cameras match what our eyes can do, you still do not have a processing engine on the level of our brain to make sense of those inputs.
- AtlasBarfed 4y ago
  soooo... you're agreeeing that non-vision isn't necessary since the control domain is so much simpler? I personally think they should use as much data inputs as possible: radar, IR, LIDAR, mesh networks, fixed route information. Where tesla went particularly wrong IMO is ignoring some sort of route-based chunk information which is how humans navigate. IIRC Elon said something to the effect of just having an algorithm to work everywhere. Humans use the basic algorithm "stay in lane, drive forward" and then decorate with signs, knowledge of curves, locations of potholes, dangerous low-viz corners, likelihood of surprise stopped traffic, obscured driveways, general character of neighborhoods, road purpose. Weather. Windy sections, icy sections, light availability anomalies. What type of vehicle. Repair state of vehicle. A general AI algorithm will never be able to properly account for flavors/tags/chunk info on routes. Especially since cloud precomputation is so available these days. Anyway, while recognizing that Tesla's "Fully Self Driving" is not as advertised, and we are a ways from self driving for any statistical measure of superiority to a healthy aware adult, it is still damn impressive what FSD vids show. Do AI driving systems try to make "subsystems" of AI networks to reduce inputs to various higher-level inputs, or do other just throw a ton of inputs at a big ass network and just let the entire system rise from the soup of information?
  - latchkey 4y ago
    > Humans use the basic algorithm "stay in lane, drive forward" If you've ever driven in Vietnam, that is so not true.
    - pclmulqdq 4y ago
      Hell, even in the northeast US (particularly the cities) this isn't true. Self-driving cars today seem to have a dogmatic focus on California-style driving.
  - 113245 4y ago
    The Tesla AI day videos [1] go into some detail about this. They use multiple networks that are dedicated to specific tasks. [1]https://www.youtube.com/watch?v=j0z4FweCy4M https://www.youtube.com/watch?v=j0z4FweCy4M (2021), https://www.youtube.com/watch?v=ODSJsviD_SU https://www.youtube.com/watch?v=ODSJsviD_SU (2022)
- Nomentatus 4y ago
  No, he precisely said that the difference Lidar made was tested, and the delta (difference made) was quite small; not enough to outweigh the downsides. Elon has noted that humans do well, and that's relevant, but that observation was also tested, re lidar.
  - NBJack 4y ago
    Basically everything he said as a justification (sourcing, firmware, etc.) applies to every sufficiently advanced part of the vehicle. By that logic, they should not be using touchscreens on the center console, etc.
    - Nomentatus 4y ago
      If the safety delta was low after removing them, as 'tis with Lidar, absolutely.
      - paulryanrogers 4y ago
        It's also possible their LIDAR implementation was poor. Waymo uses LIDAR and has fewer incidents per mile.
        Nomentatus 4y ago
        Good point. I'm pretty sure it was poorer, Waymo really goes all-in on Lidar, they still top-mount it don't they? Waymo is also solving a much more constrained problem, with far fewer vehicles, so fewer accidents doesn't surprise me.
        freejazz 4y ago
        They are doing exactly what Tesla isn't willing to do and are being responsible in exactly the way Tesla isn't.
        4y ago
- m463 4y ago
  > "humans don't need it so cars should not need it too" I think of parking and I'm reminded of "the camry dent" https://duckduckgo.com/?q=the+camry+dent&iax=images&ia=images https://duckduckgo.com/?q=the+camry+dent&iax=images&ia=image...
- dmix 4y ago
  > This whole question about the vision boils down to Is that really what the problem boils down to? Or how it was decided? Or are you just questioning a common meme that comes up in internet debates about car AI?
- yarg 4y ago
  Human binocular vision is what has been used to drive cars up until now, so it can be done (with a few thousand million years of iteration). Ideally cars will be self-driving using only passive sensors - but I do think that Musk/Tesla completely missed the value of active sensors in training.
  - gibolt 4y ago
    Pretty sure humans haven't been striving for drivers licenses for millions of years... Tesla does use Lidar on a small number of test vehicles for assessing ground truth. However, they have built enough of a data pipeline and fleet data acquisition to use repeat clips to determine ground truth better than human labelers.
    - tibbydudeza 4y ago
      But the "system" is so adaptable from bipedal locomotion, spotting predators or prey and identifying unfit food, figure out social hierarchy, human facial expressions that driving a car is easy.
- codeflo 4y ago
  Exactly. The way biology solved something may not always be the best way to do it with technology, because the constraints or so different. And to be more blunt, I think none of the problems where technology surpassed human performance were achieved by doing it the exact same way. From locomotion (legs vs. wheels) to playing chess (strategic intuition vs. billions of calculations).
eachro 4y ago
So the key question is how much of an improvement does radar/sensors/etc give you over just using computer vision?
- throwntoday 4y ago
  If we're to trust what Elon and the team said during the last few AI day, none. They stated that the ultrasonic and radar sensors were actually performing worse than their pure vision stack.
  - justapassenger 4y ago
    Real life performance of vision only stack doesn’t agree with it.
  - quonn 4y ago
    I‘m ready to be convinced that this will be true at some point for the ultrasonic sensors. But by design the radar can see things that vision can never see. It seems like a bad idea to take that away.
    - throwntoday 4y ago
      Right but I think the signal to noise ratio was eating to much compute for what was little payoff. And either the ultrasonic or radar don't even work above 10mph, I forget which. They are used purely for parking.
  - deleted 4y ago
    [deleted]
- diskzero 4y ago
  As someone working in the field, I would never choose to eliminate the information provided by radar, lidar and any other sensor technology. Depending only on camera information would be too limiting.
  - 4y ago
dane-pgp 4y ago
I think there's an interesting general optimisation problem here of balancing the accuracy/performance of a software/hardware system, against the goal of making that system easier to iterate on and develop. Presumably this is a matter of working out if you are at a local maximum or not, and thinking about what properties the ideal solution will have. It also matters if you have other competitors that might be racing towards the ideal solution faster than you, potentially patenting their progress along the way.
justapassenger 4y ago
TL;DW. Tesla doesn’t know how to do change management.
- Nomentatus 4y ago
  Given Andrej's explanation this verges on mere gainsaying. Could you expand on what you think they should be doing? Other firms would also encounter "entropy", they always do; what's your way of reducing that severely?
  - justapassenger 4y ago
    Andrej refused to answer the question about why, and instead gave whole speech about how change management and supply chain is hard. What else is there to expand on? And of course other companies encounter it. But this has very strong vibes of “no one knew health care can be so complicated”. And what should they do? Change management. Yeah, it’s hard, and costs money, but it’s a hard industry to be in. What they should not do? Cripple products they already sold (like disabling radar in existing cars), because they need to milk their profit margins. Andrej is a great researcher. But safety critical systems are very different from research projects.
    - Nomentatus 4y ago
      Andrej did answer, he said that the delta was tested and found wanting, then detailed a number of risks that came with keeping Lidar. Positive but small. Risks larger. That exactly answers "don't you want all the help you can get." (paraphrase, from memory.) Securing supply chains is hard period. Tesla and SpaceX have gone to heavy vertical integration because ensuring quality seemed to insist upon that. What specifically would the new management team do that would secure supply chains trivially, etc? What's not being done?
      - Veserv 4y ago
        They said “change management” as in the management of changes not the replacement of the management team.
CharlesW 4y ago
I thought it was telling that Andrej immediately "reframed" the question because Lex asked the "wrong question". This is a classic evasion technique one learns from experience and/or media training. Lex's comment immediately after was a clever and gentle dig at Andrej's response. It seemed like all the "full cost" negatives Andrej mentioned were related to Tesla's ability to execute, and not what would actually produce better results. Tesla would have to be able to reliably procure parts, write reliable firmware, create designs and processes that won't increase unexpected assembly line stops, etc. Regarding results, the best Andrej can do is, "In this case, we looked at using it and not using it, and the delta was not massive." In other words, the results are better, but not enough to make up for the fact that Telsa can't support additional sensors without incurring a prohibitive amount of additional risk to Tesla. Risk to passengers doesn't appear to be a consideration.
- latchkey 4y ago
  Not just risk to passengers, risk to any thing in proximity to the vehicle while it is in motion.
- 3apo 4y ago
  I feel the other reason is that Tesla has not figured out a way to put Radar into their ML pipeline. If you take the Range-Doppler Map from the radar as the 'pixel' map, that data is inherently very dependent on the scenario and the radar sensor intrinsic parameters. This variability in what the radar sees in the RD space is what makes this a challenge for ML/AI pipelines. If Tesla were to 'fuse' information from these sensors in the object track level - I believe they will be less susceptible to this variability.
  - dreamcompiler 4y ago
    Exactly. Radar gives you direct range data; camera pixels need to be processed by ML to infer range data, and the latter is never going to be as close to ground truth as the former, so the former should be prioritized.
    - 4y ago
bekantan 4y ago
He explains it quite well: all necessary information is already in the pixel-space and adding more sensors slows team down more than it improves the system performance. My understanding is that major blockers are not in perception area anyways, would be great if someone with relevant experience could comment if this is indeed the case.
- diskzero 4y ago
  I am a principal engineer for a major autonomous vehicle company. You can break this statement down into two components: Adding more sensors slows his team now more than it improves system performance I'll take his word on this. It is a lot of work to incorporate multiple sensors. All necessary information is already in the pixel-space. I hate to disagree with someone as distinguished as Karpathy, but this is simply not what I have observed from all of that data that we have access to. Given my knowledge of the various stacks deployed today, I would never ever ever get into a vehicle using a vision only stack and expect it to perform in some of the challenging environments encountered during testing.
  - kfarr 4y ago
    Full on agreement. There are literally videos of Teslas smashing into stationary vehicles on the highway at night using only vision camera for FSD. No way any rational actor could claim the visible pixel space is sufficient in that scenario compared to LIDAR, Radar, etc
    - alsodumb 4y ago
      It's funny you use Radar as an example of 'good sensor' while it is well known that most or maybe almost all?) of the stationary vehicle accidents you're talking about happened because of Radars inability to detect a stationary obstacle. On the other hand, RGB data does have that information, we use it everyday to avoid obstacles, even under foggy and rainy conditions (I'm no LIDAR expert but I know it sucks in rainy conditions) I am not saying I support a vision only stack, but all I am saying is it is certainly possible to deploy a vision only stack in the future.
01100011 4y ago
I still suspect it's because they need to preserve compute resources for vision processing. Sensor fusion is likely eating up too much of their current HW and limiting their progress in other areas. I suspect Tesla will have to admit they need to upgrade the current HW before they ever 'solve' FSD.
- snovv_crash 4y ago
  The amount of compute that sensor fusion uses is miniscule compared to running a NN or computing stereo depth maps. Sensor fusion runs in the background of your phone the whole time to power things like [0] for example. 0. https://sensor-js.xyz/demo.html https://sensor-js.xyz/demo.html
  - Nomentatus 4y ago
    Well, yes and no. Integrating the data and adjudicating conflicts between sensors is a real task, too. Also having just two opinions doesn't necessarily help if they conflict, and the lidar is the thinnest source of data. How do you coin flip that? You likely end up just discarding the Lidar's conflicting opinion.
    - snovv_crash 4y ago
      LIDAR data might be more sparse but it is also more reliable, being an active sensor so it isn't affected by night. The real way is to use the LIDAR to add to the depth probability distribution in the stereo depth estimation. This way you aren't throwing any data away. LIDAR often gives probabilities as well, for example, and this can be used to eliminate reflections.
  - 01100011 4y ago
    You are confused my friend. Reading sensor data is not the same as feeding that data to a neural network and asking it to form a worldview composed of possibly conflicting sensor data streams(i.e. lidar vs vision vs ultrasonic). You are somewhat correct that it is quite trivial to read sensor data. For many sensors, there is some work which needs to be done to denoise or cleanup the input data. That's not where the story ends, however.
a-dub 4y ago
i'm not sure if i buy his argument that the "delta is not big enough." i have some experience with realtime ai systems and i've noticed something interesting about them. they have a non-smooth capability curve, where they can demonstrate proficiency in activities that in regular computer programs or people would imply a complete and continuous path of capability that has been mastered to achieve the demonstration, but ai systems are weird in that can do amazing things, but have loads of little holes and failure modes along the way. for example: gpt-3 can write you a shell script that will emit a c program that prints a poem about people you know, but will fail at very basic logic, sometimes. in light of that, having additional support data like radar or lidar seems like the right move for plugging all those little holes in capability that turn up in real ai systems. because at the end of the day, when you're driving a car in the real world and lives are at stake, simply interpolating or averaging over uncertainty seems awfully deadly and the only way to ameliorate that uncertainty seems to have multiple redundant sensory systems that can stand in for each other as conditions change. just like us!
- Nomentatus 4y ago
  They do "fall off the edge of the world" a lot; but so do human neural networks; I've seen a bad crash as a result of a human simply pulling out of a driveway right in front of a motorcycle, 'cause they're rare. She had tagged the motorcycle as a bike while it was farther away, then boom. Her interpolation (while checking the other side) didn't work, and her averaging over uncertainty didn't work either because motorcycles are rather rare up north, they aren't the average vehicle. I've made a similar error re a kid on a wall (he suddenly jumped directly into the bikepath) but managed to avoid him (my bike zoomed to his left and I tumbled past his right. He wasn't hurt, although I got a severe wrist sprain from throwing the bike to the left.) As a driver I behave very differently around kids on walls, now. It was just an edge case I'd never encountered, and I didn't have enough data to calculate under uncertainty.
  - api 4y ago
    I see surprisingly little discussion of overall statistics on safety of self drive vs humans, and what I do see is often self reported by companies or by equally potentially biased sources in the media. I’ve searched many times and a straightforward stat seems hard to find.
mongol 4y ago
What is it that makes Lidar so expensive? Is it something intrinsic to the technology that prevents costs from coming down?
- diskzero 4y ago
  A LIDAR sensor is a complex device with spinning motors, mirrors, lasers and more. Costs are coming down and less-expensive and more capable devices are coming to market. Once the price-points come way down, I wouldn't be surprised to see Tesla reconsider their decision to exclude them from their sensor platform.
  - Nasrudith 4y ago
    They are also fairly power-hungry from what I heard.
  - frxx 4y ago
    There are also solid state LIDARs these days which involve no moving parts. Still use more power than radar though.
- fooblaster 4y ago
  Lidar is coming down significantly in the next few years and is available from multiple tier 1 automotive suppliers. Technologies like high power vcsel arrays and highly integrated and photosensitive SPAD arrays/detector logic are making this possible. Prior lidar devices used non automotive grade discrete components like edge emitting laser diodes, high speed adcs, and APDs. These were expensive and hard to integrate, and aren't present in mass market inexpensive AM lidar coming to the market.
petilon 4y ago
I didn't find his answers particularly convincing. His answer focused on costs mainly, and how "the best part is no part". We have already seen multiple accidents caused by camera's limitations [1] which would not have happened if Tesla used Lidars. Cameras have poor dynamic range and can be easily blinded by bright surfaces. While it is true that humans do fine with only eyes, our eyes are significantly better than cameras. More importantly, expectations are higher when an automated system is driving the car. It is not sufficient if, in aggregate, self-driving cars have fewer accidents. If you lose a loved one in an accident where the accident could have been easily avoided if a human was driving, then you're not going to be mollified to hear that in aggregate, fewer people are being killed by self-driving cars! You'd be outraged to hear such a justification! The expectation therefore is that in each individual injury accident a human clearly could not have handled the situation any better. Self-driving cars have to be significantly better than humans to be accepted by society, and that means it has to have better-than-human levels of vision (which lidars provide). [1] https://www.youtube.com/watch?v=X3hrKnv0dPQ https://www.youtube.com/watch?v=X3hrKnv0dPQ
- bergenty 4y ago
  Not to mention waymo works well with LiDAR, cameras and radars. If you’re argument is it’s too hard to deal with that much data, it’s definitely the wrong answer.
- Nomentatus 4y ago
  You're right to note the advantages of lidar and (narrow range of) contrast problem for cameras (they arent eyes.) This is why the Uber human driver shouldn't have trusted the machine at night, in particular. But you still have to address his system argument, which was that adding geegaws that added little would actually increase overall risks along the supply chain (plus maintenance) while distracting the team and adding more risk that way, for very little apparent (but only apparent) gain. The team does believe that they'll get to better than human driving, and do that without lidar.
  - Marazan 4y ago
    They cannot actually believe that.
0xfffafaCrash 4y ago
Seems like a very political answer from Andrej. Of course he’s not going to outright say “yeah, we’re prioritizing the profit margin over accuracy and safety considerations” if he wants to keep his job, but that seems to be the short of it. Others may choose to follow, at least in the short term, but it won’t be because of “entropy” making the system worse (you can always build a model without a data source and then refine the results based on the added value of a data source) but doing so just because it will save lives doesn’t cut it when the goal is to cut corners and costs to maximize profit. I can believe that some types of sensors aren’t worth the trouble in terms of additional signal to noise ratio, but I can’t believe this is one of them.
- Nomentatus 4y ago
  The entropty he's taking about comes from many sources, in particular opening yourself up to maintenance or supply side errors, and just overloading team attention (always at a premium) for no net return. It's not just CPU cycles (although that's part of it, that hardware could be doing something else useful.)
bigtex 4y ago
Did Lex ask him why Tesla love to crash into emergency vehicles?
- deleted 4y ago
  [deleted]
xnx 4y ago
Sensor fusion seems to be another thing that Tesla is not good at.
JaggerJo 4y ago
This sucks for parking. It is simply (physically) not possible for the existing cameras to see the area directly in front of the car. So how would this work for parking? A: Add more cameras so there are no dead areas in front of the car B: build a model in vector space when driving towards a parking spot and assume blind spots don't change. (still sucks)
- oxplot 4y ago
  > It is simply (physically) not possible for the existing cameras to see the area directly in front of the car. Think about how a human driver does it, given his/her even worse vantage point. They model what's in front/behind the car from afar and remember what's where as they approach it. There are other signals as well, such as continuation of a kerb, etc. I think people keep forgetting that Teslas run hundreds of ML prediction tasks all the time. Watch recent AI day and their talks about "occupancy network" to get a sense of the car's ability to: 1. Construct 3D model of its surrounding in real time; 2. Remember occluded sections based on what's it's seen previously.
  - watwut 4y ago
    Human driver constantly turns head around to where he is Mos likely to hit something.
    - oxplot 4y ago
      Well, the car has 360 degree camera view, with far wider coverage than a turning head in a driver seat. And more importantly, it sees in all directions at all times.
sidcool 4y ago
I feel that was more an operational answer than an engineering one .. I still feel that depth perception of vision alone is unreliable.
post_break 4y ago
I think this is the real reason: https://www.youtube.com/watch?v=LS3Vk0NPFDE https://www.youtube.com/watch?v=LS3Vk0NPFDE Cost cutting.
- taf2 4y ago
  Probably a benefit but also imagine the difference in software. You Boolean logic like radar says we are gonna hit , vision sensor says we have nothing there, sonar indicator says nothing there… so the idea of having just a really good single source of truth probably makes a lot of code a lot less complex… I have no way of knowing either way but from a less is less that can break point of view this seems kinda good… like many things only time can tell and at least we have different groups of people pushing on different potentially viable paths forward so that we can soon hopefully know if self driving is possible (wide scale) one way or the other
friend_and_foe 4y ago
I remember watching an interview with George Hotz when Comma.ai was young, where he essentially said this as a critique of Tesla. He's a bit of a showman and likes to invite a little controversy when he says things, but I found myself agreeing with his point. It's not surprising to see such a practical company like Tesla face the facts about all these sensors eventually.
- P_I_Staker 4y ago
  > such a practical company like Tesla Where are you getting that from? Tesla has always seemed pie in the sky, and hardly a down to earth company at all throughout the history. I'm basing this one both their public record, and reputation within the auto industry.
  - friend_and_foe 4y ago
    Tesla created the modern electric vehicle industry. They're innovators, sure, and they push limits, but their priority has always been to actually build. And they do.
woeirua 4y ago
Andrej's argument about more sensors adding entropy strikes me as disingenuous considering that in the next question he then says that Tesla's biggest advantage over everyone else is "the fleet", which clearly introduces orders of magnitude more entropy into the system than anything else. Can you imagine the infrastructure required to gather video from "the fleet" anytime a car sees something unexpected? How about diagnosing what went wrong in that specific instance? How many thousands of these cases do they see everyday? Given the progress of the FSD "beta" to date, and the fact that Andrej _left_ Tesla, I'd wager that he knows that this approach is a dead end, but he won't say that because he'd get himself in hot water with Elon.
- dmix 4y ago
  One is infrastructure entropy and the other is software engineering entropy (we're still talking about data from one type of sensor, just at larger scale). Most tech startups have 10x+ more problems with the engineering part than the infrastructure/ops part. Also this is one person's perspective from a large team. His answer might be biased because he's an engineer and I doubt his was the only voice in the debate.
- ralfd 4y ago
  Assuming Karpathy is lying is quite a hot take to disregard his opinion. No. He makes it clear that he is very convinced about it. There is no relativism, no weasel words or couching in maybes. He could be wrong, of course, but he believes in what he is saying.
  - nicbou 4y ago
    > no weasel words The video starts with him reframing the question instead of answering it
    - epgui 4y ago
      I regularly reframe questions when I think they're not the most interesting. It's a very common thing to do, especially in academia, that doesn't particularly indicate deception.
lawrenceyan 4y ago
I can see a path where with only cameras, Tesla might be able to reach level 4 autonomy in perfect conditions. But the biggest thing that comes to mind is what happens at night. Are they only going to enable self-driving during the day?
- speedgoose 4y ago
  Wouldn’t turning on the headlights fix the problem at night? Snow and ice may be another challenge but night sounds easy.
  - lawrenceyan 4y ago
    When you drive at night with headlights, tell me honestly how confident you feel driving versus during the daytime.
KVFinn 4y ago
TLDR: Tesla thinks LIDAR hardware is more expensive than the performance improvement it provides. I didn't like his line of logic about how vision is necessary and sufficient, because that's how humans drive. Okay sure, but if some combinations of non-human sensors could drive better and/or cheaper than a vision only driving system, surely he would not argue for sticking with vision only? Maybe adding non-vision sensors lets you save hardware and software resources on the vision part of the system.
dreamcompiler 4y ago
It's obviously a stupid decision to remove a direct source of range data (radar and ultrasound) in favor of an indirect one (vision). But on second thought this doesn't bother me that much because Tesla FSD is absolute garbage even with radar (and I don't think Tesla will get away with selling the FSD snake oil for much longer), so if vision-only is good enough for the base-level lane-keeping autopilot functionality and it makes the cars cheaper, maybe that's a good thing.
- dmix 4y ago
  Even though the risks are high the outcomes will always keep them honest. Whether they like it or not. This isn't like Facebook continually releasing a product that sucks but people will use anyway. Tesla is constantly working against the clock and everything they do has real world consequences. There are multiple gov agencies watching over it at all times. Of course there's lots of people with far higher risk tolerance than is being exhibited but if it does turn out badly IRL this will get shut down pretty quickly. The good news is Tesla has the ability to cripple this feature remotely without a costly/lengthy recall if that does happen.
  - AlotOfReading 4y ago
    Outcomes have to be truly and utterly fucked for "new" types of problems to be noticed in the automotive industry. Take the Toyota Unintended Acceleration case, where completely negligent software quality took over a decade and at least 89 lives to be noticed and (partially) rectified. Regulators have keen noses for very particular types of issues and rely heavily on manufacturer judgements on a lot of the rest. Issues that aren't in any of those fairly narrow categories need to be extremely public or extremely egregious to attract their notice.
    - dmix 4y ago
      I’ve read into the Toyota case in the past and it doesn’t seem like it was that obvious of a regulatory failure, where you could come to such a conclusion. Did I miss some good article covering it? Because everything I’ve read with the benefit of retrospect has been pretty critical of a lot the alleged ‘cases’ reported in that 89 number. Are you sure that’s the example you want to use for pre-emptive regulatory failure? And do you think Tesla could demonstrated failures at a similar rate (assuming they were legit) and get away with it? Because I dont
danbmil99 4y ago
Musk is also famously against using lidar. He doesn't understand/accept that an autonomous vehicle needs any sensors that humans do not posess.
julienreszka 4y ago
Geohot said something similar years ago already
superkuh 4y ago
Humans don't use radar or ultrasound sense to drive. If we want cars that drive like humans drive they should use the same senses. For example, in the northern parts of the USA there is snow cover for much of the year and lanes are emergent from flocking without any absolute reference to the actual location of the lanes. The reasons everyone choses the same places to drive are that they see the same environment with the same senses. Even if autonomous driving with radar and ultrasound was made to work if it picks the correct lane position and all the humans pick the wrong new lane position then the car is wrong, not the humans.
oxplot 4y ago
Here's the summary (mixed with observations from Munro and past Tesla presentations): - Costs money: the physical sensors (a dozen of them), wiring it up, assembling it, maintain inventory, code it, etc. - Time spent on maintaining, improving software stack for the non-vision sensors as well as efforts needed to fuse the data with vision, takes away from focusing on vision alone. It also holds back vision in relevant areas. - Existing non-vision sensors used by Tesla are orders of magnitude lower fidelity than vision. It has historically (as the case with radar) led to vision essentially having to overriding radar because vision just performed much better (see AI day 2021). My take: As with any new tech, it likely sucks at the start (think HDD and SSDs, and how a mechanical thing with lots of moving parts was way more reliable than SSDs at the start). However, by essentially moving past the local maxima, you get to innovate better, faster in the future. In case of ultrasonic sensors, they are for low speed cases anyway and most people are fine without them. Majority of fatalities and injuries happen at higher speeds.
- rootusrootus 4y ago
  That's great for them. But when I'm shopping for a car, I get to choose between a manufacturer that installs the extra sensors and seems to be able to get them to work, and Tesla. Used to be that Tesla was blazing a trail and if you wanted a good EV, that was what you got. Now, if you want the best EV, it's usually not going to be a Tesla. And I don't see that they're making any decisions that will regain them that title. The incumbent manufacturers are quickly proceeding to eat their lunch, just like many of us predicted would happen. Turns out the hard part of making a successful car isn't the drivetrain.
  - oxplot 4y ago
    > Now, if you want the best EV, it's usually not going to be a Tesla. Would love to hear what you consider "good" and what specific EV ticks the most good features that a Tesla Model 3 doesn't.
    - lnsru 4y ago
      360 degree camera on BMW i4. Overall build quality too.
smrtinsert 4y ago
Hm I'd rather have someone from Twitter audit this decision
gnicholas 4y ago
How does it make sense to not even have sensors for parking? If you think they don't help during normal-speed driving, that's one thing. But they obviously help during parking, since (IIRC) they've had to disable certain autonomous features until they get their vision-based systems upgraded to be able to fill in this gap.
ornel 4y ago
Video summary: https://www.summarize.tech/www.youtube.com/watch?v=_W1JBAfV4Io https://www.summarize.tech/www.youtube.com/watch?v=_W1JBAfV4...
nelox 4y ago
“The world is designed for human visual consumption” and “[vision] has all the information you need for driving”. While vision may be sufficient, I would say that other senses, such as hearing, touch and smell augment driving very well. Especially with regard to situational awareness. e.g. The sirens of emergency vehicles are typically the first indication of their presence, which often can be felt as well. The wail of a tornado siren, similarly. Loud throbbing motorcycles do much to improve rider safety simply due to them broadcasting their presence. At railway level crossings, drivers should slow down, look and listen for oncoming trains. The smell of wildfire or bushfire smoke provides enhanced warning or nearby danger. So to say vision is sufficient, does not fully take in to account the driver experience, especially where safety and situational awareness are concerned.
- djleni 4y ago
  THANK YOU. Reading this thread was getting to me because so many comments say humans drive eyes only. I use far more than just vision driving: - sound, for emergency vehicles, detecting vehicles outside of my field of view if my windows are down or the vehicle is loud, tire sound (especially in snow and rain), engine sound (more feedback in snow or ice about what my tires are doing) - touch (steering feedback, gives information about grip in some circumstances) - acceleration (can feel if the rear tires break loose in a turn on snow or ice, or if I’m sliding while breaking) And probably many more
  - mola 4y ago
    We even use Doppler! Our hearing is capable of sensing movement (speed and acceleration) using the Doppler effect. Our hearing also has a remarkable ability to locating sound source incoming direction.
  - AlotOfReading 4y ago
    It's worth noting that most autonomous vehicle solutions have dedicated microphones for emergency vehicles and sensors that can detect slip. I had a little dashboard measuring wheel slip at <company>. It mainly ended up mapping train tracks and potholes.
hbarka 4y ago
Intuition + other examples tell you that radar and ultrasonic sensors work. Why do we twist ourselves to believe otherwise? Elon removed the radar and ultrasonics for the simple fact that its supply chain logjam was screwing up the manufacturing schedule. They also realized that the profit margin can be sustained in an inflationary environment by simply removing these parts [1]. “Oh, we were going to remove them anyway because humans can see fine with just eyes and no radar, why can’t cars?” Tesla then turned up the marketing of the AI/vision hype lever once more to toss another shiny tech object and get buyers to ignore the fact that there is a regression of features in the newer cars going forward. [1] https://youtu.be/LS3Vk0NPFDE https://youtu.be/LS3Vk0NPFDE
- mgoetzke 4y ago
  So all those engineers are lying when they talk about this topic ?
  - 9935c101ab17a66 4y ago
    Uh, which engineers are you referring to?
EVa5I7bHFq9mnYK 4y ago
From the first principles point of view, it comes down to radar and ultrasonic having much higher wavelengths than optical. Which results in much lower amount of incoming information, worse resolution and higher interference if many cars radiate the same signals on a busy street.
jakeogh 4y ago
Giving human[1][2] drivers better situational awareness[3] is the future. Specifically open[4]: a. Windshields that clean the inside as well as the outside. b. Better eyeglasses[5]. c. User controllable hi-res HUD thermal IR overlay. d. Headlights with adaptive notch filters so the oncoming vehicle can pick an empty spectral range... without the source being monochromatic (with required adaptive filters on the recieving end)... and/or really good coronagraph's. e. Brake control[6]. Any entity capable of driving[7] in a population of humans (including adversarial humans) is sentient[8], and has real skin in the game. It would be unethical to lock one in a car: [1] https://news.ycombinator.com/item?id=33213860 https://news.ycombinator.com/item?id=33213860 (analog FPGA) [2] https://news.ycombinator.com/item?id=21106367 https://news.ycombinator.com/item?id=21106367 (general AI) [3] https://news.ycombinator.com/item?id=16646112 https://news.ycombinator.com/item?id=16646112 (2018) [4] https://www.tesla.com/blog/all-our-patent-are-belong-you https://www.tesla.com/blog/all-our-patent-are-belong-you (2014) [5] https://patents.google.com/patent/US7744217 https://patents.google.com/patent/US7744217 (2007) [6] https://news.ycombinator.com/item?id=18013388 https://news.ycombinator.com/item?id=18013388 (2018) [7] no human behind the wheel, no human to correct impending mistakes, but (critically) with one or more humans in the car. [8] The idea that non-biological machines can have 'self' is a window into modern mass transformation. Please checkout the analog FPGA experiments linked above.
sgjohnson 4y ago
Does this mean that now when someone smashes one of their bumpers on a Tesla, the insurance will no longer have to total the entire vehicle?
mavili 4y ago
Did anyone else catch Andrej's "sqlite" comment? If that is not just a simple analogy, Tesla may be using sqlite in their cars? :D
ra7 4y ago
If I were a Tesla fan/investor/FSD customer, I’d be very concerned that the former (effective) tech lead of FSD doesn’t know about sensor fusion or that it’s a solved problem for majority of the companies in this space.
Animats 4y ago
From my DARPA Grand Challenge days, I used to have an Eaton VORAD automotive radar. This was an early design - 24GHZ, 1 scanning axis. It could see cars, but not bicycles, at least not reliably. For several months, I had one pointed out the window of my house, looking at an intersection. So I had a V-shaped wedge on screen, and could watch the cars go by. It's a Doppler radar, so you don't get any info from things stationary relative to the radar, but you do get range and range rate. And the quality of that data is independent of distance. We used it mainly as a backup system for the world model built with LIDAR and (to a very limited extent) vision. The VORAD data could lower the speed limit for the rest of the system, and if a collision was about to happen, it would slam on the brakes independently of the world model. The big problem with coarse automotive radar is that it can detect targets, but doesn't tell you much about them. Cars, trash cans, and metal road debris all look about the same. There's also a lot of trouble from big flat metal surfaces being mirrors for radar. We were willing to accept slowing down for ambiguous cases until the other sensors could get a good look. Drivers hate that if road-oriented systems do it. Modern units are up around 70-80GHz and often have 2D scanning, which is a big help. I haven't seen the output from a modern automotive radar. I was expecting that by now, low cost millimeter microwave systems (200-300GHz) would be available, providing detailed images somewhat coarser than you can get with light. You get range and range rate, and you can usually steer the beam electronically rather than mechanically. The technology exists to get high-resolution radar images, but is mostly used for scanning people for weapons at checkpoints. It hasn't become cheap yet.
nielsbot 4y ago
All I heard was "cost savings, cost savings, cost savings"
- oxplot 4y ago
  Well watch it again and again and again. He talks about the determental effect of lo-fi sensors in conjunction with vision among other things.
60Vhipx7b4JL 4y ago
From an engineering perspective I would ask: Can your sensor package understand the environment to the required (low) failure rate? Radar/Lidar/Ultrasonic is going to give you information that your camera systems will not give you. It does not matter if the delta of information is little. If this little is required because you can't obtain it otherwise, you still need it. If you just rely on the fleet, you rely on the things you have seen. What about the objects that you have not yet seen?
dncornholio 4y ago
Just remember folks, we will have full self driving vehicles by the end of this year!
throwaway4good 4y ago
Let me reframe the answer: "We removed them because they cost money. And we are trying to make money ... at least right now. Listen, this pure autonomous self-driving car stuff is never going to work, so who cares if we have these gadgets or not ..."
nova22033 4y ago
At 2:05. Suddenly you need a column in your sqlite telling you what type of sensor it is.... Seriously? This is a major technical challenge?
- danpalmer 4y ago
  The challenge isn't the storing of the flag that says which sensor it has, it's testing the combinations, training for the different scenarios, treating the incoming data differently, and so on.
deleted 4y ago
[deleted]
solardev 4y ago
Cuz Muskdaddy wanted mo money. There, mystery solved.

241 points by shekhar101 4y ago

yreg 4y ago

They are pretending as if the USS were there only for self driving. I use them as well!

georgeg23 4y ago
Indeed the ultrasonic sensors are pretty critical for (human) parking and backing up.

Haga 4y ago

TheLoafOfBread 4y ago

This whole question about the vision boils down to "humans don't need it so cars should not need it too" the problem with this statement is that humans does not have wheels to move around, they have legs, but wheels are ridiculously simple compared to 4 legs tapping 160km/h on a highway. Same for birds - they also does not need jet engines to fly around, but imagine Airbus A380 flapping its wings and what kind of complexity would you need to flap 800km/h through air.

ajross 4y ago
But the question at hand is system control, not locomotion! You're not asking the automation to walk (well, I mean someday we will, but Teslas have wheels), nor the aircraft to flap. We want the automation to do what a human pilot would do. And that works with eyes. No, I think this argument is largely correct. And frankly settled: anyone who's driven recent FSD beta versions knows very well that the cars "see just fine". They don't hit anything, they see and avoid obstacles. Frankly they're much more observant than humans are, my car will twitch when pedestrians turn as if they're going to enter the road (where human drivers mostly don't notice, and if they do they ignore it). What problems still exist are in planning: things like sign reading, lane selection, etc... still need some work. But collision avoidance just isn't an issue. It isn't. The LIDAR folks were wrong, basically. (I will admit though that I'm a little sad about the removal of the ultrasound sensors though. It's true the autonomy probably doesn't need them, but I really like having the chimes to guide parking and garage maneuvering.)
- elteto 4y ago
  > The LIDAR folks were wrong, basically. This is far, far from settled at this point.
  - ajross 4y ago
    No, it's over. Look, the LIDAR value proposition was necessarily "Yes, we're outrageously expensive and involve major tradeoffs in physical design of the vehicle, but vision can simply never do what we do". And... vision does. It does, every day. On hundreds of thousands of cars. In point of fact FSD beta vehicles are out there every day in environments where LIDAR has never been deployed, nor likely ever tested. And we're not seeing clear "it can't do this" failures. Anywhere. The closest you're going to get to evidence against vision are things like that "It Hit A Kid!" stunt from a few months back that turned out to be basically faked[1]. [1] At least the perps went silent and no one was ever able to reproduce. I mean the whole idea was ridiculous: my car twitches at pedestrians, including kids, including my kids, every day. It literally draws them on the screen.
    - deleted 4y ago
      [deleted]
    - threeseed 4y ago
      > And we're not seeing clear "it can't do this" failures. Anywhere Do you have evidence of the number of accidents, disengagements etc by region ? Because you're making awfully definitive statements about FSD safety.
      - ra7 4y ago
        Tesla doesn’t release any data. But there are community trackers [1] that puts the disengagement rate in the single digit miles. In comparison, Waymo and Cruise had a rate of roughly 30,000 miles/disengagement according to CA DMV data. That’s how much worse Tesla FSD is. [1] https://www.teslafsdtracker.com/ https://www.teslafsdtracker.com/
    - 4y ago
elteto 4y ago
More importantly, we have a tremendous data "engine" processing input from our senses. So assuming for a second that cameras match what our eyes can do, you still do not have a processing engine on the level of our brain to make sense of those inputs.
AtlasBarfed 4y ago
soooo... you're agreeeing that non-vision isn't necessary since the control domain is so much simpler? I personally think they should use as much data inputs as possible: radar, IR, LIDAR, mesh networks, fixed route information. Where tesla went particularly wrong IMO is ignoring some sort of route-based chunk information which is how humans navigate. IIRC Elon said something to the effect of just having an algorithm to work everywhere. Humans use the basic algorithm "stay in lane, drive forward" and then decorate with signs, knowledge of curves, locations of potholes, dangerous low-viz corners, likelihood of surprise stopped traffic, obscured driveways, general character of neighborhoods, road purpose. Weather. Windy sections, icy sections, light availability anomalies. What type of vehicle. Repair state of vehicle. A general AI algorithm will never be able to properly account for flavors/tags/chunk info on routes. Especially since cloud precomputation is so available these days. Anyway, while recognizing that Tesla's "Fully Self Driving" is not as advertised, and we are a ways from self driving for any statistical measure of superiority to a healthy aware adult, it is still damn impressive what FSD vids show. Do AI driving systems try to make "subsystems" of AI networks to reduce inputs to various higher-level inputs, or do other just throw a ton of inputs at a big ass network and just let the entire system rise from the soup of information?
- latchkey 4y ago
  > Humans use the basic algorithm "stay in lane, drive forward" If you've ever driven in Vietnam, that is so not true.
  - pclmulqdq 4y ago
    Hell, even in the northeast US (particularly the cities) this isn't true. Self-driving cars today seem to have a dogmatic focus on California-style driving.
- 113245 4y ago
  The Tesla AI day videos [1] go into some detail about this. They use multiple networks that are dedicated to specific tasks. [1]https://www.youtube.com/watch?v=j0z4FweCy4M https://www.youtube.com/watch?v=j0z4FweCy4M (2021), https://www.youtube.com/watch?v=ODSJsviD_SU https://www.youtube.com/watch?v=ODSJsviD_SU (2022)
Nomentatus 4y ago
No, he precisely said that the difference Lidar made was tested, and the delta (difference made) was quite small; not enough to outweigh the downsides. Elon has noted that humans do well, and that's relevant, but that observation was also tested, re lidar.
- NBJack 4y ago
  Basically everything he said as a justification (sourcing, firmware, etc.) applies to every sufficiently advanced part of the vehicle. By that logic, they should not be using touchscreens on the center console, etc.
  - Nomentatus 4y ago
    If the safety delta was low after removing them, as 'tis with Lidar, absolutely.
    - paulryanrogers 4y ago
      It's also possible their LIDAR implementation was poor. Waymo uses LIDAR and has fewer incidents per mile.
      - Nomentatus 4y ago
        Good point. I'm pretty sure it was poorer, Waymo really goes all-in on Lidar, they still top-mount it don't they? Waymo is also solving a much more constrained problem, with far fewer vehicles, so fewer accidents doesn't surprise me.
        freejazz 4y ago
        They are doing exactly what Tesla isn't willing to do and are being responsible in exactly the way Tesla isn't.
        4y ago
m463 4y ago
> "humans don't need it so cars should not need it too" I think of parking and I'm reminded of "the camry dent" https://duckduckgo.com/?q=the+camry+dent&iax=images&ia=images https://duckduckgo.com/?q=the+camry+dent&iax=images&ia=image...
dmix 4y ago
> This whole question about the vision boils down to Is that really what the problem boils down to? Or how it was decided? Or are you just questioning a common meme that comes up in internet debates about car AI?
yarg 4y ago
Human binocular vision is what has been used to drive cars up until now, so it can be done (with a few thousand million years of iteration). Ideally cars will be self-driving using only passive sensors - but I do think that Musk/Tesla completely missed the value of active sensors in training.
- gibolt 4y ago
  Pretty sure humans haven't been striving for drivers licenses for millions of years... Tesla does use Lidar on a small number of test vehicles for assessing ground truth. However, they have built enough of a data pipeline and fleet data acquisition to use repeat clips to determine ground truth better than human labelers.
  - tibbydudeza 4y ago
    But the "system" is so adaptable from bipedal locomotion, spotting predators or prey and identifying unfit food, figure out social hierarchy, human facial expressions that driving a car is easy.
codeflo 4y ago
Exactly. The way biology solved something may not always be the best way to do it with technology, because the constraints or so different. And to be more blunt, I think none of the problems where technology surpassed human performance were achieved by doing it the exact same way. From locomotion (legs vs. wheels) to playing chess (strategic intuition vs. billions of calculations).

eachro 4y ago

So the key question is how much of an improvement does radar/sensors/etc give you over just using computer vision?

throwntoday 4y ago
If we're to trust what Elon and the team said during the last few AI day, none. They stated that the ultrasonic and radar sensors were actually performing worse than their pure vision stack.
- justapassenger 4y ago
  Real life performance of vision only stack doesn’t agree with it.
- quonn 4y ago
  I‘m ready to be convinced that this will be true at some point for the ultrasonic sensors. But by design the radar can see things that vision can never see. It seems like a bad idea to take that away.
  - throwntoday 4y ago
    Right but I think the signal to noise ratio was eating to much compute for what was little payoff. And either the ultrasonic or radar don't even work above 10mph, I forget which. They are used purely for parking.
- deleted 4y ago
  [deleted]
diskzero 4y ago
As someone working in the field, I would never choose to eliminate the information provided by radar, lidar and any other sensor technology. Depending only on camera information would be too limiting.
- 4y ago

dane-pgp 4y ago

I think there's an interesting general optimisation problem here of balancing the accuracy/performance of a software/hardware system, against the goal of making that system easier to iterate on and develop. Presumably this is a matter of working out if you are at a local maximum or not, and thinking about what properties the ideal solution will have. It also matters if you have other competitors that might be racing towards the ideal solution faster than you, potentially patenting their progress along the way.

justapassenger 4y ago

TL;DW. Tesla doesn’t know how to do change management.

Nomentatus 4y ago
Given Andrej's explanation this verges on mere gainsaying. Could you expand on what you think they should be doing? Other firms would also encounter "entropy", they always do; what's your way of reducing that severely?
- justapassenger 4y ago
  Andrej refused to answer the question about why, and instead gave whole speech about how change management and supply chain is hard. What else is there to expand on? And of course other companies encounter it. But this has very strong vibes of “no one knew health care can be so complicated”. And what should they do? Change management. Yeah, it’s hard, and costs money, but it’s a hard industry to be in. What they should not do? Cripple products they already sold (like disabling radar in existing cars), because they need to milk their profit margins. Andrej is a great researcher. But safety critical systems are very different from research projects.
  - Nomentatus 4y ago
    Andrej did answer, he said that the delta was tested and found wanting, then detailed a number of risks that came with keeping Lidar. Positive but small. Risks larger. That exactly answers "don't you want all the help you can get." (paraphrase, from memory.) Securing supply chains is hard period. Tesla and SpaceX have gone to heavy vertical integration because ensuring quality seemed to insist upon that. What specifically would the new management team do that would secure supply chains trivially, etc? What's not being done?
    - Veserv 4y ago
      They said “change management” as in the management of changes not the replacement of the management team.

CharlesW 4y ago

I thought it was telling that Andrej immediately "reframed" the question because Lex asked the "wrong question". This is a classic evasion technique one learns from experience and/or media training. Lex's comment immediately after was a clever and gentle dig at Andrej's response. It seemed like all the "full cost" negatives Andrej mentioned were related to Tesla's ability to execute, and not what would actually produce better results. Tesla would have to be able to reliably procure parts, write reliable firmware, create designs and processes that won't increase unexpected assembly line stops, etc. Regarding results, the best Andrej can do is, "In this case, we looked at using it and not using it, and the delta was not massive." In other words, the results are better, but not enough to make up for the fact that Telsa can't support additional sensors without incurring a prohibitive amount of additional risk to Tesla. Risk to passengers doesn't appear to be a consideration.

latchkey 4y ago
Not just risk to passengers, risk to any thing in proximity to the vehicle while it is in motion.
3apo 4y ago
I feel the other reason is that Tesla has not figured out a way to put Radar into their ML pipeline. If you take the Range-Doppler Map from the radar as the 'pixel' map, that data is inherently very dependent on the scenario and the radar sensor intrinsic parameters. This variability in what the radar sees in the RD space is what makes this a challenge for ML/AI pipelines. If Tesla were to 'fuse' information from these sensors in the object track level - I believe they will be less susceptible to this variability.
- dreamcompiler 4y ago
  Exactly. Radar gives you direct range data; camera pixels need to be processed by ML to infer range data, and the latter is never going to be as close to ground truth as the former, so the former should be prioritized.
  - 4y ago

bekantan 4y ago

He explains it quite well: all necessary information is already in the pixel-space and adding more sensors slows team down more than it improves the system performance. My understanding is that major blockers are not in perception area anyways, would be great if someone with relevant experience could comment if this is indeed the case.

diskzero 4y ago
I am a principal engineer for a major autonomous vehicle company. You can break this statement down into two components: Adding more sensors slows his team now more than it improves system performance I'll take his word on this. It is a lot of work to incorporate multiple sensors. All necessary information is already in the pixel-space. I hate to disagree with someone as distinguished as Karpathy, but this is simply not what I have observed from all of that data that we have access to. Given my knowledge of the various stacks deployed today, I would never ever ever get into a vehicle using a vision only stack and expect it to perform in some of the challenging environments encountered during testing.
- kfarr 4y ago
  Full on agreement. There are literally videos of Teslas smashing into stationary vehicles on the highway at night using only vision camera for FSD. No way any rational actor could claim the visible pixel space is sufficient in that scenario compared to LIDAR, Radar, etc
  - alsodumb 4y ago
    It's funny you use Radar as an example of 'good sensor' while it is well known that most or maybe almost all?) of the stationary vehicle accidents you're talking about happened because of Radars inability to detect a stationary obstacle. On the other hand, RGB data does have that information, we use it everyday to avoid obstacles, even under foggy and rainy conditions (I'm no LIDAR expert but I know it sucks in rainy conditions) I am not saying I support a vision only stack, but all I am saying is it is certainly possible to deploy a vision only stack in the future.

01100011 4y ago

I still suspect it's because they need to preserve compute resources for vision processing. Sensor fusion is likely eating up too much of their current HW and limiting their progress in other areas. I suspect Tesla will have to admit they need to upgrade the current HW before they ever 'solve' FSD.

snovv_crash 4y ago
The amount of compute that sensor fusion uses is miniscule compared to running a NN or computing stereo depth maps. Sensor fusion runs in the background of your phone the whole time to power things like [0] for example. 0. https://sensor-js.xyz/demo.html https://sensor-js.xyz/demo.html
- Nomentatus 4y ago
  Well, yes and no. Integrating the data and adjudicating conflicts between sensors is a real task, too. Also having just two opinions doesn't necessarily help if they conflict, and the lidar is the thinnest source of data. How do you coin flip that? You likely end up just discarding the Lidar's conflicting opinion.
  - snovv_crash 4y ago
    LIDAR data might be more sparse but it is also more reliable, being an active sensor so it isn't affected by night. The real way is to use the LIDAR to add to the depth probability distribution in the stereo depth estimation. This way you aren't throwing any data away. LIDAR often gives probabilities as well, for example, and this can be used to eliminate reflections.
- 01100011 4y ago
  You are confused my friend. Reading sensor data is not the same as feeding that data to a neural network and asking it to form a worldview composed of possibly conflicting sensor data streams(i.e. lidar vs vision vs ultrasonic). You are somewhat correct that it is quite trivial to read sensor data. For many sensors, there is some work which needs to be done to denoise or cleanup the input data. That's not where the story ends, however.

a-dub 4y ago

i'm not sure if i buy his argument that the "delta is not big enough." i have some experience with realtime ai systems and i've noticed something interesting about them. they have a non-smooth capability curve, where they can demonstrate proficiency in activities that in regular computer programs or people would imply a complete and continuous path of capability that has been mastered to achieve the demonstration, but ai systems are weird in that can do amazing things, but have loads of little holes and failure modes along the way. for example: gpt-3 can write you a shell script that will emit a c program that prints a poem about people you know, but will fail at very basic logic, sometimes. in light of that, having additional support data like radar or lidar seems like the right move for plugging all those little holes in capability that turn up in real ai systems. because at the end of the day, when you're driving a car in the real world and lives are at stake, simply interpolating or averaging over uncertainty seems awfully deadly and the only way to ameliorate that uncertainty seems to have multiple redundant sensory systems that can stand in for each other as conditions change. just like us!

Nomentatus 4y ago
They do "fall off the edge of the world" a lot; but so do human neural networks; I've seen a bad crash as a result of a human simply pulling out of a driveway right in front of a motorcycle, 'cause they're rare. She had tagged the motorcycle as a bike while it was farther away, then boom. Her interpolation (while checking the other side) didn't work, and her averaging over uncertainty didn't work either because motorcycles are rather rare up north, they aren't the average vehicle. I've made a similar error re a kid on a wall (he suddenly jumped directly into the bikepath) but managed to avoid him (my bike zoomed to his left and I tumbled past his right. He wasn't hurt, although I got a severe wrist sprain from throwing the bike to the left.) As a driver I behave very differently around kids on walls, now. It was just an edge case I'd never encountered, and I didn't have enough data to calculate under uncertainty.
- api 4y ago
  I see surprisingly little discussion of overall statistics on safety of self drive vs humans, and what I do see is often self reported by companies or by equally potentially biased sources in the media. I’ve searched many times and a straightforward stat seems hard to find.

mongol 4y ago

What is it that makes Lidar so expensive? Is it something intrinsic to the technology that prevents costs from coming down?

diskzero 4y ago
A LIDAR sensor is a complex device with spinning motors, mirrors, lasers and more. Costs are coming down and less-expensive and more capable devices are coming to market. Once the price-points come way down, I wouldn't be surprised to see Tesla reconsider their decision to exclude them from their sensor platform.
- Nasrudith 4y ago
  They are also fairly power-hungry from what I heard.
- frxx 4y ago
  There are also solid state LIDARs these days which involve no moving parts. Still use more power than radar though.
fooblaster 4y ago
Lidar is coming down significantly in the next few years and is available from multiple tier 1 automotive suppliers. Technologies like high power vcsel arrays and highly integrated and photosensitive SPAD arrays/detector logic are making this possible. Prior lidar devices used non automotive grade discrete components like edge emitting laser diodes, high speed adcs, and APDs. These were expensive and hard to integrate, and aren't present in mass market inexpensive AM lidar coming to the market.

petilon 4y ago

I didn't find his answers particularly convincing. His answer focused on costs mainly, and how "the best part is no part". We have already seen multiple accidents caused by camera's limitations [1] which would not have happened if Tesla used Lidars. Cameras have poor dynamic range and can be easily blinded by bright surfaces. While it is true that humans do fine with only eyes, our eyes are significantly better than cameras. More importantly, expectations are higher when an automated system is driving the car. It is not sufficient if, in aggregate, self-driving cars have fewer accidents. If you lose a loved one in an accident where the accident could have been easily avoided if a human was driving, then you're not going to be mollified to hear that in aggregate, fewer people are being killed by self-driving cars! You'd be outraged to hear such a justification! The expectation therefore is that in each individual injury accident a human clearly could not have handled the situation any better. Self-driving cars have to be significantly better than humans to be accepted by society, and that means it has to have better-than-human levels of vision (which lidars provide). [1] https://www.youtube.com/watch?v=X3hrKnv0dPQ https://www.youtube.com/watch?v=X3hrKnv0dPQ

bergenty 4y ago
Not to mention waymo works well with LiDAR, cameras and radars. If you’re argument is it’s too hard to deal with that much data, it’s definitely the wrong answer.
Nomentatus 4y ago
You're right to note the advantages of lidar and (narrow range of) contrast problem for cameras (they arent eyes.) This is why the Uber human driver shouldn't have trusted the machine at night, in particular. But you still have to address his system argument, which was that adding geegaws that added little would actually increase overall risks along the supply chain (plus maintenance) while distracting the team and adding more risk that way, for very little apparent (but only apparent) gain. The team does believe that they'll get to better than human driving, and do that without lidar.
- Marazan 4y ago
  They cannot actually believe that.

0xfffafaCrash 4y ago

Seems like a very political answer from Andrej. Of course he’s not going to outright say “yeah, we’re prioritizing the profit margin over accuracy and safety considerations” if he wants to keep his job, but that seems to be the short of it. Others may choose to follow, at least in the short term, but it won’t be because of “entropy” making the system worse (you can always build a model without a data source and then refine the results based on the added value of a data source) but doing so just because it will save lives doesn’t cut it when the goal is to cut corners and costs to maximize profit. I can believe that some types of sensors aren’t worth the trouble in terms of additional signal to noise ratio, but I can’t believe this is one of them.

Nomentatus 4y ago
The entropty he's taking about comes from many sources, in particular opening yourself up to maintenance or supply side errors, and just overloading team attention (always at a premium) for no net return. It's not just CPU cycles (although that's part of it, that hardware could be doing something else useful.)

bigtex 4y ago

Did Lex ask him why Tesla love to crash into emergency vehicles?

deleted 4y ago
[deleted]

xnx 4y ago

Sensor fusion seems to be another thing that Tesla is not good at.

JaggerJo 4y ago

This sucks for parking. It is simply (physically) not possible for the existing cameras to see the area directly in front of the car. So how would this work for parking? A: Add more cameras so there are no dead areas in front of the car B: build a model in vector space when driving towards a parking spot and assume blind spots don't change. (still sucks)

oxplot 4y ago
> It is simply (physically) not possible for the existing cameras to see the area directly in front of the car. Think about how a human driver does it, given his/her even worse vantage point. They model what's in front/behind the car from afar and remember what's where as they approach it. There are other signals as well, such as continuation of a kerb, etc. I think people keep forgetting that Teslas run hundreds of ML prediction tasks all the time. Watch recent AI day and their talks about "occupancy network" to get a sense of the car's ability to: 1. Construct 3D model of its surrounding in real time; 2. Remember occluded sections based on what's it's seen previously.
- watwut 4y ago
  Human driver constantly turns head around to where he is Mos likely to hit something.
  - oxplot 4y ago
    Well, the car has 360 degree camera view, with far wider coverage than a turning head in a driver seat. And more importantly, it sees in all directions at all times.

sidcool 4y ago

I feel that was more an operational answer than an engineering one .. I still feel that depth perception of vision alone is unreliable.

post_break 4y ago

I think this is the real reason: https://www.youtube.com/watch?v=LS3Vk0NPFDE https://www.youtube.com/watch?v=LS3Vk0NPFDE Cost cutting.

taf2 4y ago
Probably a benefit but also imagine the difference in software. You Boolean logic like radar says we are gonna hit , vision sensor says we have nothing there, sonar indicator says nothing there… so the idea of having just a really good single source of truth probably makes a lot of code a lot less complex… I have no way of knowing either way but from a less is less that can break point of view this seems kinda good… like many things only time can tell and at least we have different groups of people pushing on different potentially viable paths forward so that we can soon hopefully know if self driving is possible (wide scale) one way or the other

friend_and_foe 4y ago

I remember watching an interview with George Hotz when Comma.ai was young, where he essentially said this as a critique of Tesla. He's a bit of a showman and likes to invite a little controversy when he says things, but I found myself agreeing with his point. It's not surprising to see such a practical company like Tesla face the facts about all these sensors eventually.

P_I_Staker 4y ago
> such a practical company like Tesla Where are you getting that from? Tesla has always seemed pie in the sky, and hardly a down to earth company at all throughout the history. I'm basing this one both their public record, and reputation within the auto industry.
- friend_and_foe 4y ago
  Tesla created the modern electric vehicle industry. They're innovators, sure, and they push limits, but their priority has always been to actually build. And they do.

woeirua 4y ago

Andrej's argument about more sensors adding entropy strikes me as disingenuous considering that in the next question he then says that Tesla's biggest advantage over everyone else is "the fleet", which clearly introduces orders of magnitude more entropy into the system than anything else. Can you imagine the infrastructure required to gather video from "the fleet" anytime a car sees something unexpected? How about diagnosing what went wrong in that specific instance? How many thousands of these cases do they see everyday? Given the progress of the FSD "beta" to date, and the fact that Andrej _left_ Tesla, I'd wager that he knows that this approach is a dead end, but he won't say that because he'd get himself in hot water with Elon.

dmix 4y ago
One is infrastructure entropy and the other is software engineering entropy (we're still talking about data from one type of sensor, just at larger scale). Most tech startups have 10x+ more problems with the engineering part than the infrastructure/ops part. Also this is one person's perspective from a large team. His answer might be biased because he's an engineer and I doubt his was the only voice in the debate.
ralfd 4y ago
Assuming Karpathy is lying is quite a hot take to disregard his opinion. No. He makes it clear that he is very convinced about it. There is no relativism, no weasel words or couching in maybes. He could be wrong, of course, but he believes in what he is saying.
- nicbou 4y ago
  > no weasel words The video starts with him reframing the question instead of answering it
  - epgui 4y ago
    I regularly reframe questions when I think they're not the most interesting. It's a very common thing to do, especially in academia, that doesn't particularly indicate deception.

lawrenceyan 4y ago

I can see a path where with only cameras, Tesla might be able to reach level 4 autonomy in perfect conditions. But the biggest thing that comes to mind is what happens at night. Are they only going to enable self-driving during the day?

speedgoose 4y ago
Wouldn’t turning on the headlights fix the problem at night? Snow and ice may be another challenge but night sounds easy.
- lawrenceyan 4y ago
  When you drive at night with headlights, tell me honestly how confident you feel driving versus during the daytime.

KVFinn 4y ago

TLDR: Tesla thinks LIDAR hardware is more expensive than the performance improvement it provides. I didn't like his line of logic about how vision is necessary and sufficient, because that's how humans drive. Okay sure, but if some combinations of non-human sensors could drive better and/or cheaper than a vision only driving system, surely he would not argue for sticking with vision only? Maybe adding non-vision sensors lets you save hardware and software resources on the vision part of the system.

dreamcompiler 4y ago

It's obviously a stupid decision to remove a direct source of range data (radar and ultrasound) in favor of an indirect one (vision). But on second thought this doesn't bother me that much because Tesla FSD is absolute garbage even with radar (and I don't think Tesla will get away with selling the FSD snake oil for much longer), so if vision-only is good enough for the base-level lane-keeping autopilot functionality and it makes the cars cheaper, maybe that's a good thing.

dmix 4y ago
Even though the risks are high the outcomes will always keep them honest. Whether they like it or not. This isn't like Facebook continually releasing a product that sucks but people will use anyway. Tesla is constantly working against the clock and everything they do has real world consequences. There are multiple gov agencies watching over it at all times. Of course there's lots of people with far higher risk tolerance than is being exhibited but if it does turn out badly IRL this will get shut down pretty quickly. The good news is Tesla has the ability to cripple this feature remotely without a costly/lengthy recall if that does happen.
- AlotOfReading 4y ago
  Outcomes have to be truly and utterly fucked for "new" types of problems to be noticed in the automotive industry. Take the Toyota Unintended Acceleration case, where completely negligent software quality took over a decade and at least 89 lives to be noticed and (partially) rectified. Regulators have keen noses for very particular types of issues and rely heavily on manufacturer judgements on a lot of the rest. Issues that aren't in any of those fairly narrow categories need to be extremely public or extremely egregious to attract their notice.
  - dmix 4y ago
    I’ve read into the Toyota case in the past and it doesn’t seem like it was that obvious of a regulatory failure, where you could come to such a conclusion. Did I miss some good article covering it? Because everything I’ve read with the benefit of retrospect has been pretty critical of a lot the alleged ‘cases’ reported in that 89 number. Are you sure that’s the example you want to use for pre-emptive regulatory failure? And do you think Tesla could demonstrated failures at a similar rate (assuming they were legit) and get away with it? Because I dont

danbmil99 4y ago

Musk is also famously against using lidar. He doesn't understand/accept that an autonomous vehicle needs any sensors that humans do not posess.

julienreszka 4y ago

Geohot said something similar years ago already

superkuh 4y ago

Humans don't use radar or ultrasound sense to drive. If we want cars that drive like humans drive they should use the same senses. For example, in the northern parts of the USA there is snow cover for much of the year and lanes are emergent from flocking without any absolute reference to the actual location of the lanes. The reasons everyone choses the same places to drive are that they see the same environment with the same senses. Even if autonomous driving with radar and ultrasound was made to work if it picks the correct lane position and all the humans pick the wrong new lane position then the car is wrong, not the humans.

oxplot 4y ago

Here's the summary (mixed with observations from Munro and past Tesla presentations): - Costs money: the physical sensors (a dozen of them), wiring it up, assembling it, maintain inventory, code it, etc. - Time spent on maintaining, improving software stack for the non-vision sensors as well as efforts needed to fuse the data with vision, takes away from focusing on vision alone. It also holds back vision in relevant areas. - Existing non-vision sensors used by Tesla are orders of magnitude lower fidelity than vision. It has historically (as the case with radar) led to vision essentially having to overriding radar because vision just performed much better (see AI day 2021). My take: As with any new tech, it likely sucks at the start (think HDD and SSDs, and how a mechanical thing with lots of moving parts was way more reliable than SSDs at the start). However, by essentially moving past the local maxima, you get to innovate better, faster in the future. In case of ultrasonic sensors, they are for low speed cases anyway and most people are fine without them. Majority of fatalities and injuries happen at higher speeds.

rootusrootus 4y ago
That's great for them. But when I'm shopping for a car, I get to choose between a manufacturer that installs the extra sensors and seems to be able to get them to work, and Tesla. Used to be that Tesla was blazing a trail and if you wanted a good EV, that was what you got. Now, if you want the best EV, it's usually not going to be a Tesla. And I don't see that they're making any decisions that will regain them that title. The incumbent manufacturers are quickly proceeding to eat their lunch, just like many of us predicted would happen. Turns out the hard part of making a successful car isn't the drivetrain.
- oxplot 4y ago
  > Now, if you want the best EV, it's usually not going to be a Tesla. Would love to hear what you consider "good" and what specific EV ticks the most good features that a Tesla Model 3 doesn't.
  - lnsru 4y ago
    360 degree camera on BMW i4. Overall build quality too.

smrtinsert 4y ago

Hm I'd rather have someone from Twitter audit this decision

gnicholas 4y ago

How does it make sense to not even have sensors for parking? If you think they don't help during normal-speed driving, that's one thing. But they obviously help during parking, since (IIRC) they've had to disable certain autonomous features until they get their vision-based systems upgraded to be able to fill in this gap.

ornel 4y ago

Video summary: https://www.summarize.tech/www.youtube.com/watch?v=_W1JBAfV4Io https://www.summarize.tech/www.youtube.com/watch?v=_W1JBAfV4...

nelox 4y ago

“The world is designed for human visual consumption” and “[vision] has all the information you need for driving”. While vision may be sufficient, I would say that other senses, such as hearing, touch and smell augment driving very well. Especially with regard to situational awareness. e.g. The sirens of emergency vehicles are typically the first indication of their presence, which often can be felt as well. The wail of a tornado siren, similarly. Loud throbbing motorcycles do much to improve rider safety simply due to them broadcasting their presence. At railway level crossings, drivers should slow down, look and listen for oncoming trains. The smell of wildfire or bushfire smoke provides enhanced warning or nearby danger. So to say vision is sufficient, does not fully take in to account the driver experience, especially where safety and situational awareness are concerned.

djleni 4y ago
THANK YOU. Reading this thread was getting to me because so many comments say humans drive eyes only. I use far more than just vision driving: - sound, for emergency vehicles, detecting vehicles outside of my field of view if my windows are down or the vehicle is loud, tire sound (especially in snow and rain), engine sound (more feedback in snow or ice about what my tires are doing) - touch (steering feedback, gives information about grip in some circumstances) - acceleration (can feel if the rear tires break loose in a turn on snow or ice, or if I’m sliding while breaking) And probably many more
- mola 4y ago
  We even use Doppler! Our hearing is capable of sensing movement (speed and acceleration) using the Doppler effect. Our hearing also has a remarkable ability to locating sound source incoming direction.
- AlotOfReading 4y ago
  It's worth noting that most autonomous vehicle solutions have dedicated microphones for emergency vehicles and sensors that can detect slip. I had a little dashboard measuring wheel slip at <company>. It mainly ended up mapping train tracks and potholes.

hbarka 4y ago

Intuition + other examples tell you that radar and ultrasonic sensors work. Why do we twist ourselves to believe otherwise? Elon removed the radar and ultrasonics for the simple fact that its supply chain logjam was screwing up the manufacturing schedule. They also realized that the profit margin can be sustained in an inflationary environment by simply removing these parts [1]. “Oh, we were going to remove them anyway because humans can see fine with just eyes and no radar, why can’t cars?” Tesla then turned up the marketing of the AI/vision hype lever once more to toss another shiny tech object and get buyers to ignore the fact that there is a regression of features in the newer cars going forward. [1] https://youtu.be/LS3Vk0NPFDE https://youtu.be/LS3Vk0NPFDE

mgoetzke 4y ago
So all those engineers are lying when they talk about this topic ?
- 9935c101ab17a66 4y ago
  Uh, which engineers are you referring to?

EVa5I7bHFq9mnYK 4y ago

From the first principles point of view, it comes down to radar and ultrasonic having much higher wavelengths than optical. Which results in much lower amount of incoming information, worse resolution and higher interference if many cars radiate the same signals on a busy street.

jakeogh 4y ago

Giving human[1][2] drivers better situational awareness[3] is the future. Specifically open[4]: a. Windshields that clean the inside as well as the outside. b. Better eyeglasses[5]. c. User controllable hi-res HUD thermal IR overlay. d. Headlights with adaptive notch filters so the oncoming vehicle can pick an empty spectral range... without the source being monochromatic (with required adaptive filters on the recieving end)... and/or really good coronagraph's. e. Brake control[6]. Any entity capable of driving[7] in a population of humans (including adversarial humans) is sentient[8], and has real skin in the game. It would be unethical to lock one in a car: [1] https://news.ycombinator.com/item?id=33213860 https://news.ycombinator.com/item?id=33213860 (analog FPGA) [2] https://news.ycombinator.com/item?id=21106367 https://news.ycombinator.com/item?id=21106367 (general AI) [3] https://news.ycombinator.com/item?id=16646112 https://news.ycombinator.com/item?id=16646112 (2018) [4] https://www.tesla.com/blog/all-our-patent-are-belong-you https://www.tesla.com/blog/all-our-patent-are-belong-you (2014) [5] https://patents.google.com/patent/US7744217 https://patents.google.com/patent/US7744217 (2007) [6] https://news.ycombinator.com/item?id=18013388 https://news.ycombinator.com/item?id=18013388 (2018) [7] no human behind the wheel, no human to correct impending mistakes, but (critically) with one or more humans in the car. [8] The idea that non-biological machines can have 'self' is a window into modern mass transformation. Please checkout the analog FPGA experiments linked above.

sgjohnson 4y ago

Does this mean that now when someone smashes one of their bumpers on a Tesla, the insurance will no longer have to total the entire vehicle?

mavili 4y ago

Did anyone else catch Andrej's "sqlite" comment? If that is not just a simple analogy, Tesla may be using sqlite in their cars? :D

ra7 4y ago

If I were a Tesla fan/investor/FSD customer, I’d be very concerned that the former (effective) tech lead of FSD doesn’t know about sensor fusion or that it’s a solved problem for majority of the companies in this space.

Animats 4y ago

From my DARPA Grand Challenge days, I used to have an Eaton VORAD automotive radar. This was an early design - 24GHZ, 1 scanning axis. It could see cars, but not bicycles, at least not reliably. For several months, I had one pointed out the window of my house, looking at an intersection. So I had a V-shaped wedge on screen, and could watch the cars go by. It's a Doppler radar, so you don't get any info from things stationary relative to the radar, but you do get range and range rate. And the quality of that data is independent of distance. We used it mainly as a backup system for the world model built with LIDAR and (to a very limited extent) vision. The VORAD data could lower the speed limit for the rest of the system, and if a collision was about to happen, it would slam on the brakes independently of the world model. The big problem with coarse automotive radar is that it can detect targets, but doesn't tell you much about them. Cars, trash cans, and metal road debris all look about the same. There's also a lot of trouble from big flat metal surfaces being mirrors for radar. We were willing to accept slowing down for ambiguous cases until the other sensors could get a good look. Drivers hate that if road-oriented systems do it. Modern units are up around 70-80GHz and often have 2D scanning, which is a big help. I haven't seen the output from a modern automotive radar. I was expecting that by now, low cost millimeter microwave systems (200-300GHz) would be available, providing detailed images somewhat coarser than you can get with light. You get range and range rate, and you can usually steer the beam electronically rather than mechanically. The technology exists to get high-resolution radar images, but is mostly used for scanning people for weapons at checkpoints. It hasn't become cheap yet.

nielsbot 4y ago

All I heard was "cost savings, cost savings, cost savings"

oxplot 4y ago
Well watch it again and again and again. He talks about the determental effect of lo-fi sensors in conjunction with vision among other things.

60Vhipx7b4JL 4y ago

From an engineering perspective I would ask: Can your sensor package understand the environment to the required (low) failure rate? Radar/Lidar/Ultrasonic is going to give you information that your camera systems will not give you. It does not matter if the delta of information is little. If this little is required because you can't obtain it otherwise, you still need it. If you just rely on the fleet, you rely on the things you have seen. What about the objects that you have not yet seen?

dncornholio 4y ago

Just remember folks, we will have full self driving vehicles by the end of this year!

throwaway4good 4y ago

Let me reframe the answer: "We removed them because they cost money. And we are trying to make money ... at least right now. Listen, this pure autonomous self-driving car stuff is never going to work, so who cares if we have these gadgets or not ..."

nova22033 4y ago

At 2:05. Suddenly you need a column in your sqlite telling you what type of sensor it is.... Seriously? This is a major technical challenge?

danpalmer 4y ago
The challenge isn't the storing of the flag that says which sensor it has, it's testing the combinations, training for the different scenarios, treating the incoming data differently, and so on.

deleted 4y ago

[deleted]

solardev 4y ago

Cuz Muskdaddy wanted mo money. There, mystery solved.