6 ms·

Karpathy gave a talk about radar - they were getting worse results with radar than without (according to him). Mostly because they would get false readings fro

Karpathy gave a talk about radar - they were getting worse results with radar than without (according to him). Mostly because they would get false readings from radar shooting under the car and stuff like that - vision alone was more accurate.

ryanwaggoner 4y ago
If you're talking about his appearance on Lex's podcast, he didn't say this. He said that the additional accuracy wasn't worth it. Which to me sounds like it was more accurate, but they didn't want to deal with the additional cost and complexity and thought that vision only would be good enough. But the link we're commenting on, along with the ongoing complaints about FSD, seem to indicate that the real world results haven't proven that theory out.
- JoshCole 4y ago
  He is talking about the long technical talk at CVPR. https://www.youtube.com/watch?v=g6bOwQdCJrc https://www.youtube.com/watch?v=g6bOwQdCJrc For example look at this section: https://youtu.be/g6bOwQdCJrc?t=1370 https://youtu.be/g6bOwQdCJrc?t=1370 Also if you look into it you will find Google's self-driving car project ran into similar issues. They upgraded the radar in response to the issues rather than removing it, but there are actual issues. Vision only was better than radar plus vision in their stack. For real world situations that frequently come up in driving radar wasn't worth it. The theoretical accuracy of the sensor fusion didn't happen in practice because in practice you needed to have vision tell you when sensor fusion wasn't going to be accurate. It isn't naive sensor fusion as in toy math models of sensor fusion. So when there was a disagreement you needed to go with vision. Yet vision was already able to predict both the failure of and the actual result of radar. In theory, you could just have done the more complicated sensor fusion. In practice, the relative advantage of improving other parts of the vision stack far exceeded improving the sensor fusion. Over time one would expect the relative advantage to change as other improvements were made such that this was no longer true. That doesn't mean the time at the decision wasn't correct, it just means people who evaluate it with future changes already in place are suffering from hindsight bias/committing an anachronistic fallacy.
  - gonehome 4y ago
    Yep - this is the one I was talking about, thanks for linking!
  - ryanwaggoner 4y ago
    Yeah, Google actually made the right call. Thinking that Tesla made a stupid and dangerous call isn't "hindsight bias", their vision-only approach has always been inferior and reckless.
    - JoshCole 4y ago
      The practical impact of the change you call stupid and reckless was that the car stopped slamming its brakes when going under underpasses. So when it comes down to it what you are claiming is that it is stupid and reckless to decide not to slam on the brakes. Look - I get it. I too thought radar sensor fusion would improve the results. Guess who else thought this? The people you consider stupid and reckless thought this. Do you know why they changed their mind? According to you, because they are stupid and reckless. I can't stress this enough: you are telling yourself inaccurate stories about what happened. If you just /look/ at the /measured impact/ instead of /wrongly guessing what Karpathy thought/ and /wrongly guessing what Karpathy has said/ and /wrongly guessing what other people were referencing when they talked about what Karpathy said/ then you would realize this.
      - ryanwaggoner 4y ago
        Haha, it’s funny that you circled back a day later to leave another comment lambasting me. It’s so naive to just take Tesla’s word for all of this, and to believe that they took this approach because they decided it was the safest and most effective. Meanwhile, other credible teams out there have not gone with vision-only, and as a result their systems aren’t a pile of dangerous dogshit that are killing customers. But Tesla is incredibly arrogant and reckless, and they just keep doubling down. Vision-only is the wrong choice, it won’t get us to L4, and that should have been pretty easy to see from the start.
pclmulqdq 4y ago
Later, Musk came out and said that they were using cheap radar sensors, and that higher-quality radar sensors would likely have been better than vision alone.
- judge2020 4y ago
  But radar only does anything for detecting cars in front of you, so the only deficiency by removing it is using vision to perform that distance-keeping when using traffic-aware cruise control, which is not a pain point in my 2022 M3LR even in the rain.
  - saurik 4y ago
    But it used to be better: it was able to follow closer to the car in front--which is really important when you are in stop-and-go 3mph traffic on the highway--and it was supposedly able to see under the car in front of you (reflecting off the highway) to track the car ahead of them <- a friend of mine who's entire family is super into Tesla has all cars with the radar sensors, and when I was driving her around she was commenting about how many dumb decisions the newer Model 3 I have been renting was making, seemingly because of this. She was not happy about it when she heard Tesla started deactivating radar in old cars. But yeah: the feature minimally seems to work well enough--...even surprisingly well!--given the lack of having the sensors that Tesla previously swore were amazing, and so in some sense that particular cost cutting measure was a success. I was even thinking of buying one to own... but then they removed the parking sensors, and frankly that feature only barely works WITH the sensors--I drive a ton of random cars as I am always renting stuff, and the Tesla's parking radius feature is a cool UX but has tons of slop and has massively high latency... it is frankly just super dumb--and I cancelled my order when it finally arrived a week after it was for sure I wouldn't have the sensors. Regardless, though, even if you feel it works perfectly, radar was another situation of just hastily scrapping stuff and working on it in the field, as they removed the sensor before they had the software which was tolerable, and then they just kept promising people it was going to get better as they frantically tried to turn their marketing into reality. Over time it got better and after a long time it finally started to work at higher speeds, but for a long time it had a cap lower than the in-practice speed of even my nearby highways here in the United States, and still hasn't managed to regain all of the functionality of the prior design. Can you imagine Apple removing a sensor in a new iPhone without already having the software to replace it, and then just promising new customers they'll figure it out in the hope it doesn't affect sales?

ryanwaggoner 4y ago

If you're talking about his appearance on Lex's podcast, he didn't say this. He said that the additional accuracy wasn't worth it. Which to me sounds like it was more accurate, but they didn't want to deal with the additional cost and complexity and thought that vision only would be good enough. But the link we're commenting on, along with the ongoing complaints about FSD, seem to indicate that the real world results haven't proven that theory out.

JoshCole 4y ago
He is talking about the long technical talk at CVPR. https://www.youtube.com/watch?v=g6bOwQdCJrc https://www.youtube.com/watch?v=g6bOwQdCJrc For example look at this section: https://youtu.be/g6bOwQdCJrc?t=1370 https://youtu.be/g6bOwQdCJrc?t=1370 Also if you look into it you will find Google's self-driving car project ran into similar issues. They upgraded the radar in response to the issues rather than removing it, but there are actual issues. Vision only was better than radar plus vision in their stack. For real world situations that frequently come up in driving radar wasn't worth it. The theoretical accuracy of the sensor fusion didn't happen in practice because in practice you needed to have vision tell you when sensor fusion wasn't going to be accurate. It isn't naive sensor fusion as in toy math models of sensor fusion. So when there was a disagreement you needed to go with vision. Yet vision was already able to predict both the failure of and the actual result of radar. In theory, you could just have done the more complicated sensor fusion. In practice, the relative advantage of improving other parts of the vision stack far exceeded improving the sensor fusion. Over time one would expect the relative advantage to change as other improvements were made such that this was no longer true. That doesn't mean the time at the decision wasn't correct, it just means people who evaluate it with future changes already in place are suffering from hindsight bias/committing an anachronistic fallacy.
- gonehome 4y ago
  Yep - this is the one I was talking about, thanks for linking!
- ryanwaggoner 4y ago
  Yeah, Google actually made the right call. Thinking that Tesla made a stupid and dangerous call isn't "hindsight bias", their vision-only approach has always been inferior and reckless.
  - JoshCole 4y ago
    The practical impact of the change you call stupid and reckless was that the car stopped slamming its brakes when going under underpasses. So when it comes down to it what you are claiming is that it is stupid and reckless to decide not to slam on the brakes. Look - I get it. I too thought radar sensor fusion would improve the results. Guess who else thought this? The people you consider stupid and reckless thought this. Do you know why they changed their mind? According to you, because they are stupid and reckless. I can't stress this enough: you are telling yourself inaccurate stories about what happened. If you just /look/ at the /measured impact/ instead of /wrongly guessing what Karpathy thought/ and /wrongly guessing what Karpathy has said/ and /wrongly guessing what other people were referencing when they talked about what Karpathy said/ then you would realize this.
    - ryanwaggoner 4y ago
      Haha, it’s funny that you circled back a day later to leave another comment lambasting me. It’s so naive to just take Tesla’s word for all of this, and to believe that they took this approach because they decided it was the safest and most effective. Meanwhile, other credible teams out there have not gone with vision-only, and as a result their systems aren’t a pile of dangerous dogshit that are killing customers. But Tesla is incredibly arrogant and reckless, and they just keep doubling down. Vision-only is the wrong choice, it won’t get us to L4, and that should have been pretty easy to see from the start.

pclmulqdq 4y ago

Later, Musk came out and said that they were using cheap radar sensors, and that higher-quality radar sensors would likely have been better than vision alone.

judge2020 4y ago
But radar only does anything for detecting cars in front of you, so the only deficiency by removing it is using vision to perform that distance-keeping when using traffic-aware cruise control, which is not a pain point in my 2022 M3LR even in the rain.
- saurik 4y ago
  But it used to be better: it was able to follow closer to the car in front--which is really important when you are in stop-and-go 3mph traffic on the highway--and it was supposedly able to see under the car in front of you (reflecting off the highway) to track the car ahead of them <- a friend of mine who's entire family is super into Tesla has all cars with the radar sensors, and when I was driving her around she was commenting about how many dumb decisions the newer Model 3 I have been renting was making, seemingly because of this. She was not happy about it when she heard Tesla started deactivating radar in old cars. But yeah: the feature minimally seems to work well enough--...even surprisingly well!--given the lack of having the sensors that Tesla previously swore were amazing, and so in some sense that particular cost cutting measure was a success. I was even thinking of buying one to own... but then they removed the parking sensors, and frankly that feature only barely works WITH the sensors--I drive a ton of random cars as I am always renting stuff, and the Tesla's parking radius feature is a cool UX but has tons of slop and has massively high latency... it is frankly just super dumb--and I cancelled my order when it finally arrived a week after it was for sure I wouldn't have the sensors. Regardless, though, even if you feel it works perfectly, radar was another situation of just hastily scrapping stuff and working on it in the field, as they removed the sensor before they had the software which was tolerable, and then they just kept promising people it was going to get better as they frantically tried to turn their marketing into reality. Over time it got better and after a long time it finally started to work at higher speeds, but for a long time it had a cap lower than the in-practice speed of even my nearby highways here in the United States, and still hasn't managed to regain all of the functionality of the prior design. Can you imagine Apple removing a sensor in a new iPhone without already having the software to replace it, and then just promising new customers they'll figure it out in the hope it doesn't affect sales?