13 ms·
Karpathy's presentation is really good. Watch it later if you get some time. The key points are - Telsa is using fleet for learning (Fleet learning). This has
by sairahul82 7y ago
Karpathy's presentation is really good. Watch it later if you get some time. The key points are
- Telsa is using fleet for learning (Fleet learning). This has 3 components,
a. Trigger infrasture that collects the kind of data telsa is looking for training. The rational here is we don't need massive amounts of same kind of data but needs right kind of data for training neural networks correctly.
b. Data Engine which learns from these examples.
c. Shadow mode where they deploy the learned model and test how its doing in real world and iterate this process.
- Drivers are themselves acting as labelers and tesla is in a unique position to take advantage of this.
- People drive vision only. Visual recognition is essential for autonomy. Lidar has much less information than vision, for example to identify the thing on the road is a plastic bag, lidar provides few points. But vision gives more data for telling this.
- slg 7y ago>Karpathy's presentation is really good. Watch it later if you get some time. I would second this recommendation. He did a really good job not only explaining Tesla's approach, but also breaking down the concepts in an easy to understand way for laypeople. That is a rare skill to have especially among someone who is obviously so technically accomplished.
- chibg10 7y agoBased on what I know about how high-level executive meetings and presentations (which is the role Karpathy is playing here) go, there's a pretty good chance the presentation was not actually developed by Karpathy but a Tesla team of significant size.
- DeonPenny 7y agoIt was great including his explanation of how Tesla drives in snow and rain which is hard to understand how a car can do it.
- tjoff 7y agoDoes anyone doing lidar not also do visual recognition?
- sairahul82 7y agoI don't think any one is doing this. This is the point Karpathy is making. Vision is essential and needs to be perfect and also has all the information to recreate the 3d environment. That's why lidar is not required and is expensive.
- michaelt 7y agolidar is not required and is expensive. Given that the only the company claiming that is the one whose cars sometimes hit (or even swerve into) clearly visible stationary concrete barriers in broad daylight [1, 2, 3, 4], I consider that claim far from proven. [1] 2019 https://arstechnica.com/cars/2019/03/dashcam-video-shows-tesla-steering-toward-lane-divider-again/ https://arstechnica.com/cars/2019/03/dashcam-video-shows-tes... [2] 2018 https://news.ycombinator.com/item?id=16772748 https://news.ycombinator.com/item?id=16772748 [3] 2017 https://bgr.com/2017/03/02/tesla-crash-video-texas/ https://bgr.com/2017/03/02/tesla-crash-video-texas/ [4] 2016 https://www.thesun.co.uk/news/1402992/tesla-self-driving-car-involved-in-another-crash-as-gallery-owners-model-x-flips-in-collision-with-concrete-barrier/ https://www.thesun.co.uk/news/1402992/tesla-self-driving-car...
- SheinhardtWigCo 7y agoThis argument misses the forest for the trees. Even level 5 autopilot will sometimes crash into objects that are clearly visible to most humans, as will lidar-based solutions. But if the robot fleet is an order of magnitude safer than the human fleet, then it makes rational sense to prefer a robot driver to a human.
- michaelt 7y agoI think you might have replied to the wrong post here? I didn't say anything about humans vs self-driving cars. I'm arguing about self-driving cars with LIDAR vs those without.
- torpfactory 7y agoDoes anyone know how Lidar would work in the future when many other vehicles are also using lidar? From working with some different types of photo sensors (CCD, avalanche photodiodes, photomultiplier tubes), you might imagine that incident light from another car's emitter would be much brighter than the reflected light the lidar sensors are reading and cause saturation.
- toomuchtodo 7y agoPhase lock or introducing jitter to laser timings. Check out Animats' comments on this topic, it's where I've learned from regarding this.
- torpfactory 7y agoThanks!
- salty_biscuits 7y agoI've only got experience with aerial survey lidar, but "blinding" the photomultiplier with your own transmit pulse is pretty common, e.g. hitting something retroreflective like a cats eye marker or a street sign. Even the sun off a puddle can do it too. It is a pretty obvious artifact to recognize and remove. I guess it would depend on the mechanism of the lidar (i.e. if it is scanning then what is the probability that the different cars are scanning at each other at the same time, probably low). Might be a problem if they use flash or geiger mode lidar. But then the pulse rate would be lower. Interesting to think about. I'm sure the people working on it would have an answer.
- CamperBob2 7y agoProbably the same way GPS satellites avoid jamming each other: modulate the lidar transmitter with a unique coded sequence that can be detected by correlation.
- b_tterc_p 7y ago> The rational here is we don't need massive amounts of same kind of data but needs right kind of data for training neural networks correctly. This is really important. At a certain point additional miles on similar roads in nice conditions with fair traffic won’t help anymore. The data variety is really important.
- millettjon 7y agoThey also probably want to learn from the best human drivers and not repeat the mistakes of the average.
- mzs 7y agoThe livestream is gone though https://youtu.be/Ucp0TTmvqOE https://youtu.be/Ucp0TTmvqOE
- Maxious 7y agoOther channels have recorded it https://www.youtube.com/watch?v=tbgtGQIygZQ https://www.youtube.com/watch?v=tbgtGQIygZQ
- krzkaczor 7y agoWhat I found interesting as well is that they (him and Elon) hated the idea of using high definition maps, saying that if you need them you don't do self-driving cars really. For example, when driving condition change (roadblock or anything) you're done.
- RivieraKid 7y agoIf this is true, why do Waymo (or any of the top 10 self-driving companies besides Tesla) use Lidar? Two facts: 1. Tesla can't use Lidar because of cost and aesthetics. 2. Elon has a strong incentive to oversell their approach to self-driving.
- wrs 7y agoFrom what I can see, overselling is Elon’s standard strategy for whatever technical approach he’s decided is the right one. (And I can’t really say that doesn’t work for him.)
- krzkaczor 7y agoAs far as I understood HD maps are static data obtained by many various ways (not only lidar)[1]. AFAIR they claimed that Lidar itself provides some value (teaching neural networks how to measure the distance to various objects etc) but the gist is that image recognition is an absolute must-have for self-driving cars (which seems right to me). 1 - https://medium.com/@LyftLevel5/https-medium-com-lyftlevel5-rethinking-maps-for-self-driving-a147c24758d6 https://medium.com/@LyftLevel5/https-medium-com-lyftlevel5-r...
- salty_biscuits 7y agoApart from the map data going stale as it gets out of date, you always have the problem of registration of the vehicles position within the map. GNSS data can have a really long tailed error distribution (particularly within urban canyons and in tunnels). So you always need some way of making a local map and placing it within the high detail global map. There is a pragmatic argument saying if you already have a detailed local map, do you really need the global map apart from high level navigation tasks.
- bogomipz 7y ago>"People drive vision only." Can you elaborate on what you mean here? Do you mean that Tesla will only be using Computer Vision for autonomy? Doesn't or wouldn't Lidar compliment CV? Or is there a practical design consideration that makes them mutual exclusive?
- emef 7y agoTesla has take a firm stance against using LiDAR and is doubling down on CV. I'm not sure if they have radars for parking assistance, but they are using cameras as primary sensor for perception and localization.
- threeseed 7y agoThat people drive vision is so idiotic. Humans have far higher resolution sensors and the most advanced computer ever seen behind them. And even that fails at a far higher rate than we want self driving cars to be.
- XorNot 7y agoConversely computers don't get tired, or "distracted", or drunk. They're also faster - human cognition is running about 150-200ms behind what you actually see. Some reflexes are running faster then that, but overall the human platform is not being used with that super-computer in full-control while driving at high speed - our hardware isn't capable of it. The thing we most bring to driving is navigating complex, low speed environment changes - but we're not great at that either (see the number of toddlers run over in their drive ways, for example).
- X6S1x6Okd1st 7y agoThe resolution outside of the fovea is pretty poor. It's a little unclear what portion of accidents we could cut out if we had human level performance, but never let the attention waver or disallowed reckless driving. Judging from NMVCCS it'd cut out somewhere between 34 - 75% of accidents. https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/811059 https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...
- ibeckermayer 7y ago> People drive vision only. No we don't. We have vision which we then interpret with our _minds_. We have the ability to _understand_ what we are seeing, reason from it, and make decisions accordingly. "Deep Learning", no matter how big your dataset, fundamentally cannot possess this capability. Which goes a long way into explaining why Tesla's "self driving" tech keeps on killing people in situations that are trivially simple for humans. There's no way to rigorously test for edge cases in opaque deep learning algorithms, nor is there any way to implement redundancy as there is in ordinary deterministic systems. This is a mad attempt to replace human cognition with "AI".
- paulrosenzweig 7y ago> "Deep Learning", no matter how big your dataset, fundamentally cannot possess this capability. Why not? I’d agree that current models can’t, but what’s the fundamental shortcoming of deep learning?
- ibeckermayer 7y agoThe fundamental shortcoming is that there can be no understanding, and therefore no judgement, because there is no mind. See Searle's [in]famous Chinese Room Argument (https://plato.stanford.edu/entries/chinese-room/ https://plato.stanford.edu/entries/chinese-room/). Or for a more concrete example from today: https://twitter.com/mit_csail/status/1121447605412741120?s=21 https://twitter.com/mit_csail/status/1121447605412741120?s=2...
- lojack 7y agoI question how much of our driving is logic and analytics based as opposed to instinct and reaction based.
- sytelus 7y agoKarpathy is great at presenting executive-101. However self-driving is much much more than image classification. You have a time component, you need to integrate information as it comes in to meaningful state (aka SLAM) and make sequential decisions that has potential to kill people. This is still unsolved research problem unlike punny ImageNet classification. Karpathy could have easily spent entire day talking about current state of the art and how they have improved on it but instead we got Deep Learning 101. I hope he didn't intentionally used this tactic to avoid talking about the elephant in the room. It seems they are hell bent on vision-based end-to-end learning which vast majority of experts would agree that is far away in the future. So this whole thing is quite bold and if they really make this happen in 1yr3mo then I'd say you should dump your entire 401K in buying Tesla stock. Collecting data is actually more trivial part of this problem.
- p1esk 7y agoThis is still unsolved research problem Luckily, Karpathy is pretty good at solving research problems. Karpathy could have easily spent entire day talking about current state of the art and how they have improved on it Why would he do that? To help Uber? Collecting data is actually more trivial part of this problem. For Tesla, yes. For all others that’s the hardest part.
- eanzenberg 7y agoMaybe when Tesla cars stop driving into medians at full speed...
- adrianmonk 7y agoWhen they say that collecting data is the trivial part, they mean that (in comparison to the other parts of the problem) it is easy for everyone to solve. In other words, they are saying Tesla has an advantage in area that doesn't matter.
- tomp 7y agoIt’s “easy” in the sense that reusable rockets are “easy” - they’re possible, but noone else is working on them. Musk was way ahead of time, again.
- Animats 7y agoIt was significant that the brief bits of video showed the system identifying "driveable" areas, shown in blue. That's important. The first step is to conservatively mark driveable areas, where you can physically drive. Then you get to plan where to go. Amusingly, if you wanted to train a vision system to recognize driveable areas, the straightforward approach would be to use a LIDAR system to measure road flatness. Watching the road go by on video as a human driver drives is not very useful for this. That just tells you about the part of the road a human driver chose to use, not about the road geometry itself. The driver probably won't use the road shoulders, but the drivable area system needs to evaluate them. Recognizing drivable areas is what keeps you from hitting solid objects. You don't have to classify them, just note that they're not flat road.