8 ms·
What Self-Driving Cars See
- candiodari 9y agoThis video convinced me that self-driving cars are definitely going to beat humans at safety. Freaky how early the thing gets that there's going to be a crash and the distance at which it responds: https://www.youtube.com/watch?v=WZ-d9k6JFA8 https://www.youtube.com/watch?v=WZ-d9k6JFA8
- joering2 9y agoCan Lidar distinquish between an empty plastic white bag dancing on the wind in the middle of the highway, and a baby that fell out of a stroller? In both scenarios whether it can or cannot, its a distaster.
- dingo_bat 9y agoMany humans also won't be able to distinguish between inanimate objects and babies/animals on the road. When there isn't sufficient data, there isn't sufficient data. Machine learning is not magic.
- make3 9y agoyes, it definitely can. deep learning is what is used in these things, and identifies objects and their properties
- Piskvorrr 9y agoYou are taking it for magic. Do not. http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.html http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.ht...
- tbabb 9y agoI don't know why this is getting downvoted. That's on of the harder problems in vehicle autonomy.
- joering2 9y agoI wouldn't worry much. I have a few "fans" that follow my comments regularly and mass down-vote me. I also realized lately that my own upvote/downvote does not get accounted for.
- AstralStorm 9y agoBetter than a human driver can detect a cat in the road. In fact a failure where it slows until it confirms with another sensor (probably optical) is ok. Many a driver won't even see the object and will drive though it.
- Someone 9y agoThat problem is hard, but because of cultural norms not as hard as it seems at first sight. You rarely will find that baby without a stroller next to it, and without a human close by hurrying towards it. So, if you avoid hitting that stroller and that human, and, next, avoid almost hitting that human, you probably are safe. A jury wouldn't easily convict you for accidentally hitting a baby that fell out of a stroller on the middle of the road and then was left alone without any adult supervision. I would think the difference between a solid object such and a piece of concrete or steel girder and various low-density objects (plastic bags, empty carton boxes) is harder. On the other hand, I guess it might be easier than laymen would think to use machine learning to build a "what's the density of the object on this photo" program that is in par with humans.
- axiom92 9y agoFrom the comments: "It has been brought to my attention, that some of these clips may be fake. A new video has been made that eliminates the fake clips https://www.youtube.com/watch?v=rphN3R6KKyU https://www.youtube.com/watch?v=rphN3R6KKyU ".
- oculusthrift 9y agowatching these videos just freaks me out at how bad human drivers are.
- zkms 9y agoNot to detract from the fact that autonomous driving has the potential to be safer than humans (if only because it doesn't get distracted), but none of these autonomous responses are "freakishly early". I've seen attentive human drivers respond with similar or better performance. As a matter of fact, let's examine how human drivers responded to one of the collisions in this video. At 00:28.828, a human could tell that an accident is imminent -- the oncoming car will violate the median should it not apply immediate corrective action. However, other drivers are not obligated to wait until the aggressor vehicle crosses the median to take evasive action! Indeed, the silver vehicle (ahead of the dashcam'd vehicle) begins decisive and correct evasive action at 00:28.328. That vehicle maintains the same steering angle and at 00:29.295 crosses the solid white shoulder/"fog" line -- it's unambiguously an evasive action and absolutely not a normal lane change. The autonomous vehicle only begins to brake/turn at 00:30.130 (the first frame where the car begins to pitch forwards due to the weight transfer). The autonomous vehicle has begun to take evasive action a full 1.8 seconds later than the silver non-autonomous vehicle. Awkwardly, the Tesla's reaction seems to be triggered not by the median-violating vehicle but by the white vehicle (who is attempting to take evasive action) cutting it off. Worse, the silver vehicle is already crossing the shoulder line a full second before the Tesla begins to brake/turn. This isn't even commensurate with human performance -- the two human drivers visible in the dashcam video react significantly earlier than the autonomous vehicle. It's absolutely possible to make vehicles that respond quicker and more effectively to impending collisions than humans do. Sensing with radar and IR can give autonomous vehicles information that human eyes simply can't detect -- IR is useful for detecting human presence, for example, and radar can be used to detect positions of cars beyond-line-of-sight. Better sensing translates to earlier warning. Also, when the computer has full control authority over individual wheel torques there's a whole new range of operating regimes that get opened up -- and autonomous vehicles that exploit those have the potential to make cars much safer (see research like http://news.stanford.edu/2015/10/20/marty-autonomous-delorean-102015/ http://news.stanford.edu/2015/10/20/marty-autonomous-delorea...). This sort of tactic can use the tyre grip to the absolute limit can let the computers craft the optimal (given constraints such as the vehicle's position, surrounding vehicles/barriers, and tyre/road conditions) trajectory out of a collision and execute it with precise per-wheel torque control (via the ABS/ESC system) and high-speed steering input. Even the best drift drivers don't have that level of control authority over their cars! There's lots of room to make autonomous vehicles safer than human drivers but this video is a demonstration that autonomous vehicles can be a lot worse than human drivers.
- deepnotderp 9y agoLIDAR is quite interesting. If there's anyone who has more information about automotive lidar, I'd love to chat with them, my email is visible on my tag :) It'll be interesting to see how SDC makers handle the computational complexity of deep nets operating on these massive point clouds generated from LIDAR. It seems like these aren't getting any easier with Luminar claiming ~10 million points. Running that through a hefty 3D ConvNet could easily soak up a petaflop...
- argonaut 9y agoTo my limited knowledge, they don't. Deep nets are mostly used by the self-driving companies for processing camera data for image detection and segmentation tasks like detecting other vehicles, recognizing street signs, etc. Presumably, having 3D point clouds lets you hand-code solutions to a lot of these tasks without having to use ML.
- deepnotderp 9y agoThat's been my experience as well, I was just wondering if anything has changed in the year or so that's passed.
- nerfhammer 9y agoDoes any of this work in rain or snow?
- keerthiko 9y agoLIDAR is supposed to have low attenuation in water, however it will still suffer refraction. Hopefully it will function about as well as a human. The vision processor will have to understand that it's raining and correct for refraction where possible. Snow is a different challenge, but a sufficiently smart system should be able to composite frames and appropriately fill in the holes created by being obscured by snow. It won't be trivial, but it should all be doable.
- tim333 9y agoYes, kinda https://qz.com/637509/driverless-cars-have-a-new-way-to-navigate-in-rain-or-snow/ https://qz.com/637509/driverless-cars-have-a-new-way-to-navi...
- ruleabidinguser 9y agoSorry, I know very little about this stuff, but how does LIDAR avoid interference with other cars emitting LIDAR?
- brians 9y agoCoded pulses: rather than emitting a uniform beam, they emit a pulse and listen for that back. You can think of it a little like a bar code, a sequence of short and long, on and off. Then select for that on receive.
- rubicon33 9y agowhat happens if another source uses the same pattern?
- DaiPlusPlus 9y agoLIDAR units could cycle through randomly-generated patterns for each pulse - if they can be made unpredictable then it would be impossible for someone to confuse a LIDAR unit by copying its pulses. Assuming it's a time-of-flight LIDAR then the time required for an attacker to read a pulse and send back their own signal would be greater than the time of the original signal to return - and if the sending LIDAR has a sufficiently short timeout then even an attacker using Vanta-black to absorb all of the victim's lasers still won't be able to respond quickly enough.
- Judgmentality 9y agoNobody likes to talk about it, but they don't. It turns out there are some clever tricks you can do if you control all the cars, but if not you have to get competitors to agree to some protocols.
- CamperBob2 9y agoWhat's wrong with modulating the beam with a PRN code that's unique to each vehicle?
- friedman23 9y agoI really do not believe lidar is going to be the right answer for self driving cars. We already know that you can be successful at driving a car in various weather conditions with two stereoscopic cameras on a swivel and limited sonar (humans). Cameras, infrared cameras, and sonar will likely be much cheaper and just as capable as lidar.
- irq11 9y agoIf you think humans are "two stereoscopic cameras and a limited sonar", you need to learn more about humans. Not only are our eyes far more versatile than the most sophisticated camera equipment, we constantly maintain a mental model of the world that allows us to focus only on the things that matter, and accurately infer state. This affects everything from vision, to hearing, to proprioception and situational awareness. Machines don't come close, which is why the quality of the sensors matters so much more. This trope gets repeated in nearly every thread about self-driving cars, and reveals mainly that engineers don't know much about biology.
- madeofpalk 9y ago> reveals mainly that engineers don't know much about biology. reveals mainly that people who don't work on self driving cars know nothing about self driving cars.
- irq11 9y agowhy would you make assumptions like that?
- chrismcb 9y agoOK, but we developed or mental model of the world through those two cameras. I agree that we still have aways to go, the fact is only two cameras and processing is all that is needed. But we can do better with more sensors
- argonaut 9y ago
- wnevets 9y ago> You can’t afford to miss a single object because that object could be a person. Humans do all the time and we accept that risk, why are self driving cars held to a higher standard?
- c22 9y agoPerhaps because machines can't be held responsible for deaths they've caused. Though labelled 'autonomous' theirs is a somewhat lesser form of autonomy than you or I possess.
- D-Coder 9y agoIt's an argument, IMHO not a very good one. It's basically the same as "but they don't cure cancer!" argument.
- jweir 9y agoReflections. How many self driving cars can see these? Every day that is what I have to look for when leaving a garage - the reflections, or shadows, of people and pets, as I come out of a parking garage. The edge cases for self driving cars are massive - unless we dictate our roads and street to be machine friendly.
- taeric 9y agoPretty sure this is anthropomorphizing too much. Why look for the reflection when you can literally detect around the corner? More, if the system is fully automatic, just regulate the speed so that it can stop in time for events like these.
- oculusthrift 9y agookay, you drive those 10 seconds until you're on the road and it'll do the rest.
- Piskvorrr 9y agoThen you're back to "completely autonomous until OH WAIT CRAP WHAT NOW HUMAN TAKE THE WHEEL [crashes into a white truck across the highway]". Useful? Perhaps. Autonomous? Nope.
- frodeopdahl 9y agoI don't know the exact state of recognition used in the state-of-the-art self-driving cars, but I wouldn't you be able to handle reflections if you use normal camera image recognition combined with LADAR? From a simple image, you can infer the distance to an observed object by looking at the size of its bounding box, and compare that to what the size of a bounding box of that type of object is at some known distance. After that, you can validate this distance using LADAR. If there are large discrepancies, you can either infer that it is a reflection, or you can maybe have trained a system specifically to classify if something is a reflection or not. That system would probably look at how much distortion there is on an object, compared to the rest of the scene, which could be caused by e.g. a non-flat reflective surface. Like I said this is just some thinking out loud on my part, and I am by no means an expert. Does anyone that are more knowledgeable about this topic know if this strategy could be feasable?
- theprop 9y ago1. There are supposedly new lidar systems coming out in the hundreds of dollars. 2. You can be "superhuman" with just image sensors (i.e. without lidar). That said, I believe lidar is effective for precise object determination e.g. whether a small thing on the road is soft or hard which would help determine if it's a stuffed animal or a real one or a plastic bag in the shape of an animal.
- MrQuincle 9y agoWhat is difficult for lidar are the following: * not-reflective / dark surfaces, i.e. http://www.thedrive.com/tech/9293/luminar-says-its-lidar-tech-will-help-self-driving-cars-see-farther http://www.thedrive.com/tech/9293/luminar-says-its-lidar-tec... has 10 percent reflectivity at 50 meters. As stated in the article Luminar is able to do the same at 200 meters, but violating eye safety standards. We might introduce paint restrictions - http://blog.caranddriver.com/why-better-paint-coatings-are-critical-for-autonomous-cars/ http://blog.caranddriver.com/why-better-paint-coatings-are-c... - but it's not really acceptable to paint everything I think. :-) * fog and dust particles (already known since lidar use for aerial imaging). * interference with other cars not so much (much worse with radar). I don't know if soft/hard objects are so easy to recognize for lidar though! That's pretty challenging!
- DonHopkins 9y agoWill a bunch of self driving cars all using LIDAR interfere with each other? Busy intersections could have strategically placed high resolution LIDAR "base stations" that wirelessly transmit the model to cars as they near the intersection.
- nkjoep 9y agoI'd say that using "base stations" or "turrets" like in airports wouldn't make a autonomous-self-driving technology. I guess I prefer a fully independent car capable of driving anywhere without the help 3rd party devices. Ok maybe just gps.
- DonHopkins 9y agoI think the more important goal than being completely autonomous is killing and maiming as few people as possible. Cars depend on a lot of immobile "base stations" like gas and charging stations, traffic lights, drive through fast food restaurants, and the roads themselves. Until cars can refine their own petroleum or generate their own electricity, replicate their own fast food, and pave their own roads [1], they're never going to be purely autonomous. My question is how much do a bunch of cars an intersection all using LIDAR at the same time interfere with each other? Would it make sense for them to wirelessly share their models, and share an even better model made possible by high resolution LIDAR placed on poles like street lights? Then approaching cars could see around corners, and know about otherwise invisible oncoming traffic. I'm not suggesting that every intersection and long stretch of road through the desert should be festooned with LIDAR base stations. But how about starting with every busy intersection on Market street, for example? [1] https://www.youtube.com/watch?v=8hG7-AQfbjE https://www.youtube.com/watch?v=8hG7-AQfbjE
- IgorPartola 9y agoWait you don't want your car to send all the inputs to Amazon or Googlefor processing in their data center, and just her responses back? "We see you are driving by a McDonald's. Your route has been updated to include a burger and fries!"
- deevolution 9y ago
- ge96 9y agoIs there a reason most lidars still use the rotating design and not phased array (faster). - cost - redundancy - depth Just a thought, not a scientist/engineer edit: then imagine it could be made into strips and that goes around the car... no rotating-delay and full-view all the time.
- acomjean 9y agoI walk/drive/bike in a city. I'm wondering if we'll ever get to fully autonomous. The thing is I'm often looking at other drivers and when driving pedestrians look at me. You can send them across a crosswalk you are stopped at in a car with a wave or a tilt of your head. You can tell that person isn't going to cross the street and is waiting for a ride because they are looking at their phone and then cars. Sometimes pedestrians wave me though when I'm in my car. I had a situation where an adult waved me across at an intersection but her kids kept going. I stopped, kids stopped after mom told them to and we were back to square one of our intersection dance. Things that sensors are going to have a hard time doing. I think we'll get driver assisted as sensors can look 360 and further ahead. highways and less dense areas seem ideal for self driving.
- mee_too 9y agoI'm wondering if we'll ever be able to replace horses with cars. The thing is I'm often looking at the horses and they are looking at me :)
- Symbiote 9y agoThat's not such a daft comment. Various old people in England have told me about when goods were delivered by horse and cart (milk, bread etc). The horses knew the route, a man would walk alongside the cart and make the delivery while the horse waited, before proceeding to the next customer's house. Usually, the purpose of telling me this story has been the time the delivery man was absent; either through a sudden illness or drunkeness. The customer notices the horse stop outside, but then goes to investigate when the horse walks away but there's been no delivery.
- rockinghigh 9y agoSelf driving on highways is already working. There are solutions for interacting with periastrians. Mercedes had a prototype where a virtual cross walk was displayed in front of the car to show it was waiting for the pedestrian.
- 9y ago
- acomjean 9y agoI walk/drive/bike in a city. I'm wondering if we'll ever get to fully autonomous. The thing is I'm often looking at other drivers and when driving pedestrians look at me. You can send them across a crosswalk you are stopped at in a car with a wave or a tilt of your head. You can tell that person isn't going to cross the street and is waiting for a ride because they are looking at their phone and then cars. Sometimes pedestrians wave me though when I'm in my car. I had a situation where an adult waved me across at an intersection but her kids kept going. I stopped, kids stopped after mom told them to and we were back to square one of our intersection dance. Things that sensors are going to have a hard time doing. I think we'll get driver assisted as sensors can look 360 and further ahead. highways and less dense areas seem ideal for self driving.
- kleiba 9y agoLidar is pretty much my favorite animal. Bred for its skills in magic.
- cmurf 9y agoFor as much as Americans complain about regulations, there's very little standardization in automobile driving environment (rules of the road, road paint and signage) or in the driver (uncle Joe can teach you how to drive in a few weeks, and essentially once licensed you're licensed for life as long as you pay a fee and don't become legally blind). Anyone who drives in the U.S. and either is a pilot or has driven in another industrialized country knows how non-standard automobile driving is in the U.S. This might make us fairly defensive and adaptive drivers, because our fellow drivers are so unpredictable due to non-standardization. I think that makes the task of integrating autonomous cars a difficult task, not for the human driver, but for the computer. It has to be more adaptive and responsive with a lower error rate than the best American driver, while simultaneously sharing the exact same physical environment with non-deterministic actors. Once we solve the pollution problem, all the HOV lanes can become autonomous car lanes. Only qualified cars use those lanes and they must be in autonomous mode. And conversely in mixed zones they can only be in some kind of hybrid mode. How insurance sorts out liability in the hybrid case will be interesting.
- Piskvorrr 9y agoFor all the non-regulation of US roadspace, I don't see any significantly different driving in Europe: having driven enough on both continents, it feels the same w/r/t unpredictable drivers. IMHO it doesn't matter whether the unpredictability comes from "has no training" or "has training but ignores it".
- omarforgotpwd 9y agoIt sounds to me like they are discounting the possibility that an entirely camera based solution could be developed that bypassed the whole lidar cost issue altogether.